Visual working memory directly alters perception

Perceptual comparisons induce lasting and generalizing changes to face memory reports

Humans are often tasked to remember new faces so that they can recognize the faces later in time. Previous studies found that memory reports for basic visual features (e.g., colors and shapes) are susceptible to systematic distortions as a result of comparison with new visual input, especially when the input is perceived as similar to the memory. The current study tested whether this similarity-induced memory bias (SIMB) would also occur with more complex face stimuli. The results showed that faces that are just perceptually encoded into visual working memory as well as retrieved from visual long-term memory are also susceptible to SIMB. Furthermore, once induced, SIMB persisted over time across cues through which the face memory was accessed for memory report. These results demonstrate the generalizability of SIMB to more complex and practically relevant stimuli, and thus, suggest potential real-world implications.

Centering cognitive neuroscience on task demands and generalization

Cognitive neuroscience seeks generalizable theories explaining the relationship between behavioral, physiological and mental states. In pursuit of such theories, we propose a theoretical and empirical framework that centers on understanding task demands and the mutual constraints they impose on behavior and neural activity. Task demands emerge from the interaction between an agent's sensory impressions, goals and behavior, which jointly shape the activity and structure of the nervous system on multiple spatiotemporal scales. Understanding this interaction requires multitask studies that vary more than one experimental component (for example, stimuli and instructions) combined with dense behavioral and neural sampling and explicit testing for generalization across tasks and data modalities. By centering task demands rather than mental processes that tasks are assumed to engage, this framework paves the way for the discovery of new generalizable concepts unconstrained by existing taxonomies, and moves cognitive neuroscience toward an action-oriented, dynamic and integrated view of the brain.

Working memory expedites the processing of visual signals within the extrastriate cortex

Working memory is the ability to maintain information in the absence of sensory input. In this study, we investigated how working memory benefits processing in visual areas. Using a measure of phase consistency to detect the arrival time of visual signals to the middle temporal (MT) area, we assessed the impact of working memory on the speed of sensory processing. We recorded from MT neurons in two monkeys during a spatial working memory task with visual probes. When the memorized location closely matches the receptive field center of the recording site, visual input arrives sooner, but if the memorized location does not match the receptive field center then the arrival of visual information is delayed. Thus, working memory expedites the arrival of visual input in MT. These results reveal that even in the absence of firing rate changes, working memory can still benefit the processing of information within sensory areas.

Integration of history information Drives Serial Dependence and Stabilizes Working Memory Representations

Serial dependence has shown seemingly contradictory effects on visual perception and working memory. While serial dependence promotes perpetual and mnemonic stability, it biases behavioral reports toward prior information. The neural mechanisms that drive both biasing and adaptive stabilizing effects are not well understood. We proposed and tested a reactivation and integration mechanism that can account for these contradictory effects. We used multivariate pattern analyses of EEG data (26 human participants, 17 females, 9 males) to examine the reactivation of prior reported orientation during the delay period of a visual working memory task. The reactivation strength of prior reports, but not prior sensory items, was predictive of the magnitude of serial dependency biases. These reactivated representations integrated with the representation of the current memory item and improved the ability to decode the current contents of memory. Overall, our data provide convergent evidence suggesting that prior reports in a visual working memory task are reactivated on the subsequent trial and become integrated with current memory representations. This similarity-dependent reactivation mechanism drives both report biasing and stabilization effects attributed to serial dependence in working memory.

Perceptual similarity judgments do not predict the distribution of errors in working memory

Population coding models provide a quantitative account of visual working memory (VWM) retrieval errors with a plausible link to the response characteristics of sensory neurons. Recent work has provided an important new perspective linking population coding to variables of signal detection, including d-prime, and put forward a new hypothesis: that the distribution of recall errors on, for example, a color wheel, is a consequence of the psychological similarity between points in that stimulus space, such that the exponential-like psychophysical distance scaling function can fulfil the role of population tuning and obviate the need to fit a tuning width parameter to recall data. Using four different visual feature spaces, we measured psychophysical similarity and memory errors in the same participants. Our results revealed strong evidence for a common source of variability affecting similarity judgments and recall estimates but did not support any consistent relationship between psychophysical similarity functions and VWM errors. At the group level, the responsiveness functions obtained from the psychophysical similarity task diverged strongly from those that provided the best fit to working memory errors. At the individual level, we found convincing evidence against an association between observed and best-fitting similarity functions. Finally, our results show that the newly proposed exponential-like responsiveness function has in general no advantage over the canonical von Mises (circular normal) function assumed by previous population coding models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Shifting attention between perception and working memory

Most everyday tasks require shifting the focus of attention between sensory signals in the external environment and internal contents in working memory. To date, shifts of attention have been investigated within each domain, but shifts between the external and internal domain remain poorly understood. We developed a combined perception and working-memory task to investigate and compare the consequences of shifting spatial attention within and between domains in the service of a common orientation-reproduction task. Participants were sequentially cued to attend to items either in working memory or to an upcoming sensory stimulation. Stay trials provided a baseline condition, while shift trials required participants to shift their attention to another item within the same or different domain. Validating our experimental approach, we found evidence that participants shifted attention effectively in either domain (Experiment 1). In addition, we observed greater costs when transitioning attention between as compared to within domains (Experiments 1, 2). Strikingly, these costs persisted even when participants were given more time to complete the attentional shift (Experiment 2). Biases in fixational gaze behaviour tracked attentional orienting in both domains, but revealed no latency or magnitude difference for within- versus between-domain shifts (Experiment 1). Collectively, the results from Experiments 1 and 2 suggest that shifting between attentional domains might be regulated by a unique control function. Our results break new ground for exploring the ubiquitous act of shifting attention between perception and working memory to guide adaptive behaviour in everyday cognition.

Mapping visual working memory models to a theoretical framework

The body of research on visual working memory (VWM)-the system often described as a limited memory store of visual information in service of ongoing tasks-is growing rapidly. The discovery of numerous related phenomena, and the many subtly different definitions of working memory, signify a challenge to maintain a coherent theoretical framework to discuss concepts, compare models and design studies. A lack of robust theory development has been a noteworthy concern in the psychological sciences, thought to be a precursor to the reproducibility crisis (Oberauer & Lewandowsky, Psychonomic Bulletin & Review, 26, 1596-1618, 2019). I review the theoretical landscape of the VWM field by examining two prominent debates-whether VWM is object-based or feature-based, and whether discrete-slots or variable-precision best describe VWM limits. I share my concerns about the dualistic nature of these debates and the lack of clear model specification that prevents fully determined empirical tests. In hopes of promoting theory development, I provide a working theory map by using the broadly encompassing memory for latent representations model (Hedayati et al., Nature Human Behaviour, 6, 5, 2022) as a scaffold for relevant phenomena and current theories. I illustrate how opposing viewpoints can be brought into accordance, situating leading models of VWM to better identify their differences and improve their comparison. The hope is that the theory map will help VWM researchers get on the same page-clarifying hidden intuitions and aligning varying definitions-and become a useful device for meaningful discussions, development of models, and definitive empirical tests of theories.

Representational Dissimilarity of Faces and Places during a Working Memory Task is Associated with Subsequent Recognition Memory during Development

Nearly 50 years of research has focused on faces as a special visual category, especially during development. Yet it remains unclear how spatial patterns of neural similarity of faces and places relate to how information processing supports subsequent recognition of items from these categories. The current study uses representational similarity analysis and functional imaging data from 9- and 10-year-old youth during an emotional n-back task from the Adolescent Brain and Cognitive Development Study 3.0 data release to relate spatial patterns of neural similarity during working memory to subsequent out-of-scanner performance on a recognition memory task. Specifically, we examine how similarities in representations within face categories (neutral, happy, and fearful faces) and representations between visual categories (faces and places) relate to subsequent recognition memory of these visual categories. Although working memory performance was higher for faces than places, subsequent recognition memory was greater for places than faces. Representational similarity analysis revealed category-specific patterns in face-and place-sensitive brain regions (fusiform gyrus, parahippocampal gyrus) compared with a nonsensitive visual region (pericalcarine cortex). Similarity within face categories and dissimilarity between face and place categories in the parahippocampus was related to better recognition of places from the n-back task. Conversely, in the fusiform, similarity within face categories and their relative dissimilarity from places was associated with better recognition of new faces, but not old faces. These findings highlight how the representational distinctiveness of visual categories influence what information is subsequently prioritized in recognition memory during development.

Working memory signals in early visual cortex are present in weak and strong imagers

It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.

New Evidence for Retrospectively Cued Perception

Past research suggests a continuity between perception and memory, as reflected in influences of orienting of spatial attention by cues presented after a visual target offset (post-target cues) on target perception. Conducting two experiments, we tested and confirmed this claim. Our study revealed an elevated reliance on post-target cues for target detection with diminishing target visibility, leading to better performance in validly versus invalidly cued trials, indicative of contrast gain. We demonstrated this post-target cueing impact on target perception without a postcue response prompt, meaning that our results truly reflected a continuity between perception and memory rather than a task-specific impact of having to memorize the target due to a response prompt. While previous studies found an improvement in accuracy through valid compared to invalid cues using liminal targets, in Experiment 1, we further showed an influence of attention on participants' response time by the post-target cues with cues presented away from a clearly visible target. This suggests that visual interactions at the target location provided no better explanation of post-target cueing effects. Our results generalize prior research with liminal targets and confirm the view of a perception-memory continuum so that visual target processing is not shielded against visuospatial orienting of attention elicited by events following the offset of the visual target.

Geometry of visuospatial working memory information in miniature gaze patterns

Stimulus-dependent eye movements have been recognized as a potential confound in decoding visual working memory information from neural signals. Here we combined eye-tracking with representational geometry analyses to uncover the information in miniature gaze patterns while participants (n = 41) were cued to maintain visual object orientations. Although participants were discouraged from breaking fixation by means of real-time feedback, small gaze shifts (<1°) robustly encoded the to-be-maintained stimulus orientation, with evidence for encoding two sequentially presented orientations at the same time. The orientation encoding on stimulus presentation was object-specific, but it changed to a more object-independent format during cued maintenance, particularly when attention had been temporarily withdrawn from the memorandum. Finally, categorical reporting biases increased after unattended storage, with indications of biased gaze geometries already emerging during the maintenance periods before behavioural reporting. These findings disclose a wealth of information in gaze patterns during visuospatial working memory and indicate systematic changes in representational format when memory contents have been unattended.

Behavioral fluctuation reflecting theta-rhythmic activation of sequential working memory

Sequential working memory, the ability to actively maintain sequential information, is essential for human cognition. The neural representation of each item in sequential working memory is thought to be activated rhythmically within the theta (3-7 Hz) range of human electrophysiology. In the current study, we predicted that if neural representations of sequential working memory items were truly activated rhythmically, periodic fluctuations in behavior would be evident. That is, the ease and speed of recalling each memory item would oscillate depending on the interval between memory encoding and recall, affected by the rhythmic neural representation. We conducted detailed analyses of reaction times for retrieving sequential and non-sequential information in eight experiments (total n = 125). The results revealed that reaction times for recalling sequential information showed fluctuation in the theta range as a function of the interval between memory encoding and recall, which was significantly stronger than that observed when the task did not require participants to remember the sequential order. Taken together, the current findings revealed that participants' behavior exhibited theta-rhythmic fluctuation when recalling sequential information in a relatively large sample, supporting theta phase-dependent coding of sequential working memory.

Examining the role of attention during feature binding in visuospatial working memory

The current experiments address the enduring debate regarding the role of attention in feature binding in visuospatial working memory by considering the nature of the to-be-bound features, i.e., whether they are intrinsic (integrated within the object, such as its color and shape) or extrinsic (not part of the object, such as its spatial location). Specifically, arrays of different-colored shapes in different locations were followed by probed recall: One feature of the probed object prompted recall of one of its remaining two features (e.g., a shape probe prompts recall of color, with the probe displayed at the center of the screen (i.e., without spatial information)) to test the retention of intrinsic (shape, color) and extrinsic (location) features. During the retention interval, we manipulated attention via disruption (Experiment 1) and retro-cues (Experiment 2) to determine their impacts on binding errors, as estimated from a three-parameter mixture model fit to recall error (i.e., the distance between the target and response). Disrupting central versus peripheral attention in Experiment 1 did not respectively increase extrinsic and intrinsic binding errors as predicted, but disrupting central attention reduced target memory of the extrinsic feature relative to a no-disruption baseline. Guiding attention via extrinsic and intrinsic retro-cues in Experiment 2 did not respectively reduce extrinsic and intrinsic binding errors as predicted, but we observed retro-cue benefits to target memory that did not distinguish between extrinsic and intrinsic features. Thus, this work highlights that attentional resources aid target memory, with no consistent distinction between intrinsic and extrinsic features.

Bend but don't break: Prioritization protects working memory from displacement but leaves it vulnerable to distortion from distraction

Perceptual distraction distorts visual working memory representations. Previous research has shown that memory responses are systematically biased towards passively viewed visual distractors that are similar to the memoranda. However, it remains unclear whether the prioritization of one working memory representation over another reduces the impact of perceptual distractors. We designed a study with five different types of visual distraction that varied in engagement and found evidence for both subtle distortions and catastrophic failures of memory. Importantly, prioritization protected working memories from catastrophic loss (fewer "swap errors") but rendered them more vulnerable to distortion (greater attractive "biases" towards the distractor). Our findings demonstrate that prioritization does not simply protect working memory from any and all interference, but rather it reduces the likelihood of catastrophic disruption from perceptual distraction at the cost of an increased likelihood of distortion.

Assessing the interaction between working memory and perception through time

Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants' ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.

Perception is iconic, perceptual working memory is discursive

The evidence that the target article cites for language-of-thought (LoT) structure in perceptual object representations concerns perceptual working memory, not perception. Perception is iconic, not structured like an LoT. Perceptual working memory representations contain the remnants of iconic perceptual representations, often recoded, in a discursive envelope.

Gaze Restriction and Reactivation of Place-bound Content Drive Eye Movements During Mental Imagery

When we imagine a picture, we move our eyes even though the picture is physically not present. These eye movements provide information about the ongoing process of mental imagery. Eye movements unfold over time, and previous research has shown that the temporal gaze dynamics of eye movements in mental imagery have unique properties, which are unrelated to those in perception. In mental imagery, refixations of previously fixated locations happen more often and in a more systematic manner than in perception. The origin of these unique properties remains unclear. We tested how the temporal structure of eye movements is influenced by the complexity of the mental image. Participants briefly saw and then maintained a pattern stimulus, consisting of one (easy condition) to four black segments (most difficult condition). When maintaining a simple pattern in imagery, participants restricted their gaze to a narrow area, and for more complex stimuli, eye movements were more spread out to distant areas. At the same time, fewer refixations were made in imagery when the stimuli were complex. The results show that refixations depend on the imagined content. While fixations of stimulus-related areas reflect the so-called 'looking at nothing' effect, gaze restriction emphasizes differences between mental imagery and perception.

Familiarity enhances mnemonic precision but impairs mnemonic accuracy in visual working memory

Prior stimulus familiarity has a variety of effects on visual working memory representations and processes. However, it is still unclear how familiarity interacts with the veridical correspondence between mnemonic representation and external stimuli. Here, we examined the effect of familiarity on two aspects of mnemonic correspondence, precision and accuracy, in visual working memory. Specifically, we used a hierarchical Bayesian method to model task performance in a change detection task with celebrity lookalikes (morphed faces between celebrities and noncelebrities with various ratios) as the memory stimuli. We found that familiarity improves memory precision by sharpening mnemonic representation but impairs memory accuracy by biasing mnemonic representation toward familiar faces (i.e., celebrity faces). These findings provide an integrated account of the puzzling celebrity sighting phenomena with the dissociable effects on mnemonic imprecision and bias and further highlight the importance of assessing these two aspects of memory correspondence in future research.

Perceptual comparisons modulate memory biases induced by new visual inputs

It is well-established that stimulus-specific information in visual working memory (VWM) can be systematically biased by new perceptual inputs. These memory biases are commonly attributed to interference that arises when perceptual inputs are physically similar to VWM contents. However, recent work has suggested that explicitly comparing the similarity between VWM contents and new perceptual inputs modulates the size of memory biases above and beyond stimulus-driven effects. Here, we sought to directly investigate this modulation hypothesis by comparing the size of memory biases following explicit comparisons to those induced when new perceptual inputs are ignored (Experiment 1) or maintained in VWM alongside target information (Experiment 2). We found that VWM reports showed larger attraction biases following explicit perceptual comparisons than when new perceptual inputs were ignored or maintained in VWM. An analysis of participants' perceptual comparisons revealed that memory biases were amplified after perceptual inputs were endorsed as similar-but not dissimilar-to one's VWM representation. These patterns were found to persist even after accounting for variability in the physical similarity between the target and perceptual stimuli across trials, as well as the baseline memory precision between the distinct task demands. Together, these findings illustrate a causal role of perceptual comparisons in modulating naturally-occurring memory biases.

Interference between items stored for distinct tasks in visual working memory

The action perspective on working memory suggests that memory representations are coded according to their specific temporal and behavioral task demands. This stands in contrast to theories that assume representations are stored in a task-agnostic format within a "common workspace". Here, we tested whether visual items that are memorized for different tasks are stored separately from one another or show evidence of inter-item interference during concurrent maintenance, indicating a common storage. In two experiments, we combined a framing memory task (memorize a motion direction for continuous direction report) with an embedded memory task (memorize a motion direction for a binary direction discrimination) that was placed within the retention period of the framing task. Even though the temporal and action demands were item specific, we observed two types of interference effects between the items: The embedded motion direction was (1) repulsed away and (2) degraded in precision by the motion direction of the item in the framing task. Repulsion and precision degradation increased with item similarity when both items were concurrently held in working memory. In contrast, perceptual and iconic memory control conditions revealed weaker repulsion overall and no interference effect on precision during the stimulus processing stages prior to working memory consolidation. Thus, additional inter-item interference arose uniquely within working memory. Together, our results present evidence that items that are stored for distinct tasks to be performed at distinct points in time, reside in a common workspace in working memory.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

Voluntary self-initiation of the stimuli onset improves working memory and accelerates visual and attentional processing

The ability of an organism to voluntarily control the stimuli onset modulates perceptual and attentional functions. Since stimulus encoding is an essential component of working memory (WM), we conjectured that controlling the initiation of the perceptual process would positively modulate WM. To corroborate this proposition, we tested twenty-five healthy subjects in a modified-Sternberg WM task under three stimuli presentation conditions: an automatic presentation of the stimuli, a self-initiated presentation of the stimuli (through a button press), and a self-initiated presentation with random-delay stimuli onset. Concurrently, we recorded the subjects' electroencephalographic signals during WM encoding. We found that the self-initiated condition was associated with better WM accuracy, and earlier latencies of N1, P2 and P3 evoked potential components representing visual, attentional and mental review of the stimuli processes, respectively. Our work demonstrates that self-initiated stimuli enhance WM performance and accelerate early visual and attentional processes deployed during WM encoding. We also found that self-initiated stimuli correlate with an increased attentional state compared to the other two conditions, suggesting a role for temporal stimuli predictability. Our study remarks on the relevance of self-control of the stimuli onset in sensory, attentional and memory updating processing for WM.

Flexible top-down control in the interaction between working memory and perception

Successful goal-directed behavior often requires continuous sensory processing while simultaneously maintaining task-related information in working memory (WM). Although WM and perception are known to interact, little is known about how their interactions are controlled. Here, we tested the hypothesis that WM perception interactions engage two distinct modes of control – proactive and reactive – in a manner similar to classic conflict-adaptation tasks (e.g. Stroop, flanker, and Simon). Participants performed a delayed recall-of-orientation WM task, plus a standalone visual discrimination-of-orientation task the occurred during the delay period, and with the congruity in orientation between the tasks manipulated. Proactive control was seen in the sensitivity of task performance to the previous trial's congruity (i.e. a Gratton effect). Reactive control was observed in a repulsive serial-dependence produced by incongruent discriminanda. Quantitatively, these effects were explained by parameters from a reinforcement learning-based model that tracks trial-to-trial fluctuations in control demand: reactive control by a phasic control prediction error (control PE), and proactive control by a tonic level of predicted conflict updated each trial by the control PE. Thus, WM-perception interactions may be controlled by the same mechanisms that govern conflict in other domains of cognition, such as response selection.

Cyclic reactivation of distinct feature dimensions in human visual working memory

Several recent behavioral studies have observed 4-10 Hz rhythmic fluctuations in attention-related performance over time. So far, this rhythmic attentional sampling has predominantly been demonstrated with regards to external visual attention, directed toward one single feature dimension. Whether and how attention might sample from concurrent internal representations of different feature dimensions held in working memory (WM) is currently largely unknown. To elucidate this issue, we conducted a human behavioral dense-sampling experiment, in which participants had to hold representations of two distinct feature dimensions (color and orientation) in WM. By querying the contents of WM at 72 time-points after encoding, we estimated the activity time course of the individual feature representations. Our results demonstrate an oscillatory component at 9.4 Hz in the joint time courses of both representations, presumably reflecting a common early perceptual sampling process in the alpha-frequency range. Furthermore, we observed an oscillatory component at 3.5 Hz in the time course difference between the two representations. This likely corresponds to a later attentional sampling process and indicates that internal representations of distinct features are activated in alteration. In summary, we demonstrate the cyclic reactivation of internal WM representations of distinct feature dimensions, as well as the co-occurrence of behavioral fluctuations at distinct frequencies, presumably associated to internal perceptual- and attentional rhythms. In addition, our findings also challenge a model of strict parallel processing in visual search, thus, providing novel input to the ongoing debate on whether search for more than one target feature constitutes a parallel- or a sequential mechanism.

Attentional Prioritization of Complex, Naturalistic Stimuli Maintained in Working-Memory-A Dot-Probe Event-Related Potentials Study

Recent studies suggest that a stimulus actively maintained in working memory (WM) automatically captures visual attention when subsequently perceived. Such a WM-based guidance effect has been consistently observed for stimuli defined by simple features, such as color or orientation, but studies using more complex stimuli provided inconclusive results. Therefore, we investigated whether the WM-based guidance effect occurs also for naturalistic stimuli, whose identity is defined by multiple features and relations among them, specifically for faces and houses. The experiment consisted of multiple blocks in which participants (N = 28) either memorized or merely saw (WM or exposure condition, respectively) a template stimulus and then performed several dot-probe trials, with pairs of stimuli (template and control) presented laterally as distractors and followed by a target-asterisk. Analysis of reaction-times (RT) in the dot-probe task shows that memorized stimuli were prioritized by attention and points toward attention-hold, rather than capture, as a mechanism of attentional prioritization. Consistent with this interpretation, memorized items did not evoke a lateralized N2pc ERP component, thought to indicate attention shifts. However, in an exploratory ERP analysis we found evidence for a very early (100-200 ms post-stimulus) prioritization specific to the memorized faces, which is in line with the sensory recruitment theory of WM. In conclusion, our data provide evidence that complex stimuli are prioritized by attention when maintained in WM, and that the mechanism of such prioritization is based on a prolonged hold of spatial attention.

Working Memory Content Is Distorted by Its Use in Perceptual Comparisons

Visual information around us is rarely static. To perform a task in such a dynamic environment, we often have to compare current visual input with our working memory (WM) representation of the immediate past. However, little is known about what happens to a WM representation when it is compared with perceptual input. To test this, we asked young adults (N = 170 total in three experiments) to compare a new visual input with a WM representation prior to reporting the WM representation. We found that the perceptual comparison biased the WM report, especially when the input was subjectively similar to the WM representation. Furthermore, using computational modeling and individual-differences analyses, we found that this similarity-induced memory bias was driven by representational integration, rather than incidental confusion, between the WM representation and subjectively similar input. Together, our findings highlight a novel source of WM distortion and suggest a general mechanism that determines how WM interacts with new visual input.

Understanding occipital and parietal contributions to visual working memory: Commentary on Xu (2020)

In her commentary, Xu (2020) admonishes the reader that "To have a full understanding of the cognitive mechanisms underlying VWM [visual working memory], both behavioral and neural evidence needs to be taken into account. This is a must, and not a choice, for any study that attempts to capture the nature of VWM" (p. 11). Although we don't disagree with this statement, our overall assessment of this commentary is that it, itself, fails to satisfy several "musts" and, consequently, does not pose a serious challenge for the sensory recruitment framework for understanding visual working memory. These "musts" include accurately characterizing the framework being critiqued, not favoring verbal models and intuition at the expense of formal quantitative models, and providing even-handed interpretation of the work of others. We'll conclude with a summary of how the sensory recruitment framework can be incorporated into a broader working model of visual working memory.

Visuospatial working memory abilities and spontaneous sensations perception

Aim: Body awareness arises when attending to and maintaining awareness of visuospatial body representations. By the same token, focussing on representations transfers them to working memory. Body awareness and working memory seemingly rely on similar processes and recruit common parietal areas involved in perception. Therefore, we asked whether visuospatial working memory abilities would define individual differences in the perception of spontaneous sensations (SPS), i.e., bodily sensations perceived in the absence of triggers (e.g., tactile stimulation or movement), when attending to the body.Method: Participants completed two visuospatial working memory tasks to assess various mechanisms: (i) the decay of representations was assessed through a Brown-Peterson task in which the delay between the memorandum presentation and its recall was manipulated, and (ii) the impact of distractors' interference and cognitive load (i.e., complexity) on recall performances were assessed through a complex span task that required the processing of distractors while maintaining a memorandum. A standard SPS task involving localization and characterization of SPS perceived on the hands was completed afterwards.Results: Low performance due to decay, distractors' interference and cognitive load in visuospatial working memory was associated with a decrease in the frequency of SPS. Additionally, low performance due to distractors' cognitive load predicted a decrease in the perception of surface-type sensations, and high performance despite distractors' interference led to a better perception of SPS on less sensitive areas of the hand.Conclusion: We discuss how visuospatial working memory processes might contribute to body awareness and perceptual distortions of the body.

Investigating Object Files in Spatial Cueing

In spatial cueing, cues presented at target position (valid condition) can capture visual attention and facilitate responses to the target relative to cues presented away from target position (invalid condition). If cues and targets carry different features, the necessary updating of the object representation from the cue to the target display sometimes counteracts and even reverses facilitation in valid conditions, resulting in an inverted validity effect. Previous studies reached partly divergent conclusions regarding the conditions under which object-file updating occurs, and little is known about the exact nature of the processes involved. Object-file updating has so far been investigated by manipulating cue-target similarities in task-relevant target features, but other features that change between the cue and target displays might also contribute to object-file updating. This study examined the conditions under which object-file updating could counteract validity effects by systematically varying task-relevant (color), response-relevant (identity), and response-irrelevant (orientation) features between cue and target displays. The results illustrate that object-file updating is largely restricted to task-relevant features. In addition, the difficulty of the search task affects the degree to which object-file updating costs interact with spatial cueing.

Irrelevant features of distractors in intervening visual search tasks cause active visual working memory interference - the more difficult the search task, the more interference it causes

Visual working memory (VWM) content disrupts visual search performance when there is a singleton in the search array that is similar to the content in VWM, even when this singleton is task irrelevant. Typically, the memory-similar singleton captures attention, which results in slower search performance for memory-similar conditions compared to conditions where memory-similar content is absent. Recently, it has also been shown that VWM content may be affected when memory-similar stimuli are processed. Specifically, it appears that VWM representations bias toward memory-similar information that is processed but not memory-dissimilar information. Here, we test whether the bias caused by processing memory-similar information is an active interference process (growing with engagement with the memory-similar stimuli) or a passive interference process (indifferent to the engagement with memory-similar stimuli). To test this, observers were tasked with memorizing a single color followed by a search task. The search task was either easy or difficult, and the search items could either be memory-similar or memory-dissimilar. Critically, the target in the search task was defined by its shape, so the color of the search items was irrelevant to the search task. At the end of each trial, participants reported the color in memory using a continuous report color wheel. The results showed that VWM representations drifted towards the irrelevant color of the search items in the memory-similar conditions, and this effect was larger in the difficult search condition. The results provide evidence that VWM representations receive active interference from processing memory-similar stimuli.

Adaptive Repulsion of Long-Term Memory Representations Is Triggered by Event Similarity

We tested whether similarity between events triggers adaptive biases in how those events are remembered. We generated pairs of competing objects that were identical except in color and varied the degree of color similarity for the competing objects. Subjects (N = 123 across four experiments) repeatedly studied and were tested on associations between each of these objects and corresponding faces. As expected, high color similarity between competing objects created memory interference for object-face associations. Strikingly, high color similarity also resulted in a systematic bias in how the objects themselves were remembered: Competing objects with highly similar colors were remembered as being further apart (in color space) than they actually were. This repulsion of color memories increased with learning and served a clear adaptive purpose: Greater repulsion was associated with lower associative-memory interference. These findings reveal that similarity between events triggers adaptive-memory distortions that minimize interference.

Concurrent visual working memory bias in sequential integration of approximate number

Previous work has shown bidirectional crosstalk between Working Memory (WM) and perception such that the contents of WM can alter concurrent percepts and vice versa. Here, we examine WM-perception interactions in a new task setting. Participants judged the proportion of colored dots in a stream of visual displays while concurrently holding location- and color information in memory. Spatiotemporally resolved psychometrics disclosed a modulation of perceptual sensitivity consistent with a bias of visual spatial attention towards the memorized location. However, this effect was short-lived, suggesting that the visuospatial WM information was rapidly deprioritized during processing of new perceptual information. Independently, we observed robust bidirectional biases of categorical color judgments, in that perceptual decisions and mnemonic reports were attracted to each other. These biases occurred without reductions in overall perceptual sensitivity compared to control conditions without a concurrent WM load. The results conceptually replicate and extend previous findings in visual search and suggest that crosstalk between WM and perception can arise at multiple levels, from sensory-perceptual to decisional processing.

Distraction in Visual Working Memory: Resistance is Not Futile

Over half a century of research focused on understanding how working memory is capacity constrained has overshadowed the fact that it is also remarkably resistant to interference. Protecting goal-relevant information from distraction is a cornerstone of cognitive function that involves a multifaceted collection of control processes and storage mechanisms. Here, we discuss recent advances in cognitive psychology and neuroscience that have produced new insights into the nature of visual working memory and its ability to resist distraction. We propose that distraction resistance should be an explicit component in any model of working memory and that understanding its behavioral and neural correlates is essential for building a comprehensive understanding of real-world memory function.

Psychophysical dual-task setups do not measure pre-saccadic attention but saccade-related strengthening of sensory representations

Visual attention and saccadic eye movements are linked in a tight, yet flexible fashion. In humans, this link is typically studied with dual‐task setups. Participants are instructed to execute a saccade to some target location, while a discrimination target is flashed on a screen before the saccade can be made. Participants are also instructed to report a specific feature of this discrimination target at the trial end. Discrimination performance is usually better if the discrimination target occurred at the same location as the saccade target compared to when it occurred at a different location, which is explained by the mandatory shift of attention to the saccade target location before saccade onset. This pre‐saccadic shift of attention presumably enhances the perception of the discrimination target if it occurred at the same, but not if it occurred at a different location. It is, however, known that a dual‐task setup can alter the primary process under investigation. Here, we directly compared pre‐saccadic attention in single‐task versus dual‐task setups using concurrent electroencephalography (EEG) and eye‐tracking. Our results corroborate the idea of a pre‐saccadic shift of attention. They, however, question that this shift leads to the same‐position discrimination advantage. The relation of saccade and discrimination target position affected the EEG signal only after saccade onset. Our results, thus, favor an alternative explanation based on the role of saccades for the consolidation of sensory and short‐term memory. We conclude that studies with dual‐task setups arrived at a valid conclusion despite not measuring exactly what they intended to measure.

In humans, the relation between visual attention and saccadic eye movements is usually studied with psychophysical dual‐task setups. Here, we employ concurrent EEG and eye‐tracking to directly compare dual‐task to single‐task setups and conclude in line with previous research that attention precedes saccades. However, our results suggest that dual‐task setups do not measure what they are supposed to measure, that is, the pre‐saccadic shift of attention.

Eye movements during visual imagery and perception show spatial correspondence but have unique temporal signatures

Eye fixation patterns during mental imagery are similar to those during perception of the same picture, suggesting that oculomotor mechanisms play a role in mental imagery (i.e., the "looking at nothing" effect). Previous research has focused on the spatial similarities of eye movements during perception and mental imagery. The primary aim of this study was to assess whether the spatial similarity translates to the temporal domain. We used recurrence quantification analysis (RQA) to assess the temporal structure of eye fixations in visual perception and mental imagery and we compared the temporal as well as the spatial characteristics in mental imagery with perception by means of Bayesian hierarchical regression models. We further investigated how person and picture-specific characteristics contribute to eye movement behavior in mental imagery. Working memory capacity and mental imagery abilities were assessed to either predict gaze dynamics in visual imagery or to moderate a possible correspondence between spatial or temporal gaze dynamics in perception and mental imagery. We were able to show the spatial similarity of fixations between visual perception and imagery and we provide first evidence for its moderation by working memory capacity. Interestingly, the temporal gaze dynamics in mental imagery were unrelated to those in perception and their variance between participants was not explained by variance in visuo-spatial working memory capacity or vividness of mental images. The semantic content of the imagined pictures was the only meaningful predictor of temporal gaze dynamics. The spatial correspondence reflects shared spatial structure of mental images and perceived pictures, while the unique temporal gaze behavior could be driven by generation, maintenance and protection processes specific to visual imagery. The unique temporal gaze dynamics offer a window to new insights into the genuine process of mental imagery independent of its similarity to perception.

The state of memory-matched distractor in working memory influence the visual attention

Information in working memory (WM) can guide visual attention towards matched features. While recent work has suggested that cognitive control can act upon WM guidance of visual attention, little is known about how the state of memorized items retaining in WM contribute to its influence over attention. Here, we disentangle the role of inhibition and maintenance on WM-guided attention with a novel delayed match-to-sample dual-task. The results showed that active inhibition facilitated searching by diminishing sensory processing and deterring attentional guidance, indexed by an attenuated P1 amplitude and unaffected N2pc amplitude, respectively. By contrast, active maintenance impaired searching by attentional guidance while sensory processing remained unimpaired, indexed by an enhanced N2pc amplitude and unchanged P1 amplitude, respectively. Furthermore, multivariate pattern analyses could sucessfully decode maintenance and inhibition, suggesting that two states differed in modulating visual attention. We propose that remembered contents may play an anchoring role for attentional guidance, and the state of those contents retaining in WM may directly influence the shifting of attention. The maintenance could guide attention by accessing input information, while the inhibition could deter the shifting of attention by suppressing sensory processing. These findings provide a possible reinterpretation of the influence of WM on attention.

Revisit once more the sensory storage account of visual working memory

Recent work has highlighted the role of early visual areas in visual working memory (VWM) storage and put forward a sensory storage account of VWM. Using a distractor interference paradigm, however, we previolsy showed that the contribution of early visual areas to VWM storage may not be essential. Instead, higher cortical regions such as the posterior parietal cortex may play a more significant role in VWM storage. This is consistent with reviews of other available behavioral, neuroimaging and neurophysiology results. Recently, a number of studies brought forward new evidence regarding this debate. Here I review these new pieces of evidence in detail and show that there is still no strong and definitive evidence supporting an essential role of the early visual areas in VWM storage. Instead, converging evidence suggests that early visual areas may contribute to the decision stage of a VWM task by facilitating target and probe comparison. Aside from further clarifying this debate, it is also important to note that whether or not VWM storage uses a sensory code depends on how it is defined, and that behavioral interactions between VWM and perception tasks do not necessarily support the involvement of sensory regions in VWM storage.

Memory integration into visual perception in infancy, childhood, and adulthood

We compared the influence of prior knowledge on visual perception in infants, children, and adults in order to explore the developmental trajectory by which prior knowledge is integrated with new sensory input. Using an identical task across age groups, we tested how participants' accumulated experience affected their ability to judge the relative saturation levels within a pair of sequentially-presented stimuli. We found that infants and children, relative to adults, showed greater influence of the current observation and reduced influence of memory in their perception. In fact, infants and children outperformed adults in discriminating between different levels of saturation, and their performance was less biased by previously-experienced exemplars. Thus, the development of perceptual integration of memory leads to less precise discrimination in the moment, but allows observers to make use of their prior experience in interpreting a complex sensory environment.

No evidence for mnemonic modulation of interocularly suppressed visual input

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Precision, binding, and the hippocampus: Precisely what are we talking about?

Endel Tulving's proposal that episodic memory is distinct from other memory systems like semantic memory remains an extremely influential idea in cognitive neuroscience research. As originally suggested by Tulving, episodic memory involves three key components that differentiate it from all other memory systems: spatiotemporal binding, mental time travel, and autonoetic consciousness. Here, we focus on the idea of spatiotemporal binding in episodic memory and, in particular, how consideration of the precision of spatiotemporal context helps expand our understanding of episodic memory. Precision also helps shed light on another key issue in cognitive neuroscience, the role of the hippocampus outside of episodic memory in perception, attention, and working memory. By considering precision alongside item-context bindings, we attempt to shed new light on both the nature of how we represent context and what roles the hippocampus plays in episodic memory and beyond.