Distortions in recall from visual memory: two classes of attractors at work

Priming effect of individual similarity and ensemble perception in visual search and working memory

Perceptual priming is a well-known phenomenon showing that the repetition of an object's feature can facilitate subsequent detection of that item. Although the priming effect has been rigorously studied in visual search, less is known about its effect on working memory and it is unclear whether the repetition of similar features, and furthermore, ensemble perception created by a large set of similar features, can induce priming. In this study, we investigated the priming effects of individual similarity and ensemble perception in visual search and visual working memory (VWM). We replicated the classic perceptual priming effect (Experiment 1a) and found that visual search was enhanced when the current target had a similar color to the previous target (Experiment 1b), but not when the similar color had been shown as a distractor before (Experiment 1c). However, if the target and distractors of similar colors formed ensemble perception, the search efficiency was again promoted even when the current target shared the same color with the previous distractor (Experiment 1d). For VWM, repeating the ensembles of the target- and nontarget-color subsets did not significantly affect the memory capacity, while switching the two harmed the memory fidelity but not capacity (Experiment 2). We suggest different underlying mechanisms for priming in visual search and VWM: in the former, the perception history of individual similarity and stimuli ensemble exert their effects on through the priority map, by forming a gradient distribution of attentional weights that peak at the previous target feature and diminish as stimulus diverges from the previously selected one; while in the latter, perception history of memory ensemble may influence the deployment of existing memory resources across trials, thereby affecting the memory fidelity but not its capacity.

Memory reports are biased by all relevant contents of working memory

Sensory input is inherently noisy while the world is inherently predictable. When multiple observations of the same object are available, integration of the available information necessarily increases the reliability of a world estimate. Optimal integration of multiple instances of sensory evidence has already been demonstrated during multisensory perception but could benefit unimodal perception as well. In the present study 330 participants observed a sequence of four orientations and were cued to report one of them. Reports were biased by all simultaneously memorized items that were similar and relevant to the target item, weighted by their reliability (signal-to-noise ratio). Orientations presented before and presented after the target biased report, demonstrating that the bias emerges in memory and not (exclusively) during perception or encoding. Only attended, task-relevant items biased report. We suggest that these results reflect how the visual system integrates information that is sampled from the same object at consecutive timepoints to promote perceptual stability and behavioural effectiveness in a dynamic world. We suggest that similar response biases, such as serial dependence, might be instances of a more general mechanism of working memory averaging. Data is available at https://osf.io/embcf/ .

Familiarity enhances mnemonic precision but impairs mnemonic accuracy in visual working memory

Prior stimulus familiarity has a variety of effects on visual working memory representations and processes. However, it is still unclear how familiarity interacts with the veridical correspondence between mnemonic representation and external stimuli. Here, we examined the effect of familiarity on two aspects of mnemonic correspondence, precision and accuracy, in visual working memory. Specifically, we used a hierarchical Bayesian method to model task performance in a change detection task with celebrity lookalikes (morphed faces between celebrities and noncelebrities with various ratios) as the memory stimuli. We found that familiarity improves memory precision by sharpening mnemonic representation but impairs memory accuracy by biasing mnemonic representation toward familiar faces (i.e., celebrity faces). These findings provide an integrated account of the puzzling celebrity sighting phenomena with the dissociable effects on mnemonic imprecision and bias and further highlight the importance of assessing these two aspects of memory correspondence in future research.

Independent short- and long-term dependencies in perception

Perception is biased by stimulus history. Both long-term effects such as the central-tendency bias (CTB) and short-term effects such as serial dependence (SD) have been described, but research into the two has remained largely separate. The sources of these effects, however, are highly correlated in stimulus statistics, which can result in a misinterpretation of experimental data. We compared CTB and SD in the perception of color and line length. Observers judged the relative hue or length of consecutive stimuli in a delayed-matching task. Two interstimulus intervals were used to investigate whether elapsed time or the number of stimulus occurrences was more important for SD. We estimated biases by fitting psychometric functions to the data split based on the history features, and we also fit generalized linear mixed models with either CTB, SD, or both included as regressors. We found biases to both recent stimulus history and the cumulative average of stimulus values for both color and line length judgments. The strength and pattern of each of the biases depended on whether all sources of bias were included in the analysis. Within the range of interstimulus intervals tested, the number of intervening stimuli was more important than elapsed time for SD. We conclude that both SD and CTB independently affect perceptual judgments, and that one effect is not an artifact caused by the other. Failing to consider both effects in data analysis can give an erroneous picture of the phenomenon under study.

Multiple and Dissociable Effects of Sensory History on Working-Memory Performance

Behavioral reports of sensory information are biased by stimulus history. The nature and direction of such serial-dependence biases can differ between experimental settings; both attractive and repulsive biases toward previous stimuli have been observed. How and when these biases arise in the human brain remains largely unexplored. They could occur either via a change in sensory processing itself and/or during postperceptual processes such as maintenance or decision-making. To address this, we tested 20 participants (11 female) and analyzed behavioral and magnetoencephalographic (MEG) data from a working-memory task in which participants were sequentially presented with two randomly oriented gratings, one of which was cued for recall at the end of the trial. Behavioral responses showed evidence for two distinct biases: (1) a within-trial repulsive bias away from the previously encoded orientation on the same trial, and (2) a between-trial attractive bias toward the task-relevant orientation on the previous trial. Multivariate classification of stimulus orientation revealed that neural representations during stimulus encoding were biased away from the previous grating orientation, regardless of whether we considered the within-trial or between-trial prior orientation, despite opposite effects on behavior. These results suggest that repulsive biases occur at the level of sensory processing and can be overridden at postperceptual stages to result in attractive biases in behavior.SIGNIFICANCE STATEMENT Recent experience biases behavioral reports of sensory information, possibly capitalizing on the temporal regularity in our environment. It is still unclear at what stage of stimulus processing such serial biases arise. Here, we recorded behavior and neurophysiological [magnetoencephalographic (MEG)] data to test whether neural activity patterns during early sensory processing show the same biases seen in participants' reports. In a working-memory task that produced multiple biases in behavior, responses were biased toward previous targets, but away from more recent stimuli. Neural activity patterns were uniformly biased away from all previously relevant items. Our results contradict proposals that all serial biases arise at an early sensory processing stage. Instead, neural activity exhibited mostly adaptation-like responses to recent stimuli.

Audiovisual interaction with rate-varying signals

A task-irrelevant, amplitude-modulating sound influences perception of a size-modulating visual stimulus. To probe the limits of this audiovisual interaction we vary the second temporal derivative of object size and of sound amplitude. In the study's first phase subjects see a visual stimulus size-modulating with f ″ ( x ) > 0, 0, or <0, and judge each one's rate as increasing, constant, or decreasing. Visual stimuli are accompanied by a steady, non-modulated auditory stimulus. The novel combination of multiple stimuli and multi-alternative responses allows subjects' similarity space to be estimated from the stimulus-response confusion matrix. In the study's second phase, rate-varying visual stimuli are presented in concert with auditory stimuli whose second derivative also varied. Subjects identified each visual stimuli as one of the three types, while trying to ignore the accompanying sound. Unlike some previous results with f ″ ( x ) fixed at 0, performance benefits relatively little when visual and auditory stimuli share the same directional change in modulation. However, performance does drop when visual and auditory stimului differ in their directions of rate change. Our task's computational demands may make it particularly vulnerable to the effects of a dynamic task-irrelevant stimulus.

An adaptive perspective on visual working memory distortions

When holding multiple items in visual working memory, representations of individual items are often attracted to, or repelled from, each other. While this is empirically well-established, existing frameworks do not account for both types of distortions, which appear to be in opposition. Here, we demonstrate that both types of memory distortion may confer functional benefits under different circumstances. When there are many items to remember and subjects are near their capacity to accurately remember each item individually, memories for each item become more similar (attraction). However, when remembering smaller sets of highly similar but discernible items, memory for each item becomes more distinct (repulsion), possibly to support better discrimination. Importantly, this repulsion grows stronger with longer delays, suggesting that it dynamically evolves in memory and is not just a differentiation process that occurs during encoding. Furthermore, both attraction and repulsion occur even in tasks designed to mitigate response bias concerns, suggesting they are genuine changes in memory representations. Together, these results are in line with the theory that attraction biases act to stabilize memory signals by capitalizing on information about an entire group of items, whereas repulsion biases reflect a tradeoff between maintaining accurate but distinct representations. Both biases suggest that human memory systems may sacrifice veridical representations in favor of representations that better support specific behavioral goals. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Modeling mean estimation tasks in within-trial and across-trial contexts

The mean estimation task, which explicitly asks observers to estimate the mean feature value of multiple stimuli, is a fundamental paradigm in research areas such as ensemble coding and cue integration. The current study uses computational models to formalize how observers summarize information in mean estimation tasks. We compare model predictions from our Fidelity-based Integration Model (FIM) and other models on their ability to simulate observed patterns in within-trial weight distribution, across-trial information integration, and set-size effects on mean estimation accuracy. Experiments show non-equal weighting within trials in both sequential and simultaneous mean estimation tasks. Observers implicitly overestimated trial means below the global mean and underestimated trial means above the global mean. Mean estimation performance declined and stabilized with increasing set sizes. FIM successfully simulated all observed patterns, while other models failed. FIM's information sampling structure provides a new way to interpret the capacity limit in visual working memory and sub-sampling strategies. As a model framework, FIM offers task-dependent modeling for various ensemble coding paradigms, facilitating research synthesis across different studies in the literature.

Breaking the cardinal rule: The impact of interitem interaction and attentional priority on the cardinal biases in orientation working memory

Although it is not typically assumed in influential models of visual working memory (WM), representations in WM are systematically biased by multiple factors. Orientation representations are biased away from the cardinal axis (i.e., cardinal bias) and they are biased away from or toward the other orientation simultaneously held in WM (i.e., interitem interaction). The present study investigated the extent to which these two bias mechanisms interact in WM. In Experiment 1, participants remembered two sequentially presented orientations and reproduced both orientations after a short delay. Cardinal biases were assessed separately for the trials where the two mechanisms produce biases in the same direction (i.e., congruent trials) and the trials where they produce biases in the opposite direction (i.e., incongruent trials). Whereas congruent trials exhibited a typical cardinal bias, incongruent trials exhibited no cardinal bias, demonstrating that the cardinal bias was canceled out by the interitem interaction. Follow-up experiments extended these results by manipulating attentional priority for the two orientations by means of precue (Experiment 2) and postcue (Experiment 3). In both experiments, attentionally prioritized items exhibited a typical cardinal bias irrespective of the congruency whereas attentionally unprioritized items exhibited a reversal of the cardinal bias in the incongruent trials, demonstrating that selective attention modulates the influence of the interitem interaction. Together, these results suggest that WM leverages information about specific stimuli and their relationship to support a given behavioral goal.

Visual working memory items drift apart due to active, not passive, maintenance

How are humans capable of maintaining detailed representations of visual items in memory? When required to make fine discriminations, we sometimes implicitly differentiate memory representations away from each other to reduce interitem confusion. However, this separation of representations can inadvertently lead memories to be recalled as biased away from other memory items, a phenomenon termed repulsion bias. Using a nonretinotopically specific working memory paradigm, we found stronger repulsion bias with longer working memory delays, but only when items were actively maintained. These results suggest that (a) repulsion bias can reflect a mnemonic phenomenon, distinct from perceptually driven observations of repulsion bias; and (b) mnemonic repulsion bias is ongoing during maintenance and dependent on attention to internally maintained memory items. These results support theories of working memory where items are represented interdependently and further reveals contexts where stronger attention to working memory items during maintenance increases repulsion bias between them. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Concurrent visual working memory bias in sequential integration of approximate number

Previous work has shown bidirectional crosstalk between Working Memory (WM) and perception such that the contents of WM can alter concurrent percepts and vice versa. Here, we examine WM-perception interactions in a new task setting. Participants judged the proportion of colored dots in a stream of visual displays while concurrently holding location- and color information in memory. Spatiotemporally resolved psychometrics disclosed a modulation of perceptual sensitivity consistent with a bias of visual spatial attention towards the memorized location. However, this effect was short-lived, suggesting that the visuospatial WM information was rapidly deprioritized during processing of new perceptual information. Independently, we observed robust bidirectional biases of categorical color judgments, in that perceptual decisions and mnemonic reports were attracted to each other. These biases occurred without reductions in overall perceptual sensitivity compared to control conditions without a concurrent WM load. The results conceptually replicate and extend previous findings in visual search and suggest that crosstalk between WM and perception can arise at multiple levels, from sensory-perceptual to decisional processing.

Statistical learning as a reference point for memory distortions: Swap and shift errors

Humans use regularities in the environment to facilitate learning, often without awareness or intent. How might such regularities distort long-term memory? Here, participants studied and reported the colors of objects in a long-term memory paradigm, uninformed that certain colors were sampled more frequently overall. When participants misreported an object's color, these errors were often centered around the average studied color (i.e., "Rich" color), demonstrating swap errors in long-term memory due to imposed statistical regularities. We observed such swap errors regardless of memory load, explicit knowledge, or the distance in color space between the correct color of the tested object and the Rich color. An explicit guessing strategy where participants intentionally made swap errors when uncertain could not fully account for our results. We discuss other potential sources of observed swap errors such as false memory and implicit biased guessing. Although less robust than swap errors, evidence was also observed for subtle shift errors towards or away from the Rich color dependent on the color distance between the correct color and the Rich color. Together, these findings of swap and shift errors provide converging evidence for memory distortion mechanisms induced by a reference point, bridging a gap in the literature between how attention to regularities similarly influences visual working memory and visual long-term memory.

Differences in perceptual memory determine generalization patterns

Although memory of past experiences is crucial for the ability to transfer knowledge to new situations, surprisingly little research has directly investigated the relationship between memory and generalization. The present study sought to investigate how the perceptual memory of a trained stimulus influences generalization to similar stimuli. Forty participants underwent a fear conditioning procedure on Day 1, and separate memory recall and generalization tests on Day 2. We focused on two aspects of perceptual memory: namely memory bias (i.e., over- or underestimation of stimulus magnitude) and uncertainty. We found that memory bias predicted the pattern of generalized self-reported (expectancy ratings) and psychophysiological responses (fear-potentiated startle responses). Memory uncertainty was measured in two ways: self-reported uncertainty ratings and variability in stimulus recall. We found that higher levels of self-reported memory uncertainty corresponded with a broader generalization gradient on US expectancy, while greater variability in memory recall was associated with a broader generalization gradient on fear-potentiated startle responses. Taken together, our findings suggest that memory is an important determinant of generalized behavior and illustrate the need to account for these interindividual differences in perceptual memory when examining the generalization of learned responses.

Distraction biases working memory for faces

Working memory persists in the face of distraction, yet not without consequence. Previous research has shown that memory for low-level visual features is systematically influenced by the maintenance or presentation of a similar distractor stimulus. Responses are frequently biased in stimulus space towards a perceptual distractor, though this has yet to be determined for high-level stimuli. We investigated whether these influences are shared for complex visual stimuli such as faces. To quantify response accuracies for these stimuli, we used a delayed-estimation task with a computer-generated "face space" consisting of 80 faces that varied continuously as a function of age and sex. In a set of three experiments, we found that responses for a target face held in working memory were biased towards a distractor face presented during the maintenance period. The amount of response bias did not vary as a function of distance between target and distractor. Our data suggest that, similar to low-level visual features, high-level face representations in working memory are biased by the processing of related but task-irrelevant information.

Perceptual variability: Implications for learning and generalization

The generalization of learned behavior has been extensively investigated, but accounting for variance in generalized responding remains a challenge. Based on recent advances, we demonstrate that the inclusion of perceptual measures in generalization research may lead to a better understanding of both intra- and interindividual differences in generalization. We explore various ways through which perceptual variability can influence generalized responding. We investigate its impact on the ability to discriminate between stimuli and how similarity between stimuli may be variable, rather than fixed, because of it. Subsequently, we argue that perceptual variations can yield different learning experiences and that interindividual differences in generalized responding may be understood from this perspective. Finally, we point to the role of memory and decision-making within this context. Throughout this paper, we argue that accounting for perception in current generalization protocols will improve the precision of obtained generalization gradients and the ability to infer latent mechanisms. This can inspire future attempts to use generalization gradients as a (clinical) predictor or to relate them to individual traits and neural correlates and, ultimately, may lead to new theoretical and clinical insights.

Build-up of serial dependence in color working memory

Serial dependence, how immediately preceding experiences bias our current estimations, has been described experimentally during delayed-estimation of many different visual features, with subjects tending to make estimates biased towards previous ones. It has been proposed that these attractive biases help perception stabilization in the face of correlated natural scene statistics, although this remains mostly theoretical. Color, which is strongly correlated in natural scenes, has never been studied with regard to its serial dependencies. Here, we found significant serial dependence in 7 out of 8 datasets with behavioral data of humans (total n = 760) performing delayed-estimation of color with uncorrelated sequential stimuli. Moreover, serial dependence strength built up through the experimental session, suggesting metaplastic mechanisms operating at a slower time scale than previously proposed (e.g. short-term synaptic facilitation). Because, in contrast with natural scenes, stimuli were temporally uncorrelated, this build-up casts doubt on serial dependencies being an ongoing adaptation to the stable statistics of the environment.

Perceptual effects of fast and automatic visual ensemble statistics from faces in individuals with typical development and autism spectrum conditions

We investigated whether covert ensembles of high- (emotion), and low-level (brightness) visual information, extracted from peripheral faces (presentation/encoding:200 ms), unintentionally influences perception of a central target face stimulus in individuals typically developing (TD) and with autism spectrum condition (ASC). Graded alterations in the summary intensities of the emotion and brightness of the peripheral stimuli modulated the perceptions of the target face in both TD and ASC. In contrast, when we measured goal-directed (overt) ensemble face- emotion and brightness perception, we found that in half of ASC the overt ensemble emotion perception was impaired than TD. Additionally, we repeated both experiments with a backward visual mask to restrict not just encoding but also background processing in the visual system to 200 ms. This revealed that the effect of peripheral ensembles on centre perception was present only with brightness at least in TD but of overt ensembles was evident with both emotion and brightness in TD and ASC alike. These results suggest that while ensemble statistics of low-level information derived automatically and rapidly (200 ms) from contextualized faces are used for target face perception, the same takes longer with high-level information. However, overt facial ensembles are rapidly processed in both TD and ASC.

Central tendency representation and exemplar matching in visual short-term memory

I address a recent extension of the generalized context model (GCM), a model which excludes prototypes, to the visual short-term memory (VSTM) literature, which is currently deluged with prototype effects. The paper includes a brief review whose aim is to discuss the background and key findings suggesting that prototypes have an obligatory influence on visual short-term memory responses in the same VSTM task that the GCM's random walk extension, EBRW, was extended to account for: Sternberg scanning. I present a new model that incorporates such "central tendency representations" in memory, as well as several other regularities of the literature, and compare its prediction and postdictions to those of the GCM on some unpublished Sternberg scanning data. The GCM cannot account for the pattern in those data without post hoc modifications but the pattern is predicted nicely by the central tendency representation model. Although the new model is certainly wrong, the review and modeling exercise suggest a reconsideration of prototype models may be warranted, at least in the VSTM literature.

Distinguishing target biases and strategic guesses in visual working memory

Previous studies have shown that when there are statistical regularities in the items stored in visual working memory, the responses are biased toward the ensemble average. This statistical-regularity-induced bias could happen in two ways: (1) a target bias, where the individual memory representations are pulled toward the ensemble average; or (2) a strategic guess, for items that are not memorized, other information in the ensemble (e.g., another item) is reported as a substitute. Here, these two mechanisms are distinguished on the basis of a three-part model (target responses + swap responses + random guesses; e.g., Bays, Catalao, & Husain, 2009, Journal of Vision, 9, 7). The strategic guess is operationalized as swap responses, whereas the target bias is reflected by a bias parameter in the target responses. This model was applied on 8 data sets (22 observers each). In this model, contributions of target biases and strategic guesses can be clearly distinguished from each other because they lead to distinctive patterns in the distribution of responses. In the present results, strategic guesses always contributed substantially to the statistical-regularity-induced biases, whereas target biases were limited to specific conditions. All in all, the Bayesian inference in visual working memory is much more limited than what is previously advocated.

Working Memory Maintenance Modulates Serial Dependence Effects of Perceived Emotional Expression

The stability of face perception is vital in interpersonal interactions. Recent studies have revealed the mechanism of the stability in the perception of stable attributes of faces (such as facial identity) by serial dependence, a phenomenon in which perception of current stimuli is pulled toward recently viewed stimuli. However, whether serial dependence of perceived emotional expression (a changeable attribute of faces) exists remains controversial, and its exact nature has not been examined yet. To address these issues, we used the methods of constant stimuli and two-interval forced choice tasks in three psychophysical experiments. Participants compared two successive facial expressions selected from a continuum with 50 morphed faces ranging from sad to happy. Experiment 1a and 1b showed that a perceived facial expression pulled toward previously seen facial expressions (i.e., a significant serial dependence effect), independent of response instructions. Furthermore, a stronger serial dependence effect was found when the first facial expression was retained in working memory for a longer delay duration (Experiment 2), and yet a weaker serial dependence effect was observed when a longer delay between decision and response was performed (Experiment 3). These findings indicate that serial dependence facilitates the stability of facial expression perception and is modulated by working memory representations.

Coexisting representations of sensory and mnemonic information in human visual cortex

Traversing sensory environments requires keeping relevant information in mind while simultaneously processing new inputs. Visual information is kept in working memory via feature selective responses in early visual cortex, but recent work had suggested that new sensory inputs obligatorily wipe out this information. Here we show region-wide multiplexing abilities in classic sensory areas, with population-level response patterns in early visual cortex representing the contents of working memory alongside new sensory inputs. In a second experiment, we show that when people get distracted, this leads to both disruptions of mnemonic information in early visual cortex and decrements in behavioral recall. Representations in the intraparietal sulcus reflect actively remembered information encoded in a transformed format, but not task-irrelevant sensory inputs. Together these results suggest that early visual areas play a key role in supporting high resolution working memory representations that can serve as a template for comparing incoming sensory information.

The power of predictions: An emerging paradigm for psychological research

The last two decades of neuroscience research has produced a growing number of studies that suggest the various psychological phenomena are produced by predictive processes in the brain. When considered together, these studies form a coherent, neurobiologically-inspired research program for guiding psychological research about the mind and behavior. In this paper, we briefly consider the common assumptions and hypotheses that unify an emerging framework and discuss its ramifications, both for improving the replicability and robustness of psychological research and for innovating psychological theory by suggesting an alternative ontology of the human mind.

Reference repulsion is not a perceptual illusion

Perceptual decisions are often influenced by contextual factors. For instance, when engaged in a visual discrimination task against a reference boundary, subjective reports about the judged stimulus feature are biased away from the boundary - a phenomenon termed reference repulsion. Until recently, this phenomenon has been thought to reflect a perceptual illusion regarding the appearance of the stimulus, but new evidence suggests that it may rather reflect a post-perceptual decision bias. To shed light on this issue, we examined whether and how orientation judgments affect perceptual appearance. In a first experiment, we confirmed that after judging a grating stimulus against a discrimination boundary, the subsequent reproduction response was indeed repelled from the boundary. To investigate the perceptual nature of this bias, in a second experiment we measured the perceived orientation of the grating stimulus more directly, in comparison to a reference stimulus visible at the same time. Although we did observe a small repulsive bias away from the boundary, this bias was explained by random trial-by-trial fluctuations in sensory representations together with classical stimulus adaptation effects and did not reflect a systematic bias due to the discrimination judgment. Overall, the current study indicates that discrimination judgments do not elicit a perceptual illusion and points towards a post-perceptual locus of reference repulsion.

Decay of internal reference information in duration discrimination: Intertrial interval modulates the Type B effect

Psychophysical evidence suggests that human perception of a stimulus is assimilated towards previous stimuli. The internal reference model (IRM) explains such assimilation through an internal reference (IR), which integrates past and present stimulus representations and thus might be conceived as a form of perceptual memory. In this study, we investigated whether the IR decays with time, as previously shown for perceptual memory representations in general. One specific prediction of IRM is higher discrimination sensitivity when a constant standard precedes rather than follows a variable comparison in a two-alternative forced-choice (2AFC) discrimination task. Furthermore, the magnitude of this so-called negative Type B effect should decrease with decreasing weighting of past stimulus information in the integration process. Therefore, decay of the IR should result in a reduced Type B effect. To examine this prediction, we carried out a 2AFC duration discrimination experiment with a short (1,600 ms) and a long (3,200 ms) intertrial interval (ITI). As expected, a reduced negative Type B effect was observed at the long compared with the short ITI, consistent with the idea that humans rely on the immediate past when evaluating current sensory input, however, less so when the IR incorporating the perceptual short-term memory representation of these past stimuli has already decayed.

Two types of serial dependence in visual working memory

Stimulus representations in working memory depend on memory traces of past stimuli both from previous trials and from the current trial. However, it is unclear whether the same or different mechanisms underlie this serial dependence across and within trials. We directly contrasted estimates of bias for pairs of immediately successive stimuli across and within trials. In each trial, participants memorized two consecutive motion direction stimuli (S1 and S2) and after a short delay were cued to report one of them. We found serial dependence across trials: The current S1 was attracted towards the preceding S2 when the latter had been cued for report. In contrast, within the same trial S2 was repulsed from S1. In addition, repulsion within a trial occurred for a broader range of motion direction differences between stimuli than attraction across trials. A second experiment in which 25% of trials did not require a response demonstrated that across-trial attraction did not depend on whether the previous S2 actually had to be reported. Our findings provide evidence for two types of serial dependence operating across and within trials. They support the notion of different mechanisms integrating or segregating current from similar past memory contents depending on their task relevance.

A two-level hierarchical framework of visual short-term memory

Over the last couple of decades, a vast amount of research has been dedicated to understanding the nature and the architecture of visual short-term memory (VSTM), the mechanism by which currently relevant visual information is maintained. According to discrete-capacity models, VSTM is constrained by a limited number of discrete representations held simultaneously. In contrast, shared-resource models regard VSTM as limited in resources, which can be distributed flexibly between varying numbers of representations; and a new interference model posits that capacity is limited by interference among items. In this article, we begin by reviewing benchmark findings regarding the debate over VSTM limitations, focusing on whether VSTM storage is all-or-none and on whether object complexity affects capacity. After that, we put forward a hybrid framework of VSTM architecture, arguing that this system is composed of a two-level hierarchy of memory stores, each containing a different set of representations: (1) perceptual memory, a resourcelike level containing analog automatically formed representations of visual stimuli in varying degrees of activation, and (2) visual working memory, in which a subset of three to four items from perceptual memory are bound to conceptual representations and to their locations, thus conveying discrete (digital/symbolic) information which appears quantized. While perceptual memory has a large capacity and is relatively nonselective, visual working memory is restricted in the number of items that can be maintained simultaneously, and its content is regulated by a gating mechanism.

Flexible Coding of Visual Working Memory Representations during Distraction

Visual working memory (VWM) recruits a broad network of brain regions, including prefrontal, parietal, and visual cortices. Recent evidence supports a "sensory recruitment" model of VWM, whereby precise visual details are maintained in the same stimulus-selective regions responsible for perception. A key question in evaluating the sensory recruitment model is how VWM representations persist through distracting visual input, given that the early visual areas that putatively represent VWM content are susceptible to interference from visual stimulation.To address this question, we used a functional magnetic resonance imaging inverted encoding model approach to quantitatively assess the effect of distractors on VWM representations in early visual cortex and the intraparietal sulcus (IPS), another region previously implicated in the storage of VWM information. This approach allowed us to reconstruct VWM representations for orientation, both before and after visual interference, and to examine whether oriented distractors systematically biased these representations. In our human participants (both male and female), we found that orientation information was maintained simultaneously in early visual areas and IPS in anticipation of possible distraction, and these representations persisted in the absence of distraction. Importantly, early visual representations were susceptible to interference; VWM orientations reconstructed from visual cortex were significantly biased toward distractors, corresponding to a small attractive bias in behavior. In contrast, IPS representations did not show such a bias. These results provide quantitative insight into the effect of interference on VWM representations, and they suggest a dynamic tradeoff between visual and parietal regions that allows flexible adaptation to task demands in service of VWM.SIGNIFICANCE STATEMENT Despite considerable evidence that stimulus-selective visual regions maintain precise visual information in working memory, it remains unclear how these representations persist through subsequent input. Here, we used quantitative model-based fMRI analyses to reconstruct the contents of working memory and examine the effects of distracting input. Although representations in the early visual areas were systematically biased by distractors, those in the intraparietal sulcus appeared distractor-resistant. In contrast, early visual representations were most reliable in the absence of distraction. These results demonstrate the dynamic, adaptive nature of visual working memory processes, and provide quantitative insight into the ways in which representations can be affected by interference. Further, they suggest that current models of working memory should be revised to incorporate this flexibility.

Quantifying attentional effects on the fidelity and biases of visual working memory in young children

Attentional control enables us to direct our limited resources to accomplish goals. The ability to flexibly allocate resources helps to prioritize information and inhibit irrelevant/distracting information. We examined developmental changes in visual working memory (VWM) fidelity in 4- to 7-year-old children and the effects that a distracting non-target object can exert in biasing their memory representations. First, we showed that VWM fidelity improves from early childhood to adulthood. Second, we found evidence of working memory load on recall variability in children and adults. Next, using cues to manipulate attention, we found that older children are able to construct a more durable memory representation for an object presented following a non-target using a pre-cue (that biases encoding before presentation) compared with a retro-cue (that signals which item to recall after presentation). In addition, younger children had greater difficulties maintaining an item in memory when an intervening item was presented. Lastly, we found that memory representations are biased toward a non-target when it is presented following the target and away from a non-target when it precedes the target. These bias effects were more pronounced in children compared with adults. Together, these results demonstrate changes in attention over development that influence VWM memory fidelity.

Evidence of gradual loss of precision for simple features and complex objects in visual working memory

Previous studies have suggested that people can maintain prioritized items in visual working memory for many seconds, with negligible loss of information over time. Such findings imply that working memory representations are robust to the potential contaminating effects of internal noise. However, once visual information is encoded into working memory, one might expect it to inevitably begin degrading over time, as this actively maintained information is no longer tethered to the original perceptual input. Here, we examined this issue by evaluating working memory for single central presentations of an oriented grating, color patch, or face stimulus, across a range of delay periods (1, 3, 6, or 12 s). We applied a mixture-model analysis to distinguish changes in memory precision over time from changes in the frequency of outlier responses that resemble random guesses. For all 3 types of stimuli, participants exhibited a clear and consistent decline in the precision of working memory as a function of temporal delay, as well as a modest increase in guessing-related responses for colored patches and face stimuli. We observed a similar loss of precision over time while controlling for temporal distinctiveness. Our results demonstrate that visual working memory is far from lossless: while basic visual features and complex objects can be maintained in a quite stable manner over time, these representations are still subject to noise accumulation and complete termination. (PsycINFO Database Record

Biased by the Group: Memory for an Emotional Expression Biases Towards the Ensemble

An emotional expression can be misremembered as more similar to previously seen expressions than it actually was – demonstrating inductive category effects for emotional expressions. Given that memory is influenced over time, we sought to determine whether memory for a single expression would be similarly influenced by other expressions viewed simultaneously. In other words, we test whether the ability to encode statistical features of an ensemble (i.e., ensemble encoding) is leveraged when attempting to recall a single expression from the ensemble. In three preregistered experiments, participants saw an ensemble of 4 expressions, one neutral and the three either happy or sad. After a delay, participants were asked to reproduce the neutral face by adjusting a response face’s expression. In Experiment 1, the ensemble comprised images of the same actor; in Experiments 2 and 3, images were comprised of individuals varying race and gender. In each experiment we demonstrated that even after only a single exposure, memory for the neutral expression in the happy group was biased happier relative to the same expression in the sad group. Data and syntax can be found at https://osf.io/gcbez/.

Synaptic augmentation in a cortical circuit model reproduces serial dependence in visual working memory

Recent work has established that visual working memory is subject to serial dependence: current information in memory blends with that from the recent past as a function of their similarity. This tuned temporal smoothing likely promotes the stability of memory in the face of noise and occlusion. Serial dependence accumulates over several seconds in memory and deteriorates with increased separation between trials. While this phenomenon has been extensively characterized in behavior, its neural mechanism is unknown. In the present study, we investigate the circuit-level origins of serial dependence in a biophysical model of cortex. We explore two distinct kinds of mechanisms: stable persistent activity during the memory delay period and dynamic “activity-silent” synaptic plasticity. We find that networks endowed with both strong reverberation to support persistent activity and dynamic synapses can closely reproduce behavioral serial dependence. Specifically, elevated activity drives synaptic augmentation, which biases activity on the subsequent trial, giving rise to a spatiotemporally tuned shift in the population response. Our hybrid neural model is a theoretical advance beyond abstract mathematical characterizations, offers testable hypotheses for physiological research, and demonstrates the power of biological insights to provide a quantitative explanation of human behavior.

Serial dependence is absent at the time of perception but increases in visual working memory

Recent experiments have shown that visual cognition blends current input with that from the recent past to guide ongoing decision making. This serial dependence appears to exploit the temporal autocorrelation normally present in visual scenes to promote perceptual stability. While this benefit has been assumed, evidence that serial dependence directly alters stimulus perception has been limited. In the present study, we parametrically vary the delay between stimulus and response in a spatial delayed response task to explore the trajectory of serial dependence from the moment of perception into post-perceptual visual working memory. We find that behavioral responses made immediately after viewing a stimulus show evidence of adaptation, but not attractive serial dependence. Only as the memory period lengthens is a blending of past and present information apparent in behavior, reaching its maximum with a delay of six seconds. These results dovetail with other recent findings to bolster the interpretation that serial dependence is a phenomenon of mnemonic rather than perceptual processes. However, even while this pattern of effects in group-averaged data has now been found consistently, we show that the relative strengths of adaptation and serial dependence vary widely across individuals. Finally, we demonstrate that when leading mathematical models of working memory are adjusted to account for these trial-history effects, their fit to behavioral data is substantially improved.

Interactions between visual working memory representations

We investigated whether the representations of different objects are maintained independently in working memory or interact with each other. Observers were shown two sequentially presented orientations and required to reproduce each orientation after a delay. The sequential presentation minimized perceptual interactions so that we could isolate interactions between memory representations per se. We found that similar orientations were repelled from each other whereas dissimilar orientations were attracted to each other. In addition, when one of the items was given greater attentional priority by means of a cue, the representation of the high-priority item was not influenced very much by the orientation of the low-priority item, but the representation of the low-priority item was strongly influenced by the orientation of the high-priority item. This indicates that attention modulates the interactions between working memory representations. In addition, errors in the reported orientations of the two objects were positively correlated under some conditions, suggesting that representations of distinct objects may become grouped together in memory. Together, these results demonstrate that working-memory representations are not independent but instead interact with each other in a manner that depends on attentional priority.

Reevaluating the Sensory Account of Visual Working Memory Storage

Recent human fMRI pattern-decoding studies have highlighted the involvement of sensory areas in visual working memory (VWM) tasks and argue for a sensory account of VWM storage. In this review, evidence is examined from human behavior, fMRI decoding, and transcranial magnetic stimulation (TMS) studies, as well as from monkey neurophysiology studies. Contrary to the prevalent view, the available evidence provides little support for the sensory account of VWM storage. Instead, when the ability to resist distraction and the existence of top-down feedback are taken into account, VWM-related activities in sensory areas seem to reflect feedback signals indicative of VWM storage elsewhere in the brain. Collectively, the evidence shows that prefrontal and parietal regions, rather than sensory areas, play more significant roles in VWM storage.

When past is present: Substitutions of long-term memory for sensory evidence in perceptual judgments

When perception is underdetermined by current sensory inputs, memories for related experiences in the past might fill in missing detail. To evaluate this possibility, we measured the likelihood of relying on long-term memory versus sensory evidence when judging the appearance of an object near the threshold of awareness. Specifically, we associated colors with shapes in long-term memory and then presented the shapes again later in unrelated colors and had observers judge the appearance of the new colors. We found that responses were well characterized as a bimodal mixture of original and current-color representations (vs. an integrated unimodal representation). That is, although irrelevant to judgments of the current color, observers occasionally anchored their responses on the original colors in memory. Moreover, the likelihood of such memory substitutions increased when sensory input was degraded. In fact, they occurred even in the absence of sensory input when observers falsely reported having seen something. Thus, although perceptual judgments intuitively seem to reflect the current state of the environment, they can also unknowingly be dictated by past experiences.

Online Modulation of Selective Attention is not Impaired in Healthy Aging

Background/Study Context: Reduced processing speed pervades a great many aspects of human aging and cognition. However, little is known about one aspect of cognitive aging in which speed is of the essence, namely, the speed with which older adults can deploy attention in response to a cue.

METHODS

The authors compared rapid temporal modulation of cued visual attention in younger (M_age = 22.3 years) and older (M_age = 68.9 years) adults. On each trial of a short-term memory task, a cue identified which of two briefly presented stimuli was task relevant and which one should be ignored. After a short delay, subjects demonstrated recall by reproducing from memory the task-relevant stimulus. This produced estimates of (i) accuracy with which the task-relevant stimulus was recalled, (ii) the influence of stimuli encountered on previous trials (a prototype effect), and (iii) the influence of the trial's task-irrelevant stimulus.

RESULTS

For both groups, errors in recall were considerably smaller when selective attention was cued before rather than after presentation of the stimuli. Both groups showed serial position effects to the same degree, and both seemed equally adept at exploiting the stimuli encountered on previous trials as a means of supplementing recall accuracy on the current trial.

CONCLUSION

Younger and older subjects may not differ reliably in capacity for cue-directed temporal modulation of selective attention, or in ability to draw on previously seen stimuli as memory support.

Face familiarity promotes stable identity recognition: exploring face perception using serial dependence

Studies suggest that familiar faces are processed in a manner distinct from unfamiliar faces and that familiarity with a face confers an advantage in identity recognition. Our visual system seems to capitalize on experience to build stable face representations that are impervious to variation in retinal input that may occur due to changes in lighting, viewpoint, viewing distance, eye movements, etc. Emerging evidence also suggests that our visual system maintains a continuous perception of a face's identity from one moment to the next despite the retinal input variations through serial dependence. This study investigates whether interactions occur between face familiarity and serial dependence. In two experiments, participants used a continuous scale to rate attractiveness of unfamiliar and familiar faces (either experimentally learned or famous) presented in rapid sequences. Both experiments revealed robust inter-trial effects in which attractiveness ratings for a given face depended on the preceding face's attractiveness. This inter-trial attractiveness effect was most pronounced for unfamiliar faces. Indeed, when participants were familiar with a given face, attractiveness ratings showed significantly less serial dependence. These results represent the first evidence that familiar faces can resist the temporal integration seen in sequential dependencies and highlight the importance of familiarity to visual cognition.

Neural mechanisms of information storage in visual short-term memory

The capacity to briefly memorize fleeting sensory information supports visual search and behavioral interactions with relevant stimuli in the environment. Traditionally, studies investigating the neural basis of visual short term memory (STM) have focused on the role of prefrontal cortex (PFC) in exerting executive control over what information is stored and how it is adaptively used to guide behavior. However, the neural substrates that support the actual storage of content-specific information in STM are more controversial, with some attributing this function to PFC and others to the specialized areas of early visual cortex that initially encode incoming sensory stimuli. In contrast to these traditional views, I will review evidence suggesting that content-specific information can be flexibly maintained in areas across the cortical hierarchy ranging from early visual cortex to PFC. While the factors that determine exactly where content-specific information is represented are not yet entirely clear, recognizing the importance of task-demands and better understanding the operation of non-spiking neural codes may help to constrain new theories about how memories are maintained at different resolutions, across different timescales, and in the presence of distracting information.

Rapid forgetting results from competition over time between items in visual working memory

Working memory is now established as a fundamental cognitive process across a range of species. Loss of information held in working memory has the potential to disrupt many aspects of cognitive function. However, despite its significance, the mechanisms underlying rapid forgetting remain unclear, with intense recent debate as to whether it is interference between stored items that leads to loss of information or simply temporal decay. Here we show that both factors are essential and interact in a highly specific manner. Although a single item can be maintained in memory with high fidelity, multiple items compete in working memory, progressively degrading each other’s representations as time passes. Specifically, interaction between items is associated with both worsening precision and increased reporting errors of object features over time. Importantly, during the period of maintenance, although items are no longer visible, maintenance resources can be selectively redeployed to protect the probability to recall the correct feature and the precision with which cued items can be recalled, as if it was the only item in memory. These findings reveal that the biased competition concept could be applied not only to perceptual processes but also to active maintenance of working memory representations over time.

Divided spatial attention and feature-mixing errors

Spatial attention is thought to play a critical role in feature binding. However, often multiple objects or locations are of interest in our environment, and we need to shift or split attention between them. Recent evidence has demonstrated that shifting and splitting spatial attention results in different types of feature-binding errors. In particular, when two locations are simultaneously sharing attentional resources, subjects are susceptible to feature-mixing errors; that is, they tend to report a color that is a subtle blend of the target color and the color at the other attended location. The present study was designed to test whether these feature-mixing errors are influenced by target-distractor similarity. Subjects were cued to split attention across two different spatial locations, and were subsequently presented with an array of colored stimuli, followed by a postcue indicating which color to report. Target-distractor similarity was manipulated by varying the distance in color space between the two attended stimuli. Probabilistic modeling in all cases revealed shifts in the response distribution consistent with feature-mixing errors; however, the patterns differed considerably across target-distractor color distances. With large differences in color, the findings replicated the mixing result, but with small color differences, repulsion was instead observed, with the reported target color shifted away from the other attended color.

Turning Symbolic: The Representation of Motion Direction in Working Memory

What happens to the representation of a moving stimulus when it is no longer present and its motion direction has to be maintained in working memory (WM)? Is the initial, sensorial representation maintained during the delay period or is there another representation, at a higher level of abstraction? It is also feasible that multiple representations may co-exist in WM, manifesting different facets of sensory and more abstract features. To that end, we investigated the mnemonic representation of motion direction in a series of three psychophysical experiments, using a delayed motion-discrimination task (relative clockwise∖counter-clockwise judgment). First, we show that a change in the dots’ contrast polarity does not hamper performance. Next, we demonstrate that performance is unaffected by relocation of the Test stimulus in either retinotopic or spatiotopic coordinate frames. Finally, we show that an arrow-shaped cue presented during the delay interval between the Sample and Test stimulus, strongly biases performance toward the direction of the arrow, although the cue itself is non-informative (it has no predictive value of the correct answer). These results indicate that the representation of motion direction in WM could be independent of the physical features of the stimulus (polarity or position) and has non-sensorial abstract qualities. It is plausible that an abstract mnemonic trace might be activated alongside a more basic, analog representation of the stimulus. We speculate that the specific sensitivity of the mnemonic representation to the arrow-shaped symbol may stem from the long term learned association between direction and the hour in the clock.

Winter is coming: How humans forage in a temporally structured environment

Much is known about visual search for single targets, but relatively little about how participants "forage" for multiple targets. One important question is how long participants will search before moving to a new display. Evidence suggests that participants should leave when intake drops below the average rate ("optimal foraging," Charnov, 1976). However, the real world has temporal structure (e.g., seasons) that could influence behavior. Does it matter if winter is coming and the next display will be worse than the last? We gave participants a series of search displays and asked them to collect targets as fast as possible. Target density was structured-rising and falling systematically across trials. We measured the duration for which participants foraged in each display (trials were terminated by participants). Foraging behavior was affected by temporal structure-counter to a simple optimal foraging account, observers searched displays longer when quality was falling compared to rising (Experiments 1 and 2). Additionally, we found that temporal structure altered explicit predictions about display quality (Experiment 2). These results demonstrate that foraging theories need to consider richer models of observers' representations of the world.

Neural circuit basis of visuo-spatial working memory precision: a computational and behavioral study

The amount of information that can be retained in working memory (WM) is limited. Limitations of WM capacity have been the subject of intense research, especially in trying to specify algorithmic models for WM. Comparatively, neural circuit perspectives have barely been used to test WM limitations in behavioral experiments. Here we used a neuronal microcircuit model for visuo-spatial WM (vsWM) to investigate memory of several items. The model assumes that there is a topographic organization of the circuit responsible for spatial memory retention. This assumption leads to specific predictions, which we tested in behavioral experiments. According to the model, nearby locations should be recalled with a bias, as if the two memory traces showed attraction or repulsion during the delay period depending on distance. Another prediction is that the previously reported loss of memory precision for an increasing number of memory items (memory load) should vanish when the distances between items are controlled for. Both predictions were confirmed experimentally. Taken together, our findings provide support for a topographic neural circuit organization of vsWM, they suggest that interference between similar memories underlies some WM limitations, and they put forward a circuit-based explanation that reconciles previous conflicting results on the dependence of WM precision with load.

Toward ecologically realistic theories in visual short-term memory research

Recent evidence from neuroimaging and psychophysics suggests common neural and representational substrates for visual perception and visual short-term memory (VSTM).Visual perception is adapted to a rich set of statistical regularities present in the natural visual environment. Common neural and representational substrates for visual perception and VSTM suggest that VSTM is adapted to these same statistical regularities too. This article discusses how the study of VSTM can be extended to stimuli that are ecologically more realistic than those commonly used in standard VSTM experiments and what the implications of such an extension could be for our current view of VSTM. We advocate for the development of unified models of visual perception and VSTM—probabilistic and hierarchical in nature— incorporating prior knowledge of natural scene statistics.

Supraliminal but not subliminal distracters bias working memory recall

Information of which observers are not consciously aware can nevertheless influence perceptual processes. Whether subliminal information might exert an influence on working memory (WM) representations is less clear, and relatively few studies have examined the interactions between subliminal and supraliminal information in WM. We present 3 experiments examining this issue. Experiments 1a and b replicated the finding that orientation stimuli can influence behavior subliminally in a visuomotor priming task. Experiments 2 and 3 used the same orientation stimuli, but participants had to remember a target orientation and report it back by adjusting a probe orientation after a memory delay. Before or after presentation of the target orientation, a subliminal or supraliminal distracter orientation was presented that was either irrelevant for task completion and never had to be reported (Experiment 2), or was relevant for task completion because it had to be reported on some trials (Experiment 3). In both experiments, presentation of a supraliminal distracter influenced WM recall of the target orientation. When the distracter was presented subliminally, however, there was no bias in orientation recall. These results suggest that information stored in WM is protected from influences of subliminal stimuli, while online information processing is modulated by subliminal information.

The location but not the attributes of visual cues are automatically encoded into working memory

Although it has been well known that visual cues affect the perception of subsequent visual stimuli, relatively little is known about how the cues themselves are processed. The present study attempted to characterize the processing of a visual cue by investigating what information about the cue is stored in terms of both location ("where" is the cue) and attributes ("what" are the attributes of the cue). In 11 experiments subjects performed several trials of reporting a target letter and then answered an unexpected question about the cue (e.g., the location, color, or identity of the cue). This surprise question revealed that participants could report the location of the cue even when the cue never indicated the target location and they were explicitly told to ignore it. Furthermore, the memory trace of this location information endured during encoding of the subsequent target. In contrast to location, attributes of the cue (e.g., color) were poorly reported, even for attributes that were used by subjects to perform the task. These results shed new light on the mechanisms underlying cueing effects and suggest also that the visual system may create empty object files in response to visual cues.

Fine-grained, local maps and coarse, global representations support human spatial working memory

While sensory processes are tuned to particular features, such as an object's specific location, color or orientation, visual working memory (vWM) is assumed to store information using representations, which generalize over a feature dimension. Additionally, current vWM models presume that different features or objects are stored independently. On the other hand, configurational effects, when observed, are supposed to mainly reflect encoding strategies. We show that the location of the target, relative to the display center and boundaries, and overall memory load influenced recall precision, indicating that, like sensory processes, capacity limited vWM resources are spatially tuned. When recalling one of three memory items the target distance from the display center was overestimated, similar to the error when only one item was memorized, but its distance from the memory items' average position was underestimated, showing that not only individual memory items' position, but also the global configuration of the memory array may be stored. Finally, presenting the non-target items at recall, consequently providing landmarks and configurational information, improved precision and accuracy of target recall. Similarly, when the non-target items were translated at recall, relative to their position in the initial display, a parallel displacement of the recalled target was observed. These findings suggest that fine-grained spatial information in vWM is represented in local maps whose resolution varies with distance from landmarks, such as the display center, while coarse representations are used to store the memory array configuration. Both these representations are updated at the time of recall.