The Influence of Active Removal from Working Memory on Serial Dependence

Opposing sequential biases in direction and time reproduction: Influences of task relevance and working memory

Our current perception and decision-making are shaped by recent experiences, a phenomenon known as serial dependence. While serial dependence is well-documented in visual perception and has been recently explored in time perception, their functional similarities across non-temporal and temporal domains remain elusive, particularly in relation to task relevance and working memory load. To address this, we designed a unified experimental paradigm using coherent motion stimuli to test both direction and time reproduction. The direction and time tasks were randomly mixed across trials. Additionally, we introduced pre-cue versus post-cue settings in separate experiments to manipulate working memory load during the encoding phase. We found attractive biases in time reproduction but repulsive biases in direction estimation. Notably, the temporal attraction was more pronounced when the preceding task was also time-related. In contrast, the direction repulsion remained unaffected by the nature of the preceding task. Additionally, both attractive and repulsive biases were enhanced by the post-cue compared to the pre-cue. Our findings suggest that opposing sequential effects in non-temporal and temporal domains may originate from different processing stages linked to sensory adaptation and post-perceptual processes involving working memory.

Selectively maintaining an object's feature in visual working memory: A comparison between highly discriminable and fine-grained features

Selectively maintaining information is an essential function of visual working memory (VWM). Recent VWM studies have mainly focused on selective maintenance of objects, leaving the mechanisms of selectively maintaining an object's feature in VWM unknown. Based on the interactive model of perception and VWM, we hypothesized that there are distinct selective maintenance mechanisms for objects containing fine-grained features versus objects containing highly discriminable features. To test this hypothesis, we first required participants to memorize a dual-feature object (colored simple shapes vs. colored polygons), and informed them about the target feature via a retro-cue. Then a visual search task was added to examine the fate of the irrelevant feature. The selective maintenance of an object's feature predicted that the irrelevant feature should be removed from the active state of VWM and should not capture attention when presented as a distractor in the visual search task. We found that irrelevant simple shapes impaired performance in the visual search task (Experiment 1). However, irrelevant polygons did not affect visual search performance (Experiment 2), and this could not be explained by decay of polygons (Experiment 3) or by polygons not capturing attention (Experiment 4). These findings suggest that VWM adopts dissociable mechanisms to selectively maintain an object's feature, depending on the feature's perceptual characteristics.

Serial dependence: A matter of memory load

In serial dependence, perceptual decisions are biased towards stimuli encountered in the recent past. Here, we investigate whether and how serial dependence is affected by the availability of visual working memory (VWM) resources. In two experiments, participants reproduced the orientation of a series of stimuli. On alternating trials, we included an additional VWM task with randomly varying levels of load. Serial dependence was not only affected by the additional load task but also clearly modulated by the level of load: a high load in the previous trial reduced serial dependence while a high load in the present increased it. These results were independent of the effects of VWM load on the precision of reproduction responses. Our findings provide new insights into the mechanisms that may regulate serial dependence, revealing its intimate link with VWM resources.

Significance statement

Our perception, thoughts, and behavior are continuously influenced by recent events. For instance, the way we process and understand current visual information depends on what we have seen in the preceding seconds, a phenomenon known as serial dependence. The precise mechanisms and factors involved in serial dependence are still unclear. Here, we demonstrated that working memory resources are a crucial component. Specifically, when we are currently experiencing a heavy memory load, the influence of prior stimuli becomes stronger. Conversely, when prior stimuli were shown under a high memory load, their influence was reduced. These findings highlight the importance of working memory resources in shaping our interpretation of the present based on the recent past.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Attractive and repulsive serial dependence: The role of task relevance, the passage of time, and the number of stimuli

Visual decisions are attracted toward features of previous stimuli. This phenomenon, termed serial dependence, has been related to a mechanism that integrates present visual input with stimuli seen up to 10 to 15 s in the past. It is believed that this mechanism is "temporally tuned" and the effect of prior stimuli fades with time. Here, we investigated whether the temporal window of serial dependence is influenced by the number of stimuli shown. Observers performed an orientation adjustment task where the interval between the past and present stimulus and the number of intervening stimuli varied. First, we found that the direction-repulsive or attractive-and duration of the effect of a past stimulus depends on whether the past stimulus was relevant to behavior. Second, we show that the number of stimuli, and not only the passage of time, plays a role: The effect of a stimulus at a fixed interval depends on the number of other stimuli shown after. Our results demonstrate that neither a single mechanism nor a general tuning window can fully capture the complexity of serial dependence.

Are irrelevant items actively deleted from visual working memory?: No evidence from repulsion and attraction effects in dual-retrocue tasks

Some theories propose that working memory (WM) involves the active deletion of irrelevant information, including items that were retained in WM, but are no longer relevant for ongoing cognition. Considerable evidence suggests that active-deletion occurs for categorical representations, but whether it also occurs for recall of features that are typically bound together in an object, such as line orientations, is unclear. In two experiments, with or without binding instructions, healthy young adults maintained two orientations, focused attention to recall the orientation cued first, and then switched attention to recall the orientation cued second, at which point the uncued orientation was no longer relevant on the trial. In contrast to the active-deletion hypothesis, the results showed that the no-longer-relevant items exerted the strongest bias on participants' recall, which was either repulsive or attractive depending on both the degree of difference between the target and nontarget orientations and the proximity to cardinal axes. We suggest that visual WM can bind features like line orientations into chunked representations, and an irrelevant feature of a chunked object cannot be actively deleted - it biases recall of the target feature. Models of WM need to be updated to explain this and related dynamic phenomena.

Serial dependence in visual perception: A review

How does the visual system represent continuity in the constantly changing visual input? A recent proposal is that vision is serially dependent: Stimuli seen a moment ago influence what we perceive in the present. In line with this, recent frameworks suggest that the visual system anticipates whether an object seen at one moment is the same as the one seen a moment ago, binding visual representations across consecutive perceptual episodes. A growing body of work supports this view, revealing signatures of serial dependence in many diverse visual tasks. Yet, the variety of disparate findings and interpretations calls for a more general picture. Here, we survey the main paradigms and results over the past decade. We also focus on the challenge of finding a relationship between serial dependence and the concept of "object identity," taking centuries-long history of research into account. Among the seemingly contrasting findings on serial dependence, we highlight common patterns that may elucidate the nature of this phenomenon and attempt to identify questions that are unanswered.

Flexible top-down control in the interaction between working memory and perception

Successful goal-directed behavior often requires continuous sensory processing while simultaneously maintaining task-related information in working memory (WM). Although WM and perception are known to interact, little is known about how their interactions are controlled. Here, we tested the hypothesis that WM perception interactions engage two distinct modes of control – proactive and reactive – in a manner similar to classic conflict-adaptation tasks (e.g. Stroop, flanker, and Simon). Participants performed a delayed recall-of-orientation WM task, plus a standalone visual discrimination-of-orientation task the occurred during the delay period, and with the congruity in orientation between the tasks manipulated. Proactive control was seen in the sensitivity of task performance to the previous trial's congruity (i.e. a Gratton effect). Reactive control was observed in a repulsive serial-dependence produced by incongruent discriminanda. Quantitatively, these effects were explained by parameters from a reinforcement learning-based model that tracks trial-to-trial fluctuations in control demand: reactive control by a phasic control prediction error (control PE), and proactive control by a tonic level of predicted conflict updated each trial by the control PE. Thus, WM-perception interactions may be controlled by the same mechanisms that govern conflict in other domains of cognition, such as response selection.