Serial Dependence in Perception

Dynamic predictive templates in perception

Hallucinations are vivid and transient experiences of objects, such as images or sounds, that occur in the absence of a corresponding stimulus.1,2,3,4,5,6,7,8,9 To understand the neurocomputational mechanisms of hallucinations, cognitive neuroscience has focused on experiments that induce false alarms (FAs) in healthy participants,1,2,3,4,5,9 psychosis-prone individuals,1,3,4 and patients diagnosed with schizophrenia.5 FAs occur when participants make decisions about difficult-to-detect stimuli and indicate the presence of a signal that was, in fact, not presented. Since FAs are, at heart, reports, they must meet two criteria to serve as an experimental proxy for hallucinations: first, FAs should reflect perceptual states that are characterized by specific contents10,11,12 (criterion 1). Second, FAs should occur on a timescale compatible with the temporal dynamics of hallucinations13,14 (criterion 2). In this work, we combined a classification image approach15 with hidden Markov models16 to show that FAs can match the perceptual and temporal characteristics of hallucinations. We asked healthy human participants to discriminate visual stimuli from noise and found that FAs were more likely to occur during an internal mode of sensory processing, a minute-long state of the brain during which perception is strongly biased toward previous experiences17 (serial dependency). Our results suggest that hallucinations are driven by dynamic predictive templates that transform noise into transient, coherent, and meaningful perceptual experiences.

Opposing serial effects of stimulus and choice in speech perception scale with context variability

In this study, we investigated serial effects on the perception of auditory vowel stimuli across three experimental setups with different degrees of context variability. Aligning with recent findings in visual perception, our results confirm the existence of two distinct processes in serial dependence: a repulsive sensory effect coupled with an attractive decisional effect. Importantly, our study extends these observations to the auditory domain, demonstrating parallel serial effects in audition. Furthermore, we uncover context variability effects, revealing a linear pattern for the repulsive perceptual effect and a quadratic pattern for the attractive decisional effect. These findings support the presence of adaptive sensory mechanisms underlying the repulsive effects, while higher-level mechanisms appear to govern the attractive decisional effect. The study provides valuable insights into the interplay of attractive and repulsive serial effects in auditory perception and contributes to our understanding of the underlying mechanisms.

Multisensory perceptual and causal inference is largely preserved in medicated post-acute individuals with schizophrenia

Hallucinations and perceptual abnormalities in psychosis are thought to arise from imbalanced integration of prior information and sensory inputs. We combined psychophysics, Bayesian modeling, and electroencephalography (EEG) to investigate potential changes in perceptual and causal inference in response to audiovisual flash-beep sequences in medicated individuals with schizophrenia who exhibited limited psychotic symptoms. Seventeen participants with schizophrenia and 23 healthy controls reported either the number of flashes or the number of beeps of audiovisual sequences that varied in their audiovisual numeric disparity across trials. Both groups balanced sensory integration and segregation in line with Bayesian causal inference rather than resorting to simpler heuristics. Both also showed comparable weighting of prior information regarding the signals' causal structure, although the schizophrenia group slightly overweighted prior information about the number of flashes or beeps. At the neural level, both groups computed Bayesian causal inference through dynamic encoding of independent estimates of the flash and beep counts, followed by estimates that flexibly combine audiovisual inputs. Our results demonstrate that the core neurocomputational mechanisms for audiovisual perceptual and causal inference in number estimation tasks are largely preserved in our limited sample of medicated post-acute individuals with schizophrenia. Future research should explore whether these findings generalize to unmedicated patients with acute psychotic symptoms.

A Bayesian inference model can predict the effects of attention on the serial dependence in heading estimation from optic flow

It has been demonstrated that observers can accurately estimate their self-motion direction (i.e., heading) from optic flow, which can be affected by attention. However, it remains unclear how attention affects the serial dependence in the estimation. In the current study, participants conducted two experiments. The results showed that the estimation accuracy decreased when attentional resources allocated to the heading estimation task were reduced. Additionally, the estimates of currently presented headings were biased toward the headings of previously seen headings, showing serial dependence. Especially, this effect decreased (increased) when the attentional resources allocated to the previously (currently) seen headings were reduced. Furthermore, importantly, we developed a Bayesian inference model, which incorporated attention-modulated likelihoods and qualitatively predicted changes in the estimation accuracy and serial dependence. In summary, the current study shows that attention affects the serial dependence in heading estimation from optic flow and reveals the Bayesian computational mechanism behind the heading estimation.

The impact of task measurements on sequential dependence: a comparison between temporal reproduction and discrimination tasks

Decisions about a current stimulus are influenced by previously encountered stimuli, leading to sequential bias. However, the specific processing levels at which serial dependence emerges remain unclear. Despite considerable evidence pointing to contributions from perceptual and post-perceptual processes, as well as response carryover effects impacting subsequent judgments, research into how different task measurements affect sequential dependencies is limited. To address this gap, the present study investigated the role of task type in shaping sequential effects in time perception, employing a random-dot kinematogram (RDK) in a post-cue paradigm. Participants had to remember both the duration and the direction of the RDK movement and perform the task based on a post-cue, which was equally likely to be direction or duration. To delineate the task type, we employed the temporal bisection task in Experiment 1 and the duration reproduction task in Experiment 2. Both experiments revealed a significant sequential bias: durations were perceived as longer following longer previous durations, and vice versa. Intriguingly, the sequential effect was enhanced in the reproduction task following the same reproduction task (Experiment 2), but did not show significant variation by the task type in the bisection task (Experiment 1). Moreover, comparable response carryover effects were observed across two experiments. We argue that the differential impacts of task types on sequential dependence lies in the involvement of memory reactivation process in the decision stage, while the post-decision response carryover effect may reflect the assimilation by subjective, rather than objective, durations, potentially linking to the sticky pacemaker rate and/or decisional inertia.

Connecting past and present

A neural signature of serial dependence has been found, which mirrors the attractive bias of visual information seen in behavioral experiments.

Serial dependence: A matter of memory load

In serial dependence, perceptual decisions are biased towards stimuli encountered in the recent past. Here, we investigate whether and how serial dependence is affected by the availability of visual working memory (VWM) resources. In two experiments, participants reproduced the orientation of a series of stimuli. On alternating trials, we included an additional VWM task with randomly varying levels of load. Serial dependence was not only affected by the additional load task but also clearly modulated by the level of load: a high load in the previous trial reduced serial dependence while a high load in the present increased it. These results were independent of the effects of VWM load on the precision of reproduction responses. Our findings provide new insights into the mechanisms that may regulate serial dependence, revealing its intimate link with VWM resources.

Significance statement

Our perception, thoughts, and behavior are continuously influenced by recent events. For instance, the way we process and understand current visual information depends on what we have seen in the preceding seconds, a phenomenon known as serial dependence. The precise mechanisms and factors involved in serial dependence are still unclear. Here, we demonstrated that working memory resources are a crucial component. Specifically, when we are currently experiencing a heavy memory load, the influence of prior stimuli becomes stronger. Conversely, when prior stimuli were shown under a high memory load, their influence was reduced. These findings highlight the importance of working memory resources in shaping our interpretation of the present based on the recent past.

Eyes on the past: Gaze stability differs between temporal expectation and temporal attention

Temporal expectation and temporal attention distinctly improve performance and gaze stability, and interact at the behavioral and neural levels. Foreperiod-the interval between the preparatory signal and stimulus onset-facilitates temporal expectation. Preceding foreperiod-the foreperiod in the previous trial-modulates expectation at behavioral and oculomotor levels. Here, we investigated whether preceding foreperiod guides temporal attention. Regardless of the preceding foreperiod, temporal attention improved performance, particularly at early moments,and consistently accelerated gaze stability onset and offset by shifting microsaccade timing. However, only with preceding expected foreperiods, attention inhibited microsaccade rates. Moreover, preceding late foreperiods weakened expectation effects on microsaccade rates, but such a weakening was overridden by attention. Altogether, these findings reveal that the oculomotor system's flexibility does not translate to performance, and suggest that although selection history can be utilized as one of the sources of expectation in subsequent trials, it does not necessarily determine, strengthen, or guide attentional deployment.

Location- and feature-based selection histories make independent, qualitatively distinct contributions to urgent visuomotor performance

Attention mechanisms that guide visuomotor behaviors are classified into three broad types according to their reliance on stimulus salience, current goals, and selection histories (i.e., recent experience with events of many sorts). These forms of attentional control are clearly distinct and multifaceted, but what is largely unresolved is how they interact dynamically to determine impending visuomotor choices. To investigate this, we trained two macaque monkeys to perform an urgent version of an oddball search task in which a red target appears among three green distracters, or vice versa. By imposing urgency, performance can be tracked continuously as it transitions from uninformed guesses to informed choices, and this, in turn, permits assessment of attentional control as a function of time. We found that the probability of making a correct choice was strongly modulated by the histories of preceding target colors and target locations. Crucially, although both effects were gated by success (or reward), the two variables played dynamically distinct roles: whereas location history promoted an early motor bias, color history modulated the later perceptual evaluation. Furthermore, target color and location influenced performance independently of each other. The results show that, when combined, selection histories can give rise to enormous swings in visuomotor performance even in simple tasks with highly discriminable stimuli.

Tactile adaptation to orientation produces a robust tilt aftereffect and exhibits crossmodal transfer when tested in vision

Orientation processing is one of the most fundamental functions in both visual and somatosensory perception. Converging findings suggest that orientation processing in both modalities is closely linked: somatosensory neurons share a similar orientation organisation as visual neurons, and the visual cortex has been found to be heavily involved in tactile orientation perception. Hence, we hypothesized that somatosensation would exhibit a similar orientation adaptation effect, and this adaptation effect would be transferable between the two modalities, considering the above-mentioned connection. The tilt aftereffect (TAE) is a demonstration of orientation adaptation and is used widely in behavioural experiments to investigate orientation mechanisms in vision. By testing the classic TAE paradigm in both tactile and crossmodal orientation tasks between vision and touch, we were able to show that tactile perception of orientation shows a very robust TAE, similar to its visual counterpart. We further show that orientation adaptation in touch transfers to produce a TAE when tested in vision, but not vice versa. Additionally, when examining the test sequence following adaptation for serial effects, we observed another asymmetry between the two conditions where the visual test sequence displayed a repulsive intramodal serial dependence effect while the tactile test sequence exhibited an attractive serial dependence. These findings provide concrete evidence that vision and touch engage a similar orientation processing mechanism. However, the asymmetry in the crossmodal transfer of TAE and serial dependence points to a non-reciprocal connection between the two modalities, providing further insights into the underlying processing mechanism.

Response boosts serial dependence in the numerosity estimation task

Perceptions of current stimuli are sometimes biased toward or away from past perceptions. This phenomenon is called serial dependence. However, the strength of the effect of past responses on serial dependence has not been fully elucidated. We conducted experiments with a task in which participants estimated the number of dot arrays (numerosity estimation task) and directly compared whether the strength of serial dependence changed in the numerosity estimation task when participants responded or did not respond in the immediately preceding trial. We also examined whether the strength of serial dependence affected the accuracy of the numerosity estimation. We found that attractive serial dependence was stronger when participants responded in the immediately preceding trial than when they only saw the stimulus. The results suggest that the information from the previous stimulus must reach the higher-level processes associated with perceptual decisions to influence the estimation of the current stimulus. However, it is possible that the results of this study are specific to tasks in which participants respond with numeric symbols. The magnitude of the serial dependence effect was not observed to affect numerosity estimation performance, and no evidence was found that serial dependence enhances accuracy in the numerosity estimation task.