Serial dependence generalizes across different stimulus formats, but not different sensory modalities

Intact Serial Dependence in Schizophrenia: Evidence from an Orientation Adjustment Task

BACKGROUND AND HYPOTHESIS

For a long time, it was proposed that schizophrenia (SCZ) patients rely more on sensory input and less on prior information, potentially leading to reduced serial dependence-ie, a reduced influence of prior stimuli in perceptual tasks. However, existing evidence is constrained to a few paradigms, and whether reduced serial dependence reflects a general characteristic of the disease remains unclear.

STUDY DESIGN

We investigated serial dependence in 26 SCZ patients and 27 healthy controls (CNT) to evaluate the influence of prior stimuli in a classic visual orientation adjustment task, a paradigm not previously tested in this context.

STUDY RESULTS

As expected, the CNT group exhibited clear serial dependence, with systematic biases toward the orientation of stimuli shown in the preceding trials. Serial dependence in SCZ patients was largely comparable to that in the CNT group.

CONCLUSIONS

These findings challenge the prevailing notion of reduced serial dependence in SCZ, suggesting that observed differences between healthy CNT and patients may depend on aspects of perceptual or cognitive processing that are currently not understood.

Response boosts serial dependence in the numerosity estimation task

Perceptions of current stimuli are sometimes biased toward or away from past perceptions. This phenomenon is called serial dependence. However, the strength of the effect of past responses on serial dependence has not been fully elucidated. We conducted experiments with a task in which participants estimated the number of dot arrays (numerosity estimation task) and directly compared whether the strength of serial dependence changed in the numerosity estimation task when participants responded or did not respond in the immediately preceding trial. We also examined whether the strength of serial dependence affected the accuracy of the numerosity estimation. We found that attractive serial dependence was stronger when participants responded in the immediately preceding trial than when they only saw the stimulus. The results suggest that the information from the previous stimulus must reach the higher-level processes associated with perceptual decisions to influence the estimation of the current stimulus. However, it is possible that the results of this study are specific to tasks in which participants respond with numeric symbols. The magnitude of the serial dependence effect was not observed to affect numerosity estimation performance, and no evidence was found that serial dependence enhances accuracy in the numerosity estimation task.

Serial Dependence in Perception

Much evidence has shown that perception is biased towards previously presented similar stimuli, an effect recently termed serial dependence. Serial dependence affects nearly every aspect of perception, often causing gross perceptual distortions, especially for weak and ambiguous stimuli. Despite unwanted side-effects, empirical evidence and Bayesian modeling show that serial dependence acts to improve efficiency and is generally beneficial to the system. Consistent with models of predictive coding, the Bayesian priors of serial dependence are generated at high levels of cortical analysis, incorporating much perceptual experience, but feed back to lower sensory areas. These feedback loops may drive oscillations in the alpha range, linked strongly with serial dependence. The discovery of top-down predictive perceptual processes is not new, but the new, more quantitative approach characterizing serial dependence promises to lead to a deeper understanding of predictive perceptual processes and their underlying neural mechanisms.

Serial dependencies between form orientation and motion direction are asymmetric

Much work has been done to uncover the mechanisms underlying form and motion information integration. However, no study examined the symmetry of the integration of form and motion across the temporal domain (i.e., serial dependence). In Experiment 1, we presented form and motion displays sequentially. In the form displays, dot pairs were oriented toward one screen position, indicating the form orientation; in the motion displays, dots moved radially outward. Their motion trajectories were oriented toward one screen position, indicating the motion direction. In each trial, participants reported their perceived form orientation after the form display or their perceived motion direction after the motion display. We found that the current trial's perceived motion direction was biased toward the previous trial's form orientation and vice versa, indicating serial dependencies between form orientation and motion direction. In Experiment 2, we changed the form and motion displays' reliability by varying the two displays' dot densities. The results showed that the serial dependence of form orientation on motion direction perception decreased only with increasing the current motion display's reliability; neither the reliability of the previous motion display nor that of the current form display significantly affected the serial dependence of motion direction on form orientation perception. Hence, serial dependencies between form orientation and motion direction were asymmetric. Our across-temporal integrations between form and motion, together with the simultaneous integration of form and motion revealed in the previous studies, depict a comprehensive mechanism underlying the integration of the two pieces of information.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Serial dependence in estimates of the monetary value of coins

Perceptions of current stimuli are sometimes biased toward or away from past perceptions. This phenomenon is called serial dependence. However, it remains unclear whether serial dependence originates from lower-order perceptual processing, higher-order perceptual processing or cognitive processing. We examined the effects of serial dependence when participants estimated the total number of coins or the monetary value of coins displayed and found attractive effects in both tasks. The attractive effect observed in the value estimation task suggests that serial dependence occurs through higher-order cognitive processes during calculation. We also examined the effect of response history (i.e., the responses of participants on previous trials), with multiple regression analyses that simultaneously evaluated the effects of the previous stimuli and responses. In both number and value estimation tasks, the immediately prior response had an attractive effect on current responses, while the immediately prior stimuli exerted a repulsive effect. This pattern suggests that the attractive serial dependence found in the single regression analysis was due to the correlation between stimulus and response in the previous trials and that the effect of past stimuli per se may be an adaptation that increases sensitivity to current stimuli.

Causal evidence for the higher-order origin of serial dependence suggests a multi-area account

There has been recent interest in whether serial dependence, a temporal bias in working memory and perception, is generated by higher-order cognitive or sensory areas. Based on the literature suggesting prefrontal areas combined with a recent article by de Azevedo Neto & Bartels (1), providing causal evidence for serial dependence relying on premotor cortex but not the visual area hV5/MT+, I here argue for a higher-order and multi-area origin of serial dependence.

The effect of abstract representation and response feedback on serial dependence in numerosity perception

Serial dependence entails an attractive bias based on the recent history of stimulation, making the current stimulus appear more similar to the preceding one. Although serial dependence is ubiquitous in perception, its nature and mechanisms remain unclear. Here, in two independent experiments, we test the hypothesis that this bias originates from high-level processing stages at the level of abstract information processing (Exp. 1) or at the level of judgment (Exp. 2). In Experiment 1, serial dependence was induced by a task-irrelevant "inducer" stimulus in a numerosity discrimination task, similarly to previous studies. Importantly, in this experiment, the inducers were either arrays of dots similar to the task-relevant stimuli (e.g., 12 dots), or symbolic numbers (e.g., the numeral "12"). Both dots and symbol inducers successfully yielded attractive serial dependence biases, suggesting that abstract information about an image is sufficient to bias the perception of the current stimulus. In Experiment 2, participants received feedback about their responses in each trial of a numerosity estimation task, which was designed to assess whether providing external information about the accuracy of judgments would modulate serial dependence. Providing feedback significantly increased the attractive serial dependence effect, suggesting that external information at the level of judgment may modulate the weight of past perceptual information during the processing of the current image. Overall, our results support the idea that, although serial dependence may operate at a perceptual level, it originates from high-level processing stages at the level of abstract information processing and at the level of judgment.

Serial dependence for oculomotor control depends on early sensory signals

To create an accurate percept of the world, the visual system relies on past experience and prior assumptions.¹ For example, although the retinal projection of an object moving in depth changes drastically, we still perceive the object at a constant size and velocity.^2,3 Consequently, if we see the same object with a constant retinal size at two different depth levels, the perceived size differs (illustrated by the Ponzo illusion). Past experience also directly influences perceptual judgments, an effect known as serial dependence.^4,5 Such sequential effects have also been reported for oculomotor behavior, even on the trial-by-trial level.^6-10 An integration of past experiences seems like a smart and sophisticated mechanism to reduce uncertainty and improve behavior in a world full of statistical regularities. By leveraging the Ponzo illusion to dissociate perceived size and speed from retinal signals, we show that serial-dependence effects for oculomotor control are mediated by retinal error signals. These sequential effects likely take place in early sensory processing because they transfer to different visual stimuli. In contrast to recently reported history effects for perceptual decisions,¹¹ sequential effects for oculomotor control deviate from perceptual mechanisms by not integrating spatial context and by ignoring size and velocity constancy. Although this dissociation might appear suboptimal, we argue that this effect reveals the different goals of the oculomotor and perceptual systems. The oculomotor system tries to reduce retinal error signals to bring and keep the target close to the fovea, whereas the visual system interprets retinal input to achieve an accurate representation of the world.¹².

Deprivation of Auditory Experience Influences Numerosity Discrimination, but Not Numerosity Estimation

Number sense is the ability to estimate the number of items, and it is common to many species. Despite the numerous studies dedicated to unveiling how numerosity is processed in the human brain, to date, it is not clear whether the representation of numerosity is supported by a single general mechanism or by multiple mechanisms. Since it is known that deafness entails a selective impairment in the processing of temporal information, we assessed the approximate numerical abilities of deaf individuals to disentangle these two hypotheses. We used a numerosity discrimination task (2AFC) and an estimation task, in both cases using sequential (temporal) or simultaneous (spatial) stimuli. The results showed a selective impairment of the deaf participants compared with the controls (hearing) in the temporal numerosity discrimination task, while no difference was found to discriminate spatial numerosity. Interestingly, the deaf and hearing participants did not differ in spatial or temporal numerosity estimation. Overall, our results suggest that the deficit in temporal processing induced by deafness also impacts perception in other domains such as numerosity, where sensory information is conveyed in a temporal format, which further suggests the existence of separate mechanisms subserving the processing of temporal and spatial numerosity.

Serial Dependence of Emotion Within and Between Stimulus Sensory Modalities

How we perceive the world is not solely determined by what we sense at a given moment in time, but also by what we processed recently. Here we investigated whether such serial dependencies for emotional stimuli transfer from one modality to another. Participants were presented a random sequence of emotional sounds and images and instructed to rate the valence and arousal of each stimulus (Experiment 1). For both ratings, we conducted an intertrial analysis, based on whether the rating on the previous trial was low or high. We found a positive serial dependence for valence and arousal regardless of the stimulus modality on two consecutive trials. In Experiment 2, we examined whether passively perceiving a stimulus is sufficient to induce a serial dependence. In Experiment 2, participants were instructed to rate the stimuli only on active trials and not on passive trials. The participants were informed that the active and passive trials were presented in alternating order, so that they were able to prepare for the task. We conducted an intertrial analysis on active trials, based on whether the rating on the previous passive trial (determined in Experiment 1) was low or high. For both ratings, we again observed positive serial dependencies regardless of the stimulus modality. We conclude that the emotional experience triggered by one stimulus affects the emotional experience for a subsequent stimulus regardless of their sensory modalities, that this occurs in a bottom-up fashion, and that this can be explained by residual activation in the emotional network in the brain.

Evidence for an A-Modal Number Sense: Numerosity Adaptation Generalizes Across Visual, Auditory, and Tactile Stimuli

Humans and other species share a perceptual mechanism dedicated to the representation of approximate quantities that allows to rapidly and reliably estimate the numerosity of a set of objects: an Approximate Number System (ANS). Numerosity perception shows a characteristic shared by all primary visual features: it is susceptible to adaptation. As a consequence of prolonged exposure to a large/small quantity (“adaptor”), the apparent numerosity of a subsequent (“test”) stimulus is distorted yielding a robust under- or over-estimation, respectively. Even if numerosity adaptation has been reported across several sensory modalities (vision, audition, and touch), suggesting the idea of a central and a-modal numerosity processing system, evidence for cross-modal effects are limited to vision and audition, two modalities that are known to preferentially encode sensory stimuli in an external coordinate system. Here we test whether numerosity adaptation for visual and auditory stimuli also distorts the perceived numerosity of tactile stimuli (and vice-versa) despite touch being a modality primarily coded in an internal (body-centered) reference frame. We measured numerosity discrimination of stimuli presented sequentially after adaptation to series of either few (around 2 Hz; low adaptation) or numerous (around 8 Hz; high adaptation) impulses for all possible combinations of visual, auditory, or tactile adapting and test stimuli. In all cases, adapting to few impulses yielded a significant overestimation of the test numerosity with the opposite occurring as a consequence of adaptation to numerous stimuli. The overall magnitude of adaptation was robust (around 30%) and rather similar for all sensory modality combinations. Overall, these findings support the idea of a truly generalized and a-modal mechanism for numerosity representation aimed to process numerical information independently from the sensory modality of the incoming signals.

Serial dependence in time and numerosity perception is dimension-specific

The perception of a visual event (e.g., a flock of birds) at the present moment can be biased by a previous perceptual experience (e.g., the perception of an earlier flock). Serial dependence is a perceptual bias whereby a current stimulus appears more similar to a previous one than it actually is. Whereas serial dependence emerges within several visual stimulus dimensions, whether it could simultaneously operate across different dimensions of the same stimulus (e.g., the numerosity and the duration of a visual pattern) remains unclear. Here we address this question by assessing the presence of serial dependence across duration and numerosity, two stimulus dimensions that are often associated and can bias each other. Participants performed either a duration or a numerosity discrimination task, in which they compared a constant reference with a variable test stimulus, varying along the task-relevant dimension (either duration or numerosity). Serial dependence was induced by a task-irrelevant inducer, that is, a stimulus presented before the reference and always varying in both duration and numerosity. The results show systematic serial dependencies only within the task-relevant stimulus dimension, that is, stimulus numerosity affects numerosity perception only, and duration affects duration perception only. Additionally, at least in the numerosity condition, the task-irrelevant dimension of the inducer (duration) had an opposite, repulsive effect. These findings thus show that attractive serial dependence operates in a highly specific fashion and does not transfer across different stimulus dimensions. Instead, the repulsive influence, possibly reflecting perceptual adaptation, can transfer from one dimension to another.

Serial dependence of facial identity reflects high-level face coding

Serial dependence of facial identity is a type of bias where the perceived identity of a face is biased towards a previously presented face. There are individual differences in serial dependence strength and tuning (how the strength varies depending on stimuli similarity), and previous research has shown that both stronger and more narrowly tuned serial dependence of facial identity is associated with better face recognition abilities. These results are consistent with the idea that this bias plays a functional role in face perception. It is important, therefore, to determine whether serial dependence of facial identity reflects a high-level face-coding mechanism acting on the identity of a face or instead predominantly reflects a bias in low-level features, which are also subject to serial dependence. We first sought evidence that serial dependence of facial identity survived changes in low-level visual features, by varying face viewpoint between successive stimuli. We found that serial dependence persisted across changes in viewpoint, arguing against an entirely low-level locus for this bias. We next tested whether the bias was affected by inversion, as sensitivity to inversion is argued to be a characteristic of high-level face-selective processing. Serial dependence was stronger and more narrowly tuned for upright than inverted faces. Taken together, our results are consistent with the view that serial dependence of facial identity affects high-level visual representations and may reflect a face-coding mechanism that is operating at the level of facial identity.

Disentangling feedforward versus feedback processing in numerosity representation

Numerosity is a fundamental aspect of the external environment, needed to guide our behavior in an effective manner. Previous studies show that numerosity processing involves at least two temporal stages (~100 and ~150 msec after stimulus onset) in early visual cortex. One possibility is that the two stages reflect an initial feedforward processing followed by feedback signals from higher-order cortical areas that underlie segmentation of visual inputs into perceptual units that define numerosity. Alternatively, multiple stages of feedforward processing might progressively refine the input leading to the segmented representation. Here, we distinguish these two hypotheses by exploiting the connectedness illusion (i.e., the systematic underestimation of pairwise-connected dots), backward masking (to suppress feedback signals), and serial dependence (i.e., a perceptual bias making a stimulus appear to be more similar to its preceding one). Our results show that a connected dot array biases the numerosity representation of the subsequent dot array based on its illusory perception, irrespective of whether it is visible or suppressed by masking. These findings demonstrate that feedback processing is not strictly necessary for the perceptual segmentation that gives rise to perceived numerosity, and instead suggest that different stages of feedforward activity presumably carrying low and high spatial frequency information are sufficient to create a numerosity representation in early visual areas.

Adaptation to hand-tapping affects sensory processing of numerosity directly: evidence from reaction times and confidence

Like most perceptual attributes, the perception of numerosity is susceptible to adaptation, both to prolonged viewing of spatial arrays and to repeated motor actions such as hand-tapping. However, the possibility has been raised that adaptation may reflect response biases rather than modification of sensory processing. To disentangle these two possibilities, we studied visual and motor adaptation of numerosity perception while measuring confidence and reaction times. Both sensory and motor adaptation robustly distorted numerosity estimates, and these shifts in perceived numerosity were accompanied by similar shifts in confidence and reaction-time distributions. After adaptation, maximum uncertainty and slowest response-times occurred at the point of subjective (rather than physical) equality of the matching task, suggesting that adaptation acts directly on the sensory representation of numerosity, before the decisional processes. On the other hand, making reward response-contingent, which also caused robust shifts in the psychometric function, caused no significant shifts in confidence or reaction-time distributions. These results reinforce evidence for shared mechanisms that encode the quantity of both internally and externally generated events, and advance a useful general technique to test whether contextual effects like adaptation and serial dependence really affect sensory processing.

Neural Dynamics of Serial Dependence in Numerosity Perception

Serial dependence—an attractive perceptual bias whereby a current stimulus is perceived to be similar to previously seen ones—is thought to represent the process that facilitates the stability and continuity of visual perception. Recent results demonstrate a neural signature of serial dependence in numerosity perception, emerging very early in the time course during perceptual processing. However, whether such a perceptual signature is retained after the initial processing remains unknown. Here, we address this question by investigating the neural dynamics of serial dependence using a recently developed technique that allowed a reactivation of hidden memory states. Participants performed a numerosity discrimination task during EEG recording, with task-relevant dot array stimuli preceded by a task-irrelevant stimulus inducing serial dependence. Importantly, the neural network storing the representation of the numerosity stimulus was perturbed (or pinged) so that the hidden states of that representation can be explicitly quantified. The results first show that a neural signature of serial dependence emerges early in the brain signals, starting soon after stimulus onset. Critical to the central question, the pings at a later latency could successfully reactivate the biased representation of the initial stimulus carrying the signature of serial dependence. These results provide one of the first pieces of empirical evidence that the biased neural representation of a stimulus initially induced by serial dependence is preserved throughout a relatively long period.

Visual serial dependence in an audiovisual stimulus

Serial dependence is a phenomenon that biases the perception of features or objects systematically toward sensory input from the recent past (Fischer & Whitney, 2014). There is an active debate whether this effect is rooted directly in perception or reflects biases in decision making. We investigated serial dependence across three experiments by manipulating the decision made on each trial. A multimodal audiovisual stimulus comprising a Gabor and a vowel sound was presented repeatedly. On each trial, participants reported either the Gabor orientation or the vowel sound. Participants either ignored one modality (Experiment 1) or attended to both modalities (Experiments 2 and 3). In Experiments 2 and 3, the response task was randomized to prevent anticipating which modality to respond to until the response phase. In Experiment 3, no-response trials were additionally interleaved. Results across the three experiments demonstrated serial dependence only when participants reported the visual modality. Serial dependence was also present in visual reports when participants completed auditory reports or made no reports on previous trials. The previous stimulus alone was enough to elicit an effect. Serial dependence is unlikely to be an effect of the previous decision on the stimulus, but rather an effect of perceiving the previous stimulus.