Linear Summation of Repulsive and Attractive Serial Dependencies: Orientation and Motion Dependencies Sum in Motion Perception

Tactile adaptation to orientation produces a robust tilt aftereffect and exhibits crossmodal transfer when tested in vision

Orientation processing is one of the most fundamental functions in both visual and somatosensory perception. Converging findings suggest that orientation processing in both modalities is closely linked: somatosensory neurons share a similar orientation organisation as visual neurons, and the visual cortex has been found to be heavily involved in tactile orientation perception. Hence, we hypothesized that somatosensation would exhibit a similar orientation adaptation effect, and this adaptation effect would be transferable between the two modalities, considering the above-mentioned connection. The tilt aftereffect (TAE) is a demonstration of orientation adaptation and is used widely in behavioural experiments to investigate orientation mechanisms in vision. By testing the classic TAE paradigm in both tactile and crossmodal orientation tasks between vision and touch, we were able to show that tactile perception of orientation shows a very robust TAE, similar to its visual counterpart. We further show that orientation adaptation in touch transfers to produce a TAE when tested in vision, but not vice versa. Additionally, when examining the test sequence following adaptation for serial effects, we observed another asymmetry between the two conditions where the visual test sequence displayed a repulsive intramodal serial dependence effect while the tactile test sequence exhibited an attractive serial dependence. These findings provide concrete evidence that vision and touch engage a similar orientation processing mechanism. However, the asymmetry in the crossmodal transfer of TAE and serial dependence points to a non-reciprocal connection between the two modalities, providing further insights into the underlying processing mechanism.

Opposing serial dependencies revealed for sequences of auditory emotional stimuli

Our percept of the world is not solely determined by what we perceive and process at a given moment in time, but also depends on what we processed recently. In the present study, we investigate whether the perceived emotion of a spoken sentence is contingent upon the emotion of an auditory stimulus on the preceding trial (i.e., serial dependence). Thereto, participants were exposed to spoken sentences that varied in emotional affect by changing the prosody that ranged from 'happy' to 'fearful'. Participants were instructed to rate the emotion. We found a positive serial dependence for emotion processing whereby the perceived emotion was biased towards the emotion on the preceding trial. When we introduced 'no-go' trials (i.e., no rating was required), we found a negative serial dependence when participants knew in advance to withhold their response on a given trial (Experiment 2) and a positive serial dependence when participants received the information to withhold their response after the stimulus presentation (Experiment 3). We therefore established a robust serial dependence for emotion processing in speech and introduce a methodology to disentangle perceptual from post-perceptual processes. This approach can be applied to the vast majority of studies investigating sequential dependencies to separate positive from negative serial dependence.

Conflicts between short- and long-term experiences affect visual perception through modulating sensory or motor response systems: Evidence from Bayesian inference models

The independent effects of short- and long-term experiences on visual perception have been discussed for decades. However, no study has investigated whether and how these experiences simultaneously affect our visual perception. To address this question, we asked participants to estimate their self-motion directions (i.e., headings) simulated from optic flow, in which a long-term experience learned in everyday life (i.e., straight-forward motion being more common than lateral motion) plays an important role. The headings were selected from three distributions that resembled a peak, a hill, and a flat line, creating different short-term experiences. Importantly, the proportions of headings deviating from the straight-forward motion gradually increased in the peak, hill, and flat distributions, leading to a greater conflict between long- and short-term experiences. The results showed that participants biased their heading estimates towards the straight-ahead direction and previously seen headings, which increased with the growing experience conflict. This suggests that both long- and short-term experiences simultaneously affect visual perception. Finally, we developed two Bayesian models (Model 1 vs. Model 2) based on two assumptions that the experience conflict altered the likelihood distribution of sensory representation or the motor response system. The results showed that both models accurately predicted participants' estimation biases. However, Model 1 predicted a higher variance of serial dependence compared to Model 2, while Model 2 predicted a higher variance of the bias towards the straight-ahead direction compared to Model 1. This suggests that the experience conflict can influence visual perception by affecting both sensory and motor response systems. Taken together, the current study systematically revealed the effects of long- and short-term experiences on visual perception and the underlying Bayesian processing mechanisms.

Serial dependence in timing at the perceptual level being modulated by working memory

Recent experiences bias the perception of following stimuli, as has been verified in various kinds of experiments in visual perception. This phenomenon, known as serial dependence, may reflect mechanisms to maintain perceptual stability. In the current study, we examined several key properties of serial dependence in temporal perception. Firstly, we examined the source of the serial dependence effect in temporal perception. We found that perception without motor reproduction is sufficient to induce the sequential effect; motor reproduction caused a stronger effect and is achieved by biasing the perception of the future target duration rather than directly influencing the subsequent movement. Secondly, we ask how working memory influences serial dependence in a temporal reproduction task. By varying the delay time between standard duration and the reproduction, we showed that the strength of serial dependence is enhanced as the delay increased. Those features of serial dependence are consistent with what has been observed in visual perceptual tasks, for example, orientation perception or location perception. The similarities between the visual and the timing tasks may suggest a similar neural coding mechanism of magnitude between the visual stimuli and the duration.

Serial dependence bias can predict the overall estimation error in visual perception

Although visual feature estimations are accurate and precise, overall estimation errors (i.e., the difference between estimates and actual values) tend to show systematic patterns. For example, estimates of orientations are systematically biased away from horizontal and vertical orientations, showing an oblique illusion. Additionally, many recent studies have demonstrated that estimations of current visual features are systematically biased toward previously seen features, showing a serial dependence. However, no study examined whether the overall estimation errors were correlated with the serial dependence bias. To address this question, we enrolled three groups of participants to estimate orientation, motion speed, and point-light-walker direction. The results showed that the serial dependence bias explained over 20% of overall estimation errors in the three tasks, indicating that we could use the serial dependence bias to predict the overall estimation errors. The current study first demonstrated that the serial dependence bias was not independent from the overall estimation errors. This finding could inspire researchers to investigate the neural bases underlying the visual feature estimation and serial dependence.

Tuning perception and decisions to temporal context

Recent work suggests that serial dependence, where perceptual decisions are biased toward previous stimuli, arises from the prior that sensory input is temporally correlated. However, existing studies have mostly used random stimulus sequences that do not involve such temporal consistencies. Here, we manipulated the temporal statistics of visual stimuli to examine the role of true temporal correlations in serial dependence. In two experiments, observers reproduced the orientation of the last stimulus in a sequence, while we varied temporal correlations in the stimulus features at two timescales: stimulus history within the trial and decision history across trials. We found a clear dissociation: increasing temporal correlation in the stimulus history led to adaptation-like repulsive biases, whereas increasing temporal correlation in the decision history reduced attractive biases. Thus, we suggest that temporal correlation enhances the discriminative ability of the visual system, revealing the fundamental role of the broader temporal context.

Task feedback suggests a post-perceptual component to serial dependence

Decisions across a range of perceptual tasks are biased toward past stimuli. Such serial dependence is thought to be an adaptive low-level mechanism that promotes perceptual stability across time. However, recent studies suggest post-perceptual mechanisms may also contribute to serially biased responses, calling into question a single locus of serial dependence and the nature of integration of past and present sensory inputs. We measured serial dependence in the context of a three-dimensional (3D) motion perception task where uncertainty in the sensory information varied substantially from trial to trial. We found that serial dependence varied with stimulus properties that impact sensory uncertainty on the current trial. Reduced stimulus contrast was associated with an increased bias toward the stimulus direction of the previous trial. Critically, performance feedback, which reduced sensory uncertainty, abolished serial dependence. These results provide clear evidence for a post-perceptual locus of serial dependence in 3D motion perception and support the role of serial dependence as a response strategy in the face of substantial sensory uncertainty.

Stable perceptual phenotype of the magnitude of history biases even in the face of global task complexity

According to a Bayesian framework, visual perception requires active interpretation of noisy sensory signals in light of prior information. One such mechanism, serial dependence, is thought to promote perceptual stability by assimilating current percepts with recent stimulus history. Combining a delayed orientation-adjustment paradigm with predictable (study 1) or unpredictable (study 2) task structure, we test two key predictions of this account in a novel context: first, that serial dependence should persist even in variable environments, and, second, that, within a given observer and context, this behavioral bias should be stable from one occasion to the next. Relying on data of 41 human volunteers and two separate experimental sessions, we confirm both hypotheses. Group-level, attractive serial dependence remained strong even in the face of volatile settings with multiple, unpredictable types of tasks, and, despite considerable interindividual variability, within-subject patterns of attractive and repulsive stimulus-history biases were highly stable from one experimental session to the next. In line with the hypothesized functional role of serial dependence, we propose that, together with previous work, our findings suggest the existence of a more general individual-specific fingerprint with which the past shapes current perception. Congruent with the Bayesian account, interindividual differences may then result from differential weighting of sensory evidence and prior information.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Effects of satisfying and violating expectations on serial dependence

Serial dependence refers to the phenomenon that observers tend to report stimuli as being more similar to previous stimuli than they really are (attractive dependence) or, in some cases, as more different than they really are (repulsive dependence). Numerous experiments have demonstrated serial dependence for a range of modalities and stimulus features, highlighting the role of bottom-up sensory interactions. However, comparatively less research has focused on how higher-level cognitive factors, such as expectations, might influence serial dependence. Here, we manipulated expectations by having observers respond to target luminance gratings that occurred at the end of a sequence of non-target gratings. The sequence either rotated predictably (inducing an expectation), varied randomly (inducing no expectation), or rotated predictably but had a random target orientation (violating expectations). We found that observers produced less errors and indicated less uncertainty in their estimations of expected stimuli but their responses were biased away from the penultimate stimulus in the sequence (repulsive dependence). In contrast, following random sequences, responses showed an attractive bias to the penultimate stimulus in the sequence. Unexpected targets showed a mixture of both biases, such that when targets happened (by chance) to appear as expected, responses were repulsed, but responses to target orientations that more clearly violated expectations were attracted. These results indicate that, whereas attraction to previous stimuli may be a default strategy employed in response to random and unexpected events, certain expectations can reverse the default bias into a repulsive one.

Serial dependence in visual perception: A review

How does the visual system represent continuity in the constantly changing visual input? A recent proposal is that vision is serially dependent: Stimuli seen a moment ago influence what we perceive in the present. In line with this, recent frameworks suggest that the visual system anticipates whether an object seen at one moment is the same as the one seen a moment ago, binding visual representations across consecutive perceptual episodes. A growing body of work supports this view, revealing signatures of serial dependence in many diverse visual tasks. Yet, the variety of disparate findings and interpretations calls for a more general picture. Here, we survey the main paradigms and results over the past decade. We also focus on the challenge of finding a relationship between serial dependence and the concept of "object identity," taking centuries-long history of research into account. Among the seemingly contrasting findings on serial dependence, we highlight common patterns that may elucidate the nature of this phenomenon and attempt to identify questions that are unanswered.

Contextual Lateralization Based on Interaural Level Differences Is Preshaped by the Auditory Periphery and Predominantly Immune Against Sequential Segregation

The perceived azimuth of a target sound is determined by the interaural time difference and the interaural level difference (ILD) and is subject to contextual effects from precursor sounds. This study characterized ILD-based precursor effects (PEs) for high-frequency stimuli in a total of seven normal-hearing listeners. In Experiment 1, precursor and target were band-pass-filtered noises approximately centered at 4 kHz (1.2- and 1-octave bandwidth, respectively) separated by a 10-ms gap. The effects of precursor location (ipsilateral, contralateral, and central) on the perceived target azimuth were measured using a head-pointing task. Relative to control trials without a precursor, ipsilateral precursors biased the perceived target azimuth toward midline (medial bias) and contralateral precursors biased it contralaterally (lateral bias). Central precursors caused a symmetric lateral bias. An auditory periphery model that determines the "internal" ILD at the auditory nerve level, including either realistic efferent compression control or auditory nerve adaptation, explained about 50% of the variance in the PEs. These within-trial PEs were accompanied by an across-trial PE, inducing medial bias. Experiment 2 studied the role of sequential segregation in the within-trial PE by introducing a pitch difference between precursor and target. Segregation conditions caused increased PE for ipsilateral, no effect for contralateral, and either no effect or reduced PE for central precursors. Overall, the ILD-based within-trial PE appears to be preshaped already in the auditory periphery and the mechanism underlying at least the ipsilateral PE appears to be immune against sequential segregation.

Attractive and repulsive effects of sensory history concurrently shape visual perception

Background

Sequential effects of environmental stimuli are ubiquitous in most behavioral tasks involving magnitude estimation, memory, decision making, and emotion. The human visual system exploits continuity in the visual environment, which induces two contrasting perceptual phenomena shaping visual perception. Previous work reported that perceptual estimation of a stimulus may be influenced either by attractive serial dependencies or repulsive aftereffects, with a number of experimental variables suggested as factors determining the direction and magnitude of sequential effects. Recent studies have theorized that these two effects concurrently arise in perceptual processing, but empirical evidence that directly supports this hypothesis is lacking, and it remains unclear whether and how attractive and repulsive sequential effects interact in a trial. Here we show that the two effects concurrently modulate estimation behavior in a typical sequence of perceptual tasks.

Results

We first demonstrate that observers’ estimation error as a function of both the previous stimulus and response cannot be fully described by either attractive or repulsive bias but is instead well captured by a summation of repulsion from the previous stimulus and attraction toward the previous response. We then reveal that the repulsive bias is centered on the observer’s sensory encoding of the previous stimulus, which is again repelled away from its own preceding trial, whereas the attractive bias is centered precisely on the previous response, which is the observer’s best prediction about the incoming stimuli.

Conclusions

Our findings provide strong evidence that sensory encoding is shaped by dynamic tuning of the system to the past stimuli, inducing repulsive aftereffects, and followed by inference incorporating the prediction from the past estimation, leading to attractive serial dependence.

A key role of orientation in the coding of visual motion direction

Despite the fundamental importance of visual motion processing, our understanding of how the brain represents basic aspects of motion is incomplete. While it is generally believed that direction is the main representational feature of motion, motion processing is also influenced by nondirectional orientation signals that are present in most motion stimuli. Here, we aimed to test whether this nondirectional motion axis contributes motion perception even when orientation is completely absent from the stimulus. Using stimuli with and without orientation signals, we found that serial dependence in a simple motion direction estimation task was predominantly determined by the orientation of the previous motion stimulus. Moreover, the observed attraction profiles closely matched the characteristic pattern of serial attraction found in orientation perception. Evidently, the sequential integration of motion signals strongly depends on the orientation of motion, indicating a fundamental role of nondirectional orientation in the coding of visual motion direction.

Cartesian coordinates scaffold stable spatial perception over time

Visual systems exploit temporal continuity principles to achieve stable spatial perception, manifested as the serial dependence and central tendency effects. These effects are posited to reflect a smoothing process whereby past and present information integrates over time to decrease noise and stabilize perception. Meanwhile, the basic spatial coordinate—Cartesian versus polar—that scaffolds the integration process in two-dimensional continuous space remains unknown. The spatial coordinates are largely related to the allocentric and egocentric reference frames and presumably correspond with early and late processing stages in spatial perception. Here, four experiments consistently demonstrate that Cartesian outperforms polar coordinates in characterizing the serial bias—serial dependence and central tendency effect—in two-dimensional continuous spatial perception. The superiority of Cartesian coordinates is robust, independent of task environment (online and offline task), experimental length (short and long blocks), spatial context (shape of visual mask), and response modality (keyboard and mouse). Taken together, the visual system relies on the Cartesian coordinates for spatiotemporal integration to facilitate stable representation of external information, supporting the involvement of allocentric reference frame and top-down modulation in spatial perception over long time intervals.

The test-retest reliability and spatial tuning of serial dependence in orientation perception

Humans perceive objects and scenes consistently, even in situations where visual input is noisy and unstable. One of the mechanisms that underlies this perceptual stability is serial dependence, whereby the perception of objects or features at any given moment is pulled toward what was previously seen. Although recent findings from several studies have reported large individual differences in serial dependence, it is not clear how stable the serial dependence is within an individual. Here, we investigated the stability of serial dependence in orientation perception over two different days within the same observers. In addition, we also examined the visual field location specificity of perceptual serial dependence. On each trial, observers viewed a Gabor patch and then reported its apparent orientation by adjusting the orientation of a bar. For each observer, the Gabor was located in the foveal or peripheral (10° right or left eccentricity) visual field on both days or changed location from day to day. The results showed a very high degree of test-retest reliability in serial dependence measured across days within individual observers. Interestingly, this high within-subject consistency was only found when serial dependence was measured at the same visual field location. These results suggest that individual differences in serial dependence are stable across days, and that the spatiotemporal range in which the previous stimulus assimilates the perception of the current stimulus (the continuity field) may vary across different visual field locations in an observer-specific manner.

Stimulus uncertainty predicts serial dependence in orientation judgements

How is what you see influenced by what you saw? The visual system may use recent perception to inform responses to current stimuli. This can cause the perception of current stimuli to be attracted toward previous observations, an effect termed serial dependence. This misperception might well be useful in a noisy visual environment, where minor image distortions over time may not actually represent meaningful change. Previous work has suggested that Bayesian perceptual inference may underlie serial dependence. For this to be true, the relative uncertainty associated with both prior and current sensory input should be taken into account. In an experiment manipulating the level of noise present in orientation stimuli, we found an effect of current stimulus uncertainty on serial dependence. We found no good evidence for an effect of previous stimulus uncertainty. Our results provide only partial evidence for the Bayesian interpretation of serial dependence. Non-Bayesian models may provide a better account of the phenomenon.

A general serial dependence among various facial traits: Evidence from Markov Chain and derivative of Gaussian

The human vision system can extract a stable representation of the always-changing visual world. However, the mechanism underlying such perceptual continuity remains unclear. A possible candidate is the serial dependence: visual perception of an object is positively biased toward the visual input from the recent past. Does the visual system use one pattern of serial dependence for general purposes? Or different patterns of serial dependence for different visual tasks? Because different social facial traits (e.g., trustworthiness and dominance) are dissociable, it is reasonable to assume that the perception of different facial characteristics would require different patterns of serial dependences. In this study, we examine the existence and the similarities of the serial dependence(s) in the evaluation of seven facial characteristics (i.e., attractiveness, trustworthiness, confidence, dominance, intelligence, age, and aggressiveness). The convergent evidence from conventional Derivative of Gaussian fitting and Markov Chain modeling demonstrated that (1) serial dependence exists in judgments of all seven social facial characteristic, (2) the serial dependences of them are highly similar, and (3) the serial dependence follows the efficient coding. Thus it is highly possible that there exists a general serial dependence mechanism for (at least high-level) vision processing. Moreover, we used the Markov Chain modeling to better describe the transitional pattern of serial dependence, which is a kind of Markov process. These findings may shed light on future works regarding serial dependence, as well as face perception.

Serial dependence in the perceptual judgments of radiologists

In radiological screening, clinicians scan myriads of radiographs with the intent of recognizing and differentiating lesions. Even though they are trained experts, radiologists’ human search engines are not perfect: average daily error rates are estimated around 3–5%. A main underlying assumption in radiological screening is that visual search on a current radiograph occurs independently of previously seen radiographs. However, recent studies have shown that human perception is biased by previously seen stimuli; the bias in our visual system to misperceive current stimuli towards previous stimuli is called serial dependence. Here, we tested whether serial dependence impacts radiologists’ recognition of simulated lesions embedded in actual radiographs. We found that serial dependence affected radiologists’ recognition of simulated lesions; perception on an average trial was pulled 13% toward the 1-back stimulus. Simulated lesions were perceived as biased towards the those seen in the previous 1 or 2 radiographs. Similar results were found when testing lesion recognition in a group of untrained observers. Taken together, these results suggest that perceptual judgements of radiologists are affected by previous visual experience, and thus some of the diagnostic errors exhibited by radiologists may be caused by serial dependence from previously seen radiographs.

Serial Dependence of Emotion Within and Between Stimulus Sensory Modalities

How we perceive the world is not solely determined by what we sense at a given moment in time, but also by what we processed recently. Here we investigated whether such serial dependencies for emotional stimuli transfer from one modality to another. Participants were presented a random sequence of emotional sounds and images and instructed to rate the valence and arousal of each stimulus (Experiment 1). For both ratings, we conducted an intertrial analysis, based on whether the rating on the previous trial was low or high. We found a positive serial dependence for valence and arousal regardless of the stimulus modality on two consecutive trials. In Experiment 2, we examined whether passively perceiving a stimulus is sufficient to induce a serial dependence. In Experiment 2, participants were instructed to rate the stimuli only on active trials and not on passive trials. The participants were informed that the active and passive trials were presented in alternating order, so that they were able to prepare for the task. We conducted an intertrial analysis on active trials, based on whether the rating on the previous passive trial (determined in Experiment 1) was low or high. For both ratings, we again observed positive serial dependencies regardless of the stimulus modality. We conclude that the emotional experience triggered by one stimulus affects the emotional experience for a subsequent stimulus regardless of their sensory modalities, that this occurs in a bottom-up fashion, and that this can be explained by residual activation in the emotional network in the brain.

Individual differences in serial dependence manifest when sensory uncertainty is high

Perceptual experience in the recent past has been shown to alter subsequent perception. Recently, it has been suggested that this "serial dependence" effect is modulated by sensory uncertainty. In the current study, by overlaying three different levels of visual noise (i.e., no-, low-, or high-noise) on face stimuli, we investigated how serial dependence in face identity perception varies with sensory uncertainty. After learning two facial identities, the faces were combined at various morph levels and participants reported which identity was perceived while noise and noise-free presentations alternated over trials. Results showed that identity perception of noise-free faces was positively biased toward the past when the previous face was noise-free or highly noisy, but not when a low-noise was added. There were considerable individual differences in bias magnitude for trials preceded by high-noise stimuli which reflected individuals' general bias tendencies. When correlated with the other two conditions, a general bias tendency showed a significant relationship with low-noise trials, but not with no-noise trials. This indicates that the bias tendency of individuals manifests more strongly when the sensory information was uncertain. Therefore, the current findings suggest 1) that sensory uncertainty modulates serial dependence in face identity perception and 2) that an individual's general bias tendency interacts with sensory uncertainty to alter perception.

Serial dependence revealed in history-dependent perceptual templates

In any given perceptual task, the visual system selectively weighs or filters incoming information. The particular set of weights or filters form a kind of template, which reveals the regions or types of information that are particularly useful for a given perceptual decision.1,2 Unfortunately, sensory input is noisy and ever changing. To compensate for these fluctuations, the visual system could adopt a strategy of biasing the templates such that they reflect a temporal smoothing of input, which would be a form of serial dependence.3-5 Here, we demonstrate that perceptual templates are, in fact, altered by serial dependence. Using a simple orientation detection task and classification-image technique, we found that perceptual templates are systematically biased toward previously seen, task-irrelevant orientations. The results of an orientation discrimination task suggest that this shift in perceptual template derives from a change in the perceptual appearance of orientation. Our study reveals how serial dependence biases internal templates of orientation and suggests that the sensitivity of classification-image techniques in general could be improved by taking into account history-dependent fluctuations in templates.

A shared mechanism for facial expression in human faces and face pareidolia

Facial expressions are vital for social communication, yet the underlying mechanisms are still being discovered. Illusory faces perceived in objects (face pareidolia) are errors of face detection that share some neural mechanisms with human face processing. However, it is unknown whether expression in illusory faces engages the same mechanisms as human faces. Here, using a serial dependence paradigm, we investigated whether illusory and human faces share a common expression mechanism. First, we found that images of face pareidolia are reliably rated for expression, within and between observers, despite varying greatly in visual features. Second, they exhibit positive serial dependence for perceived facial expression, meaning an illusory face (happy or angry) is perceived as more similar in expression to the preceding one, just as seen for human faces. This suggests illusory and human faces engage similar mechanisms of temporal continuity. Third, we found robust cross-domain serial dependence of perceived expression between illusory and human faces when they were interleaved, with serial effects larger when illusory faces preceded human faces than the reverse. Together, the results support a shared mechanism for facial expression between human faces and illusory faces and suggest that expression processing is not tightly bound to human facial features.

Serial dependence in time and numerosity perception is dimension-specific

The perception of a visual event (e.g., a flock of birds) at the present moment can be biased by a previous perceptual experience (e.g., the perception of an earlier flock). Serial dependence is a perceptual bias whereby a current stimulus appears more similar to a previous one than it actually is. Whereas serial dependence emerges within several visual stimulus dimensions, whether it could simultaneously operate across different dimensions of the same stimulus (e.g., the numerosity and the duration of a visual pattern) remains unclear. Here we address this question by assessing the presence of serial dependence across duration and numerosity, two stimulus dimensions that are often associated and can bias each other. Participants performed either a duration or a numerosity discrimination task, in which they compared a constant reference with a variable test stimulus, varying along the task-relevant dimension (either duration or numerosity). Serial dependence was induced by a task-irrelevant inducer, that is, a stimulus presented before the reference and always varying in both duration and numerosity. The results show systematic serial dependencies only within the task-relevant stimulus dimension, that is, stimulus numerosity affects numerosity perception only, and duration affects duration perception only. Additionally, at least in the numerosity condition, the task-irrelevant dimension of the inducer (duration) had an opposite, repulsive effect. These findings thus show that attractive serial dependence operates in a highly specific fashion and does not transfer across different stimulus dimensions. Instead, the repulsive influence, possibly reflecting perceptual adaptation, can transfer from one dimension to another.

Expectations about motion direction affect perception and anticipatory smooth pursuit differently

Smooth pursuit eye movements and visual motion perception rely on the integration of current sensory signals with past experience. Experience shapes our expectation of current visual events and can drive eye movement responses made in anticipation of a target, such as anticipatory pursuit. Previous research revealed consistent effects of expectation on anticipatory pursuit-eye movements follow the expected target direction or speed-and contrasting effects on motion perception, but most studies considered either eye movement or perceptual responses. The current study directly compared effects of direction expectation on perception and anticipatory pursuit within the same direction discrimination task to investigate whether both types of responses are affected similarly or differently. Observers (n = 10) viewed high-coherence random-dot kinematograms (RDKs) moving rightward and leftward with a probability of 50%, 70%, or 90% in a given block of trials to build up an expectation of motion direction. They were asked to judge motion direction of interleaved low-coherence RDKs (0%-15%). Perceptual judgements were compared with changes in anticipatory pursuit eye movements as a function of probability. Results show that anticipatory pursuit velocity scaled with probability and followed direction expectation (attraction bias), whereas perceptual judgments were biased opposite to direction expectation (repulsion bias). Control experiments suggest that the repulsion bias in perception was not caused by retinal slip induced by anticipatory pursuit, or by motion adaptation. We conclude that direction expectation can be processed differently for perception and anticipatory pursuit.NEW & NOTEWORTHY We show that expectations about motion direction that are based on long-term trial history affect perception and anticipatory pursuit differently. Whereas anticipatory pursuit direction was coherent with the expected motion direction (attraction bias), perception was biased opposite to the expected direction (repulsion bias). These opposite biases potentially reveal different ways in which perception and action utilize prior information and support the idea of different information processing for perception and pursuit.

Dissecting the Roles of Supervised and Unsupervised Learning in Perceptual Discrimination Judgments

Our ability to compare sensory stimuli is a fundamental cognitive function, which is known to be affected by two biases: choice bias, which reflects a preference for a given response, and contraction bias, which reflects a tendency to perceive stimuli as similar to previous ones. To test whether both reflect supervised processes, we designed feedback protocols aimed to modify them and tested them in human participants. Choice bias was readily modifiable. However, contraction bias was not. To compare these results to those predicted from an optimal supervised process, we studied a noise-matched optimal linear discriminator (Perceptron). In this model, both biases were substantially modified, indicating that the “resilience” of contraction bias to feedback does not maximize performance. These results suggest that perceptual discrimination is a hierarchical, two-stage process. In the first, stimulus statistics are learned and integrated with representations in an unsupervised process that is impenetrable to external feedback. In the second, a binary judgment, learned in a supervised way, is applied to the combined percept.

SIGNIFICANCE STATEMENT The seemingly effortless process of inferring physical reality from the sensory input is highly influenced by previous knowledge, leading to perceptual biases. Two common ones are contraction bias (the tendency to perceive stimuli as similar to previous ones) and choice bias (the tendency to prefer a specific response). Combining human psychophysical experiments with computational modeling we show that they reflect two different learning processes. Contraction bias reflects unsupervised learning of stimuli statistics, whereas choice bias results from supervised or reinforcement learning. This dissociation reveals a hierarchical, two-stage process. The first, where stimuli statistics are learned and integrated with representations, is unsupervised. The second, where a binary judgment is applied to the combined percept, is learned in a supervised way.

Individual difference in serial dependence results from opposite influences of perceptual choices and motor responses

Natural image statistics exhibit temporal regularities of slow changes and short-term correlations and visual perception, too, is biased toward recently seen stimuli, i.e., a positive serial dependence. Some studies report strong individual differences in serial dependence in perceptual decision-making: some observers show positive serial effects, others repulsive effects, and some show no bias. To understand these contrasting results, this study separates the influences of physical stimuli per se, perceptual choices, and motor responses on serial dependence in perceptual decision making. In two experiments, human observers reported which orientation (45° or −45°, at threshold contrast) they perceived. Experiment 1, used a consistent mapping between stimulus and response buttons whereas in Experiment 2, observers did two tasks: one with a consistent stimulus-response mapping, the other with a random stimulus-response mapping (perceptual choice and motor response unrelated). Results show that the stimulus percept (not the physical stimulus per se) affected subsequent perceptual choices in an attractive way and that motor responses produced a repulsive serial effect. When the choice-response mapping was consistent (inseparable choice and response, typical of most experiments), individual differences in the overall serial effect was observed: some were positive, some repulsive, and some were bias-free. The multiple regression analysis revealed that observers’ overall serial effects in the consistent choice-response mapping task could be predicted by their serial effects for choices and motor responses in the random mapping task. These individual differences likely reflect relative weightings of a positive choice bias and a repulsive motor bias.

Auditory Rate Perception Displays a Positive Serial Dependence

We investigated perceived timing in auditory rate perception using a reproduction task. The study aimed to test (a) whether central tendency occurs in rate perception, as shown for interval timing, and (b) whether rate is perceived independently on each trial or shows a serial dependence, as shown for other perceptual attributes. Participants were well able to indicate perceived rate as reproduced and presented rates were linearly related with a slope that approached unity, although tapping significantly overestimated presented rates. While the slopes approached unity, they were significantly less than 1, indicating a central tendency in which reproduced rates tended towards the mean of the presented range. We tested for serial dependency by seeing if current trial rate reproductions depended on the preceding rate. In two conditions, a positive dependence was observed. A third condition in which participants withheld responses on every second trial produced a negative dependency. These results suggest separate components of serial dependence linked to stimulus and response: Withholding responses reveals a negative perceptual effect, whereas making responses adds a stronger positive effect that is postperceptual and makes the combined effect positive. Together, these data show that auditory rate perception exhibits both central tendency and serial dependence effects.

Serial dependence and center bias in heading perception from optic flow

Previous work shows that observers can use information from optic flow to perceive the direction of self-motion (i.e. heading) and that perceived heading exhibits a bias towards the center of the display (center bias). More recent work shows that the brain is sensitive to serial correlations and the perception of current stimuli can be affected by recently seen stimuli, a phenomenon known as serial dependence. In the current study, we examined whether, apart from center bias, serial dependence could be independently observed in heading judgments and how adding noise to optic flow affected center bias and serial dependence. We found a repulsive serial dependence effect in heading judgments after factoring out center bias in heading responses. The serial effect expands heading estimates away from the previously seen heading to increase overall sensitivity to changes in heading directions. Both the center bias and repulsive serial dependence effects increased with increasing noise in optic flow, and the noise-dependent changes in the serial effect were consistent with an ideal observer model. Our results suggest that the center bias effect is due to a prior of the straight-ahead direction in the Bayesian inference account for heading perception, whereas the repulsive serial dependence is an effect that reduces response errors and has the added utility of counteracting the center bias in heading judgments.

Attraction to the recent past in aesthetic judgments: A positive serial dependence for rating artwork

Visual perception can be systematically biased towards the recent past. Many stimulus attributes-including orientation, numerosity, facial expression and attractiveness, and perceived slimness-are systematically biased towards recent past experience. This phenomenon has been termed serial dependence. In the current study, we tested whether serial dependence occurs for aesthetic ratings of artworks. A set of 100 paintings depicting scenery and still life was collected from online archives. For each participant, 40 paintings were randomly selected from the set, and presented sequentially 20 times in random order. Serial dependence was quantified for each observer by measuring how their rating response on each trial depended on the attractiveness of the previous trial. The data were pooled across participants and fitted with a Bayesian model of serial dependence. Results showed that the current painting earned significantly higher aesthetic ratings when participants viewed a more attractive painting on the previous trial, compared to when they viewed a less attractive one. The magnitude of serial dependence was greatest when the attractiveness distance between consecutive paintings was relatively close. The effect held both for 1 s exposure times, and for brief 250 ms exposures (followed by a mask). These findings show that aesthetic judgments are not sequentially independent, showing that positive serial dependencies are not limited to low-level perceptual judgments.

Russian blues reveal the limits of language influencing colour discrimination

Chromatic stimuli across a boundary of basic colour categories (BCCs; e.g. blue and green) are discriminated faster than colorimetrically equidistant colours within a given category. Russian has two BCCs for blue, sinij 'dark blue' and goluboj 'light blue'. These language-specific BCCs were reported to enable native Russian speakers to discriminate cross-boundary dark and light blues faster than English speakers (Winawer et al., 2007, PNAS, 4, 7780-7785). We re-evaluated this finding in two experiments that employed identical tasks as in the cited study. In Experiment 1, Russian and English speakers categorised colours as sinij/goluboj or dark blue/light blue respectively; this was followed by a colour discrimination task. In Experiment 2, Russian speakers initially performed the discrimination task on sinij/goluboj and goluboj/zelënyj 'green' sets. They then categorised these colours in three frequency contexts with each stimulus presented: (i) an equal number of times (unbiased); more frequent (ii) either sinij or goluboj; (iii) either goluboj or zelënyj. We observed a boundary response speed advantage for goluboj/zelënyj but not for sinij/goluboj. The frequency bias affected only the sinij/goluboj boundary such that in a lighter context, the boundary shifted towards lighter shades, and vice versa. Contrary to previous research, our results show that in Russian, stimulus discrimination at the lightness-defined blue BCC boundary is not reflected in processing speed. The sinij/goluboj boundary did have a sharper categorical transition than the dark blue/light blue boundary, but it was also affected by frequency and order biases, demonstrating that "Russian blues" are less well-structured than previously thought.

Positive sequential dependency for face attractiveness perception

Recent findings from several groups have demonstrated that visual perception at a given moment can be biased toward what was recently seen. This is true both for basic visual attributes and for more complex representations, such as face identity, gender, or expression. This assimilation to the recent past is a positive serial dependency, similar to a temporal averaging process that capitalizes on short-term correlations in visual input to reduce noise and boost perceptual continuity. Here we examine serial dependencies in face perception using a simple attractiveness rating task and a rapid series of briefly presented face stimuli. In a series of three experiments, our results confirm a previous report that face attractiveness exhibits a positive serial dependency. This intertrial effect is not only determined by face attractiveness on the previous trial, but also depends on the faces shown up to five trials back. We examine the effect of stimulus presentation duration and find that stimuli as brief as 56 ms produce a significant positive dependency similar in magnitude to that produced by stimuli presented for 1,000 ms. We observed stronger positive dependencies between same-gender faces, and found a task dependency: Alternating gender discrimination trials with attractiveness rating trials produced no serial dependency. In sum, these findings show that a perception-stabilizing assimilation effect operates in face attractiveness perception that is task dependent and is acquired surprisingly quickly.

Where and when to look: Sequential effects at the millisecond level

Learning and imitating a complex motor action requires to visually follow complex movements, but conscious perception seems too slow for such tasks. Recent findings suggest that visual perception has a higher temporal resolution at an unconscious than at a conscious level. Here we investigate whether high-temporal resolution in visual perception relies on prediction mechanisms and attention shifts based on recently experienced sequences of visual information. To that aim we explore sequential effects during four different simultaneity/asynchrony discrimination tasks. Two stimuli are displayed on each trial with varying stimulus onset asynchronies (SOA). Subjects decide whether the stimuli are simultaneous or asynchronous and give manual responses. The main finding is an advantage for different-order over same-order trials, when subjects decided that stimuli had been simultaneous on Trial t - 1 , and when Trial t is with an SOA slightly larger than Trial t - 1, or equivalent. The advantage for different-order trials disappears when the stimuli change eccentricity but not direction between trials (Experiment 2), and persists with stimuli displayed in the centre and unlikely to elicit a sense of direction (Experiment 4). It is still observed when asynchronies on Trial t - 1 are small and undetected (Experiment 3). The findings can be explained by an attention shift that is precisely planned in time and space and that incidentally allows subjects to detect an isolated stimulus on the screen, thus helping them to detect an asynchrony.

Disambiguating serial effects of multiple timescales

What has been previously experienced can systematically affect human perception in the present. We designed a novel psychophysical experiment to measure the perceptual effects of adapting to dynamically changing stimulus statistics. Observers are presented with a series of oriented Gabor patches and are asked occasionally to judge the orientation of highly ambiguous test patches. We developed a computational model to quantify the influence of past stimuli presentations on the observers' perception of test stimuli over multiple timescales and to show that this influence is distinguishable from simple response biases. The experimental results reveal that perception is attracted toward the very recent past and simultaneously repulsed from stimuli presented at short to medium timescales and attracted to presentations further in the past. All effects differ significantly both on their relative strength and their respective duration. Our model provides a structured way of quantifying serial effects in psychophysical experiments, and it could help experimenters in identifying such effects in their data and distinguish them from less interesting response biases.

Neural dynamics of the attentional blink revealed by encoding orientation selectivity during rapid visual presentation

The human brain is inherently limited in the information it can make consciously accessible. When people monitor a rapid stream of visual items for two targets, they typically fail to see the second target if it occurs within 200–500 ms of the first, a phenomenon called the attentional blink (AB). The neural basis for the AB is poorly understood, partly because conventional neuroimaging techniques cannot resolve visual events displayed close together in time. Here we introduce an approach that characterises the precise effect of the AB on behaviour and neural activity. We employ multivariate encoding analyses to extract feature-selective information carried by randomly-oriented gratings. We show that feature selectivity is enhanced for correctly reported targets and suppressed when the same items are missed, whereas irrelevant distractor items are unaffected. The findings suggest that the AB involves both short- and long-range neural interactions between visual representations competing for access to consciousness.

People often fail to perceive the second of two brief visual targets, a phenomenon known as the attentional blink (AB). Here the authors modelled behaviour and brain activity to show that the AB arises from short- and long-range interactions between representations of elementary visual features.

Neural Dynamics of Serial Dependence in Numerosity Perception

Serial dependence—an attractive perceptual bias whereby a current stimulus is perceived to be similar to previously seen ones—is thought to represent the process that facilitates the stability and continuity of visual perception. Recent results demonstrate a neural signature of serial dependence in numerosity perception, emerging very early in the time course during perceptual processing. However, whether such a perceptual signature is retained after the initial processing remains unknown. Here, we address this question by investigating the neural dynamics of serial dependence using a recently developed technique that allowed a reactivation of hidden memory states. Participants performed a numerosity discrimination task during EEG recording, with task-relevant dot array stimuli preceded by a task-irrelevant stimulus inducing serial dependence. Importantly, the neural network storing the representation of the numerosity stimulus was perturbed (or pinged) so that the hidden states of that representation can be explicitly quantified. The results first show that a neural signature of serial dependence emerges early in the brain signals, starting soon after stimulus onset. Critical to the central question, the pings at a later latency could successfully reactivate the biased representation of the initial stimulus carrying the signature of serial dependence. These results provide one of the first pieces of empirical evidence that the biased neural representation of a stimulus initially induced by serial dependence is preserved throughout a relatively long period.

Serial dependence in a simulated clinical visual search task

In everyday life, we continuously search for and classify objects in the environment around us. This kind of visual search is extremely important when performed by radiologists in cancer image interpretation and officers in airport security screening. During these tasks, observers often examine large numbers of uncorrelated images (tumor x-rays, checkpoint x-rays, etc.) one after another. An underlying assumption of such tasks is that search and recognition are independent of our past experience. Here, we simulated a visual search task reminiscent of medical image search and found that shape classification performance was strongly impaired by recent visual experience, biasing classification errors 7% more towards the previous image content. This perceptual attraction exhibited the three main tuning characteristics of Continuity Fields: serial dependence extended over 12 seconds back in time (temporal tuning), it occurred only between similar tumor-like shapes (feature tuning), and only within a limited spatial region (spatial tuning). Taken together, these results demonstrate that serial dependence influences shape perception and occurs in visual search tasks. They also raise the possibility of a detrimental impact of serial dependence in clinical and practically relevant settings, such as medical image perception.

The perceptual continuity field is retinotopic

Visual perception is systematically biased towards input from the recent past: perceived orientation, numerosity, and face identity are pulled towards previously seen stimuli. To better understand the brain level at which serial dependence occurs, the present study examined its spatial tuning. In three experiments, serial dependence occurred between stimuli occupying the same retinal position. Serial dependence between stimuli at distant retinal locations was smaller, even when the stimuli occupied the same location in external space. The spatial window over which serial dependence occurs is thus retinotopic, but wide, suggesting that serial dependence occurs at late stages of visual processing.

Auditory Perceptual History Is Propagated through Alpha Oscillations

Perception is a proactive, “predictive” process, in which the brain relies, at least in part, on accumulated experience to make best guesses about the world to test against sensory data, updating the guesses as new experience is acquired. Using novel behavioral methods, the present study demonstrates the role of alpha rhythms in communicating past perceptual experience. Participants were required to discriminate the ear of origin of brief sinusoidal tones that were presented monaurally at random times within a burst of uncorrelated dichotic white noise masks. Performance was not constant but varied with delay after noise onset in an oscillatory manner at about 9 Hz (alpha rhythm). Importantly, oscillations occurred only for trials preceded by a target tone to the same ear, either on the previous trial or two trials back. These results suggest that communication of perceptual history generates neural oscillations within specific perceptual circuits, strongly implicating behavioral oscillations in predictive perception and with formation of working memory.

•

We demonstrate the role of alpha rhythms in the propagation of perceptual history

•

Auditory decisions were rhythmically biased by stimuli presented 1 or 2 trials back

•

Bias oscillated at ∼9 Hz only when successive stimuli occurred in the same ear

•

Alpha is strongly implicated in predictive perception and working memory formation

Visual serial dependence in an audiovisual stimulus

Serial dependence is a phenomenon that biases the perception of features or objects systematically toward sensory input from the recent past (Fischer & Whitney, 2014). There is an active debate whether this effect is rooted directly in perception or reflects biases in decision making. We investigated serial dependence across three experiments by manipulating the decision made on each trial. A multimodal audiovisual stimulus comprising a Gabor and a vowel sound was presented repeatedly. On each trial, participants reported either the Gabor orientation or the vowel sound. Participants either ignored one modality (Experiment 1) or attended to both modalities (Experiments 2 and 3). In Experiments 2 and 3, the response task was randomized to prevent anticipating which modality to respond to until the response phase. In Experiment 3, no-response trials were additionally interleaved. Results across the three experiments demonstrated serial dependence only when participants reported the visual modality. Serial dependence was also present in visual reports when participants completed auditory reports or made no reports on previous trials. The previous stimulus alone was enough to elicit an effect. Serial dependence is unlikely to be an effect of the previous decision on the stimulus, but rather an effect of perceiving the previous stimulus.

What makes a prototype a prototype? Averaging visual features in a sequence

After viewing a series of sequentially presented visual stimuli, subjects can readily generate mean representations of various visual features. Although these average representations seem to be formed effortlessly and obligatorily, it is unclear how such averages are actually computed. According to conventional prototype models, the computation entails an equally weighted average taken over all the stimuli. To test this hypothesis, we had subjects estimate the running averages of some feature in a series of sequentially presented stimuli. Part way through the series, we perturbed the distribution from which stimuli were drawn, which allowed us to test alternative models of the computations behind subjects' estimates. In both explanatory and predictive tests, a model in which the most recent items had disproportionate high weight outperformed a model in which all items carried equal weight. Such recency-weighted behavior was shown consistently in multiple experiments in which subjects estimated running averages of length of vertical lines. However, the degree to which recent items were prioritized varied with the type of stimulus, such that when estimating the running averages of a series of numerals, subjects showed less recency prioritization. We conclude that previous evaluations of prototype models have made unrealistic assumptions about the nature of a prototype, and that a reassessment of prototype models of visual memory and perceptual categorization may be in order.

Working Memory Maintenance Modulates Serial Dependence Effects of Perceived Emotional Expression

The stability of face perception is vital in interpersonal interactions. Recent studies have revealed the mechanism of the stability in the perception of stable attributes of faces (such as facial identity) by serial dependence, a phenomenon in which perception of current stimuli is pulled toward recently viewed stimuli. However, whether serial dependence of perceived emotional expression (a changeable attribute of faces) exists remains controversial, and its exact nature has not been examined yet. To address these issues, we used the methods of constant stimuli and two-interval forced choice tasks in three psychophysical experiments. Participants compared two successive facial expressions selected from a continuum with 50 morphed faces ranging from sad to happy. Experiment 1a and 1b showed that a perceived facial expression pulled toward previously seen facial expressions (i.e., a significant serial dependence effect), independent of response instructions. Furthermore, a stronger serial dependence effect was found when the first facial expression was retained in working memory for a longer delay duration (Experiment 2), and yet a weaker serial dependence effect was observed when a longer delay between decision and response was performed (Experiment 3). These findings indicate that serial dependence facilitates the stability of facial expression perception and is modulated by working memory representations.

Confidence boosts serial dependence in orientation estimation

In the absence of external feedback, a decision maker must rely on a subjective estimate of their decision accuracy in order to appropriately guide behavior. Normative models of perceptual decision-making relate subjective estimates of internal signal quality (e.g., confidence) directly to the internal signal quality itself, thereby making it unknowable whether the subjective estimate or the underlying signal is what drives behavior. We constructed stimuli that dissociated the human observer's performance on a visual estimation task from their subjective estimates of confidence in their performance, thus violating normative principles. To understand whether confidence influences future decision-making, we examined serial dependence in observer's responses, a phenomenon whereby the estimate of a stimulus on the current trial can be biased toward the stimulus from the previous trial. We found that when decisions were made with high confidence, they conferred stronger biases upon the following trial, suggesting that confidence may enhance serial dependence. Critically, this finding was true also when confidence was experimentally dissociated from task performance, indicating that subjective confidence, independent of signal quality, can amplify serial dependence. These findings demonstrate an effect of confidence on future behavior, independent of task performance, and suggest that perceptual decisions incorporate recent history in an uncertainty-weighted manner, but where the uncertainty carried forward is a subjectively estimated and possibly suboptimal readout of objective sensory uncertainty.

Serial dependence promotes the stability of perceived emotional expression depending on face similarity

Individuals can quickly and effortlessly recognize facial expressions, which is critical for social perception and emotion regulation. This sensitivity to even slight facial changes could result in unstable percepts of an individual's expression over time. The visual system must therefore balance accuracy with maintaining perceptual stability. However, previous research has focused on our sensitivity to changing expressions, and the mechanism behind expression stability remains an open question. Recent results demonstrate that perception of facial identity is systematically biased toward recently seen visual input. This positive perceptual pull, or serial dependence, may help stabilize perceived expression. To test this, observers judged random facial expression morphs ranging from happy to sad to angry. We found a pull in perceived expression toward previously seen expressions, but only when the 1-back and current face had similar identities. Our results are consistent with the existence of the continuity field for expression, a specialized mechanism that promotes the stability of emotion perception, which could help facilitate social interactions and emotion regulation.