Linear Summation of Repulsive and Attractive Serial Dependencies: Orientation and Motion Dependencies Sum in Motion Perception

Confidence boosts serial dependence in orientation estimation

In the absence of external feedback, a decision maker must rely on a subjective estimate of their decision accuracy in order to appropriately guide behavior. Normative models of perceptual decision-making relate subjective estimates of internal signal quality (e.g., confidence) directly to the internal signal quality itself, thereby making it unknowable whether the subjective estimate or the underlying signal is what drives behavior. We constructed stimuli that dissociated the human observer's performance on a visual estimation task from their subjective estimates of confidence in their performance, thus violating normative principles. To understand whether confidence influences future decision-making, we examined serial dependence in observer's responses, a phenomenon whereby the estimate of a stimulus on the current trial can be biased toward the stimulus from the previous trial. We found that when decisions were made with high confidence, they conferred stronger biases upon the following trial, suggesting that confidence may enhance serial dependence. Critically, this finding was true also when confidence was experimentally dissociated from task performance, indicating that subjective confidence, independent of signal quality, can amplify serial dependence. These findings demonstrate an effect of confidence on future behavior, independent of task performance, and suggest that perceptual decisions incorporate recent history in an uncertainty-weighted manner, but where the uncertainty carried forward is a subjectively estimated and possibly suboptimal readout of objective sensory uncertainty.

Serial dependence promotes the stability of perceived emotional expression depending on face similarity

Individuals can quickly and effortlessly recognize facial expressions, which is critical for social perception and emotion regulation. This sensitivity to even slight facial changes could result in unstable percepts of an individual's expression over time. The visual system must therefore balance accuracy with maintaining perceptual stability. However, previous research has focused on our sensitivity to changing expressions, and the mechanism behind expression stability remains an open question. Recent results demonstrate that perception of facial identity is systematically biased toward recently seen visual input. This positive perceptual pull, or serial dependence, may help stabilize perceived expression. To test this, observers judged random facial expression morphs ranging from happy to sad to angry. We found a pull in perceived expression toward previously seen expressions, but only when the 1-back and current face had similar identities. Our results are consistent with the existence of the continuity field for expression, a specialized mechanism that promotes the stability of emotion perception, which could help facilitate social interactions and emotion regulation.

Laws of concatenated perception: Vision goes for novelty, decisions for perseverance

Every instant of perception depends on a cascade of brain processes calibrated to the history of sensory and decisional events. In the present work, we show that human visual perception is constantly shaped by two contrasting forces exerted by sensory adaptation and past decisions. In a series of experiments, we used multilevel modeling and cross-validation approaches to investigate the impact of previous stimuli and decisions on behavioral reports during adjustment and forced-choice tasks. Our results revealed that each perceptual report is permeated by opposite biases from a hierarchy of serially dependent processes: Low-level adaptation repels perception away from previous stimuli, whereas decisional traces attract perceptual reports toward the recent past. In this hierarchy of serial dependence, "continuity fields" arise from the inertia of decisional templates and not from low-level sensory processes. This finding is consistent with a Two-process model of serial dependence in which the persistence of readout weights in a decision unit compensates for sensory adaptation, leading to attractive biases in sequential perception. We propose a unified account of serial dependence in which functionally distinct mechanisms, operating at different stages, promote the differentiation and integration of visual information over time.

Serial dependence in position occurs at the time of perception

Observers perceive objects in the world as stable over space and time, even though the visual experience of those objects is often discontinuous and distorted due to masking, occlusion, camouflage, or noise. How are we able to easily and quickly achieve stable perception in spite of this constantly changing visual input? It was previously shown that observers experience serial dependence in the perception of features and objects, an effect that extends up to 15 seconds back in time. Here, we asked whether the visual system utilizes an object's prior physical location to inform future position assignments in order to maximize location stability of an object over time. To test this, we presented subjects with small targets at random angular locations relative to central fixation in the peripheral visual field. Subjects reported the perceived location of the target on each trial by adjusting a cursor's position to match its location. Subjects made consistent errors when reporting the perceived position of the target on the current trial, mislocalizing it toward the position of the target in the preceding two trials (Experiment 1). This pull in position perception occurred even when a response was not required on the previous trial (Experiment 2). In addition, we show that serial dependence in perceived position occurs immediately after stimulus presentation, and it is a fast stabilization mechanism that does not require a delay (Experiment 3). This indicates that serial dependence occurs for position representations and facilitates the stable perception of objects in space. Taken together with previous work, our results show that serial dependence occurs at many stages of visual processing, from initial position assignment to object categorization.

Context Effects in the Judgment of Visual Relative-Frequency: Trial-by-Trial Adaptation and Non-linear Sequential Effect

Humans' judgment of relative-frequency, similar to their use of probability in decision-making, is often distorted as an inverted-S-shape curve-small relative-frequency overestimated and large relative-frequency underestimated. Here we investigated how the judgment of relative-frequency, despite its natural reference points (0 and 1) and stereotyped distortion, may adapt to the environmental statistics. The task was to report the relative-frequency of black (or white) dots in a visual array of black and white dots. We found that participants' judgment was distorted in the typical inverted-S-shape, but the distortion curve was influenced by both the central tendency and spread of the distribution of objective relative-frequencies: the lower the central tendency, the higher the overall judgment (contrast effect); the higher the spread, the more curved the inverted-S-shape (curvature effect). These context effects are in the spirit of efficient coding but opposite to what would be predicted by Bayesian inference. We further modeled the context effects on the level of individual trials, through which we found not only a trial-by-trial adaptation, but also the non-linear sequential effects that were recently reported mainly in circularly distributed visual stimuli.

Serial dependence in numerosity perception

Our conscious experience of the external world is remarkably stable and seamless, despite the intrinsically discontinuous and noisy nature of sensory information. Serial dependencies in visual perception-reflecting attractive biases making a current stimulus to appear more similar to previous ones-have been recently hypothesized to be involved in perceptual continuity. However, while these effects have been observed across a variety of visual features and at the neural level, several aspects of serial dependence and how it generalizes across visual dimensions is still unknown. Here we explore the behavioral signature of serial dependence in numerosity perception by assessing how the perceived numerosity of dot-array stimuli is biased by a task-irrelevant "inducer" stimulus presented before task-relevant stimuli. First, although prior work suggests that numerosity perception starts in the subcortex, the current study rules out a possible involvement of subcortical processing in serial dependence, confirming that the effect likely starts in the visual cortex. Second, we show that the effect is coarsely spatially localized to the position of the inducer stimulus. Third, we demonstrate that the effect is present even with a stimulus presentation procedure minimizing the involvement of post-perceptual processes, but only when participants actively pay attention to the inducer stimulus. Overall, these results provide a comprehensive characterization of serial dependencies in numerosity perception, demonstrating that attractive biases occur by means of spatially localized attentional modulations of early sensory activity.

Optimal multisensory integration leads to optimal time estimation

Our brain compensates sensory uncertainty by combining multisensory information derived from an event, and by integrating the current sensory signal with the prior knowledge about the statistical structure of previous events. There is growing evidence that both strategies are statistically optimal; however, how these two stages of information integration interact and shape an optimal percept remains an open question. In the present study, we investigated the perception of time as an amodal perceptual attribute. The central tendency, a phenomenon of biasing the current percept toward previous stimuli, is used to quantify and model how the prior information affects the current timing behavior. We measured the timing sensitivity and the central tendency for unisensory and multisensory stimuli with sensory uncertainty systematically manipulated by adding noise. Psychophysical results demonstrate that the central tendency increases as the uncertainty increases, and that the multisensory timing improves both the timing sensitivity and the central tendency bias compared to the unisensory timing. Computational models indicate that the optimal multisensory integration precedes the optimal integration of prior information causing the central tendency. Our findings suggest that our brain incorporates the multisensory information and prior knowledge in a statistically optimal manner to realize precise and accurate timing behavior.

Attractive Serial Dependence in the Absence of an Explicit Task

Attractive serial dependence refers to an adaptive change in the representation of sensory information, whereby a current stimulus appears to be similar to a previous one. The nature of this phenomenon is controversial, however, as serial dependence could arise from biased perceptual representations or from biased traces of working memory representation at a decisional stage. Here, we demonstrated a neural signature of serial dependence in numerosity perception emerging early in the visual processing stream even in the absence of an explicit task. Furthermore, a psychophysical experiment revealed that numerosity perception is biased by a previously presented stimulus in an attractive way, not by repulsive adaptation. These results suggest that serial dependence is a perceptual phenomenon starting from early levels of visual processing and occurring independently from a decision process, which is consistent with the view that these biases smooth out noise from neural signals to establish perceptual continuity.

A Matched Comparison Across Three Different Sensory Pairs of Cross-Modal Temporal Recalibration From Sustained and Transient Adaptation

Sustained exposure to an asynchronous multisensory signal causes perceived simultaneity to shift in the direction of the leading component of the adapting stimulus. This is known as temporal recalibration, and recent evidence suggests that it can occur very rapidly, even after a single asynchronous audiovisual (AV) stimulus. However, this form of rapid recalibration appears to be unique to AV stimuli, in contrast to recalibration following sustained asynchronies which occurs with audiotactile (AT) and visuotactile (VT) stimuli. This study examines temporal recalibration to AV, VT and AT asynchrony with spatially collocated stimuli using a design that produces both sustained and inter-trial recalibration by combining the traditional sustained adaptation approach with an inter-trial analysis of sequential dependencies in an extended test period. Thus, we compare temporal recalibration to both sustained and transient asynchrony in three crossmodal combinations using the same design, stimuli and observers. The results reveal that prolonged exposure to asynchrony produced equivalent temporal recalibration for all combinations: AV, AT and VT. The pattern for rapid, inter-trial recalibration was very different. Rapid recalibration occurred strongly for AV stimuli, weakly for AT and did not occur at all for VT. For all sensory pairings, recalibration from sustained asynchrony decayed to baseline during the test phase while inter-trial recalibration was present and stable throughout testing, suggesting different mechanisms may underlie adaptation at long and short timescales.