The challenge of measuring long-term positive aftereffects

An efficient Bayesian observer model of attractive and repulsive temporal context effects when perceiving multistable dot lattices

In multistable dot lattices, the orientation we perceive is attracted toward the orientation we perceived in the immediately preceding stimulus and repelled from the orientation for which most evidence was present previously (Van Geert, Moors, Haaf, & Wagemans, 2022). Theoretically-inspired models have been proposed to explain the co-occurrence of attractive and repulsive context effects in multistable dot lattice tasks, but these models artificially induced an influence of the previous trial on the current one without detailing the process underlying such an influence (Gepshtein & Kubovy, 2005; Schwiedrzik et al., 2014). We conducted a simulation study to test whether the observed attractive and repulsive context effects could be explained with an efficient Bayesian observer model (Wei & Stocker, 2015). This model assumes variable encoding precision of orientations in line with their frequency of occurrence (i.e., efficient encoding) and takes the dissimilarity between stimulus space and sensory space into account. An efficient Bayesian observer model including both a stimulus and a perceptual level was needed to explain the co-occurrence of both attractive and repulsive temporal context effects. Furthermore, this model could reproduce the empirically observed strong positive correlation between individuals' attractive and repulsive effects (Van Geert et al., 2022), by assuming a positive correlation between temporal integration constants at the stimulus and the perceptual level. To conclude, the study brings evidence that efficient encoding and likelihood repulsion on the stimulus level can explain the repulsive context effect, whereas perceptual prior attraction can explain the attractive temporal context effect when perceiving multistable dot lattices.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Serial dependence in perception across naturalistic generative adversarial network-generated mammogram

Purpose

Human perception and decisions are biased toward previously seen stimuli. This phenomenon is known as serial dependence and has been extensively studied for the last decade. Recent evidence suggests that clinicians' judgments of mammograms might also be impacted by serial dependence. However, the stimuli used in previous psychophysical experiments on this question, consisting of artificial geometric shapes and healthy tissue backgrounds, were unrealistic. We utilized realistic and controlled generative adversarial network (GAN)-generated radiographs to mimic images that clinicians typically encounter.

Approach

Mammograms from the digital database for screening mammography (DDSM) were utilized to train a GAN. This pretrained GAN was then adopted to generate a large set of authentic-looking simulated mammograms: 20 circular morph continuums, each with 147 images, for a total of 2940 images. Using these stimuli in a standard serial dependence experiment, participants viewed a random GAN-generated mammogram on each trial and subsequently matched the GAN-generated mammogram encountered using a continuous report. The characteristics of serial dependence from each continuum were analyzed.

Results

We found that serial dependence affected the perception of all naturalistic GAN-generated mammogram morph continuums. In all cases, the perceptual judgments of GAN-generated mammograms were biased toward previously encountered GAN-generated mammograms. On average, perceptual decisions had 7% categorization errors that were pulled in the direction of serial dependence.

Conclusions

Serial dependence was found even in the perception of naturalistic GAN-generated mammograms created by a GAN. This supports the idea that serial dependence could, in principle, contribute to decision errors in medical image perception tasks.

Serial Dependence in Dermatological Judgments

Serial Dependence is a ubiquitous visual phenomenon in which sequentially viewed images appear more similar than they actually are, thus facilitating an efficient and stable perceptual experience in human observers. Although serial dependence is adaptive and beneficial in the naturally autocorrelated visual world, a smoothing perceptual experience, it might turn maladaptive in artificial circumstances, such as medical image perception tasks, where visual stimuli are randomly sequenced. Here, we analyzed 758,139 skin cancer diagnostic records from an online app, and we quantified the semantic similarity between sequential dermatology images using a computer vision model as well as human raters. We then tested whether serial dependence in perception occurs in dermatological judgments as a function of image similarity. We found significant serial dependence in perceptual discrimination judgments of lesion malignancy. Moreover, the serial dependence was tuned to the similarity in the images, and it decayed over time. The results indicate that relatively realistic store-and-forward dermatology judgments may be biased by serial dependence. These findings help in understanding one potential source of systematic bias and errors in medical image perception tasks and hint at useful approaches that could alleviate the errors due to serial dependence.

Calibrating Vision: Concepts and Questions

The idea that visual coding and perception are shaped by experience and adjust to changes in the environment or the observer is universally recognized as a cornerstone of visual processing, yet the functions and processes mediating these calibrations remain in many ways poorly understood. In this article we review a number of facets and issues surrounding the general notion of calibration, with a focus on plasticity within the encoding and representational stages of visual processing. These include how many types of calibrations there are - and how we decide; how plasticity for encoding is intertwined with other principles of sensory coding; how it is instantiated at the level of the dynamic networks mediating vision; how it varies with development or between individuals; and the factors that may limit the form or degree of the adjustments. Our goal is to give a small glimpse of an enormous and fundamental dimension of vision, and to point to some of the unresolved questions in our understanding of how and why ongoing calibrations are a pervasive and essential element of vision.

Kinematic Studies of the Go/No-Go Task as a Dynamic Sensorimotor Inhibition Task for Assessment of Motor and Executive Function in Stroke Patients: An Exploratory Study in a Neurotypical Sample

Inhibition of reaching and grasping actions as an element of cognitive control and executive function is a vital component of sensorimotor behaviour that is often impaired in patients who have lost sensorimotor function following a stroke. To date, there are few kinematic studies detailing the fine spatial and temporal upper limb movements associated with the millisecond temporal trajectory of correct and incorrect responses to visually driven Go/No-Go reaching and grasping tasks. Therefore, we aimed to refine the behavioural measurement of correct and incorrect inhibitory motor responses in a Go/No-Go task for future quantification and personalized rehabilitation in older populations and those with acquired motor disorders, such as stroke. An exploratory study mapping the kinematic profiles of hand movements in neurotypical participants utilizing such a task was conducted using high-speed biological motion capture cameras, revealing both within and between subject differences in a sample of healthy participants. These kinematic profiles and differences are discussed in the context of better assessment of sensorimotor function impairment in stroke survivors.

The effect of perceptual history on the interpretation of causality

The ability to interpret spatiotemporal contingencies in terms of causal relationships plays a key role in human understanding of the external world. Indeed, the detection of such simple properties enables us to attribute causal attributes to interactions between objects. Here, we investigated the degree to which this perception of causality depends on recent experience, as has been found for other low-level properties of visual stimuli. Participants were shown launching sequences of colliding circles with varying collision lags and were asked to report their impression of causality. We found short-term attractive and long-term repulsive and attractive effects of perceptual history on the interpretation of causality. Stimuli directly following a causal impression were more likely to be judged as causal and vice versa. However, prior judgments on less recent (>5) trials biased current perception with both positive/attractive and negative/repulsive influences. We interpret these results in terms of two potential mechanisms: adaptive temporal binding windows and updating of internal representations of causality. Overall, these results demonstrate the important role of prior experience even for causality, a fundamental building block of how we understand our world.

Same stimulus, same temporal context, different percept? Individual differences in hysteresis and adaptation when perceiving multistable dot lattices

How we perceptually organize a visual stimulus depends not only on the stimulus itself, but also on the temporal and spatial context in which the stimulus is presented and on the individual processing the stimulus and context. Earlier research found both attractive and repulsive context effects in perception: tendencies to organize visual input similarly to preceding context stimuli (i.e., hysteresis, attraction) co-exist with tendencies that repel the current percept from the organization that is most dominant in these contextual stimuli (i.e., adaptation, repulsion). These processes have been studied mostly on a group level (e.g., Schwiedrzik et al., 2014). Using a Bayesian hierarchical model comparison approach, the present study (N = 75) investigated whether consistent individual differences exist in these attractive and repulsive temporal context effects, with multistable dot lattices as stimuli. In addition, the temporal stability of these individual differences in context effects was investigated, and it was studied how the strength of these effects related to the strength of individual biases for absolute orientations. The results demonstrate that large individual differences in the size of attractive and repulsive context effects exist. Furthermore, these individual differences are highly consistent across timepoints (one to two weeks apart). Although almost everyone showed both effects in the expected direction, not every single individual did. In sum, the study reveals differences in how individuals combine previous input and experience with current input in their perception, and more generally, this teaches us that different individuals can perceive identical stimuli differently, even within a similar context.

Serial dependence in the perceptual judgments of radiologists

In radiological screening, clinicians scan myriads of radiographs with the intent of recognizing and differentiating lesions. Even though they are trained experts, radiologists’ human search engines are not perfect: average daily error rates are estimated around 3–5%. A main underlying assumption in radiological screening is that visual search on a current radiograph occurs independently of previously seen radiographs. However, recent studies have shown that human perception is biased by previously seen stimuli; the bias in our visual system to misperceive current stimuli towards previous stimuli is called serial dependence. Here, we tested whether serial dependence impacts radiologists’ recognition of simulated lesions embedded in actual radiographs. We found that serial dependence affected radiologists’ recognition of simulated lesions; perception on an average trial was pulled 13% toward the 1-back stimulus. Simulated lesions were perceived as biased towards the those seen in the previous 1 or 2 radiographs. Similar results were found when testing lesion recognition in a group of untrained observers. Taken together, these results suggest that perceptual judgements of radiologists are affected by previous visual experience, and thus some of the diagnostic errors exhibited by radiologists may be caused by serial dependence from previously seen radiographs.

The Certainty of Ambiguity in Visual Neural Representations

Some images evoke bistable percepts: two different visual experiences seen in alternation while continuously viewing an unchanged stimulus. The Necker Cube and Rubin's Vase are classic examples, each of which gives alternating percepts of different shapes. Other bistable percepts are alternating colors or directions of motion. Although stimuli that result in salient bistability are rare and sometimes cleverly constructed to emphasize ambiguity, they have been influential for over 150 years, since the work of von Helmholtz, who considered them to be evidence for perceptual visual processes that interpret retinal stimuli. While bistability in natural viewing is uncommon, the main point of this review is that implicit ambiguity in visual neural representations is pervasive. Resolving ambiguity, therefore, is a fundamental and ubiquitous process of vision that routinely affects what we see, not an oddity arising from cleverly crafted images. This review focuses on the causes of widespread ambiguity, historical perspectives on it, and modern knowledge and theory about resolving it.

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.

Serial dependence in a simulated clinical visual search task

In everyday life, we continuously search for and classify objects in the environment around us. This kind of visual search is extremely important when performed by radiologists in cancer image interpretation and officers in airport security screening. During these tasks, observers often examine large numbers of uncorrelated images (tumor x-rays, checkpoint x-rays, etc.) one after another. An underlying assumption of such tasks is that search and recognition are independent of our past experience. Here, we simulated a visual search task reminiscent of medical image search and found that shape classification performance was strongly impaired by recent visual experience, biasing classification errors 7% more towards the previous image content. This perceptual attraction exhibited the three main tuning characteristics of Continuity Fields: serial dependence extended over 12 seconds back in time (temporal tuning), it occurred only between similar tumor-like shapes (feature tuning), and only within a limited spatial region (spatial tuning). Taken together, these results demonstrate that serial dependence influences shape perception and occurs in visual search tasks. They also raise the possibility of a detrimental impact of serial dependence in clinical and practically relevant settings, such as medical image perception.

Visual detection of time-varying signals: Opposing biases and their timescales

Human visual perception is a complex, dynamic and fluctuating process. In addition to the incoming visual stimulus, it is affected by many other factors including temporal context, both external and internal to the observer. In this study we investigate the dynamic properties of psychophysical responses to a continuous stream of visual near-threshold detection tasks. We manipulate the incoming signals to have temporal structures with various characteristic timescales. Responses of human observers to these signals are analyzed using tools that highlight their dynamical features as well. Our experiments show two opposing biases that shape perceptual decision making simultaneously: positive recency, biasing towards repeated response; and adaptation, entailing an increased probability of changed response. While both these effects have been reported in previous work, our results shed new light on the timescales involved in these effects, and on their interplay with varying inputs. We find that positive recency is a short-term bias, inversely correlated with response time, suggesting it can be compensated by afterthought. Adaptation, in contrast, reflects trends over longer times possibly including multiple previous trials. Our entire dataset, which includes different input signal temporal structures, is consistent with a simple model with the two biases characterized by a fixed parameter set. These results suggest that perceptual biases are inherent features which are not flexible to tune to input signals.

Serial dependence promotes the stability of perceived emotional expression depending on face similarity

Individuals can quickly and effortlessly recognize facial expressions, which is critical for social perception and emotion regulation. This sensitivity to even slight facial changes could result in unstable percepts of an individual's expression over time. The visual system must therefore balance accuracy with maintaining perceptual stability. However, previous research has focused on our sensitivity to changing expressions, and the mechanism behind expression stability remains an open question. Recent results demonstrate that perception of facial identity is systematically biased toward recently seen visual input. This positive perceptual pull, or serial dependence, may help stabilize perceived expression. To test this, observers judged random facial expression morphs ranging from happy to sad to angry. We found a pull in perceived expression toward previously seen expressions, but only when the 1-back and current face had similar identities. Our results are consistent with the existence of the continuity field for expression, a specialized mechanism that promotes the stability of emotion perception, which could help facilitate social interactions and emotion regulation.

Beyond Trial-Based Paradigms: Continuous Behavior, Ongoing Neural Activity, and Natural Stimuli

The vast majority of experiments examining perception and behavior are conducted using experimental paradigms that adhere to a rigid trial structure: each trial consists of a brief and discrete series of events and is regarded as independent from all other trials. The assumptions underlying this structure ignore the reality that natural behavior is rarely discrete, brain activity follows multiple time courses that do not necessarily conform to the trial structure, and the natural environment has statistical structure and dynamics that exhibit long-range temporal correlation. Modern advances in statistical modeling and analysis offer tools that make it feasible for experiments to move beyond rigid independent and identically distributed trial structures. Here we review literature that serves as evidence for the feasibility and advantages of moving beyond trial-based paradigms to understand the neural basis of perception and cognition. Furthermore, we propose a synthesis of these efforts, integrating the characterization of natural stimulus properties with measurements of continuous neural activity and behavioral outputs within the framework of sensory-cognitive-motor loops. Such a framework provides a basis for the study of natural statistics, naturalistic tasks, and/or slow fluctuations in brain activity, which should provide starting points for important generalizations of analytical tools in neuroscience and subsequent progress in understanding the neural basis of perception and cognition.

Visual Adaptation

Sensory systems continuously mold themselves to the widely varying contexts in which they must operate. Studies of these adaptations have played a long and central role in vision science. In part this is because the specific adaptations remain a powerful tool for dissecting vision, by exposing the mechanisms that are adapting. That is, "if it adapts, it's there." Many insights about vision have come from using adaptation in this way, as a method. A second important trend has been the realization that the processes of adaptation are themselves essential to how vision works, and thus are likely to operate at all levels. That is, "if it's there, it adapts." This has focused interest on the mechanisms of adaptation as the target rather than the probe. Together both approaches have led to an emerging insight of adaptation as a fundamental and ubiquitous coding strategy impacting all aspects of how we see.

Different time scales of motion integration for anticipatory smooth pursuit and perceptual adaptation

When repeatedly exposed to moving stimuli, the oculomotor system elicits anticipatory smooth pursuit (ASP) eye movements, even before the stimulus moves. ASP is affected oppositely to perceptual speed judgments of repetitive moving stimuli: After a sequence of fast stimuli, ASP velocity increases, whereas perceived speed decreases. These two effects--perceptual adaptation and oculomotor priming--could result from adapting a single common internal speed representation that is used for perceptual comparisons and for generating ASP. Here we test this hypothesis by assessing the temporal dependence of both effects on stimulus history. Observers performed speed discriminations on moving random dot stimuli, either while pursuing the movement or maintaining steady fixation. In both cases, responses showed perceptual adaptation: Stimuli preceded by fast speeds were perceived as slower, and vice versa. To evaluate oculomotor priming, we analyzed ASP velocity as a function of average stimulus speed in preceding trials and found strong positive dependencies. Interestingly, maximal priming occurred over short stimulus histories (∼two trials), whereas adaptation was maximal over longer histories (∼15 trials). The temporal dissociation of adaptation and priming suggests different underlying mechanisms. It may be that perceptual adaptation integrates over a relatively long period to robustly calibrate the operating range of the motion system, thereby avoiding interference from transient changes in stimulus speed. On the other hand, the oculomotor system may rapidly prime anticipatory velocity to efficiently match it to that of the pursuit target.

Serial dependence in visual perception

Visual input often arrives in a noisy and discontinuous stream, owing to head and eye movements, occlusion, lighting changes, and many other factors. Yet the physical world is generally stable—objects and physical characteristics rarely change spontaneously. How then does the human visual system capitalize on continuity in the physical environment over time? Here we show that visual perception is serially dependent, using both prior and present input to inform perception at the present moment. Using an orientation judgment task, we found that even when visual input changes randomly over time, perceived orientation is strongly and systematically biased toward recently seen stimuli. Further, the strength of this bias is modulated by attention and tuned to the spatial and temporal proximity of successive stimuli. These results reveal a serial dependence in perception characterized by a spatiotemporally tuned, orientation-selective operator—which we call a continuity field—that may promote visual stability over time.