Serial dependence in a simulated clinical visual search task

Visual continuum in non-human animals: serial dependence revealed in dogs

Serial dependence is a recently described phenomenon by which the perceptual evaluation of a stimulus is biased by a previously attended one. By integrating stimuli over time, serial dependence is believed to ensure a stable conscious experience. Despite increasing studies in humans, it is unknown if the process occurs also in other species. Here, we assessed whether serial dependence occurs in dogs. To this aim, dogs were trained on a quantity discrimination task before being presented with a discrimination where one of the discriminanda was preceded by a task-irrelevant stimulus. If dogs are susceptible to serial dependence, the task-irrelevant stimulus was hypothesized to influence the perception of the subsequently presented quantity. Our results revealed that dogs perceived the currently presented quantity to be closer to the one presented briefly before, in accordance with serial dependence. The direction and strength of the effect were comparable to those observed in humans. Data regarding dogs' attention during the task suggest that dogs used two different quantity estimation mechanisms, an indication of a higher cognitive mechanism involved in the process. The present results are the first empirical evidence that serial dependence extends beyond humans, suggesting that the mechanism is shared by phylogenetically distant mammals.

Super recognizers: Increased sensitivity or reduced biases? Insights from serial dependence

Super recognizers (SRs) are people that exhibit a naturally occurring superiority for processing facial identity. Despite the increase of SR research, the mechanisms underlying their exceptional abilities remain unclear. Here, we investigated whether the enhanced facial identity processing of SRs could be attributed to the lack of sequential effects, such as serial dependence. In serial dependence, perception of stimulus features is assimilated toward stimuli presented in previous trials. This constant error in visual perception has been proposed as a mechanism that promotes perceptual stability in everyday life. We hypothesized that an absence of this constant source of error in SRs could account for their superior processing-potentially in a domain-general fashion. We tested SRs (n = 17) identified via a recently proposed diagnostic framework (Ramon, 2021) and age-matched controls (n = 20) with two experiments probing serial dependence in the face and shape domains. In each experiment, observers were presented with randomly morphed face identities or shapes and were asked to adjust a face's identity or a shape to match the stimulus they saw. We found serial dependence in controls and SRs alike, with no difference in its magnitude across groups. Interestingly, we found that serial dependence impacted the performance of SRs more than that of controls. Taken together, our results show that enhanced face identity processing skills in SRs cannot be attributed to the lack of serial dependence. Rather, serial dependence, a beneficial nested error in our visual system, may in fact further stabilize the perception of SRs and thus enhance their visual processing proficiency.

Serial dependence bias can predict the overall estimation error in visual perception

Although visual feature estimations are accurate and precise, overall estimation errors (i.e., the difference between estimates and actual values) tend to show systematic patterns. For example, estimates of orientations are systematically biased away from horizontal and vertical orientations, showing an oblique illusion. Additionally, many recent studies have demonstrated that estimations of current visual features are systematically biased toward previously seen features, showing a serial dependence. However, no study examined whether the overall estimation errors were correlated with the serial dependence bias. To address this question, we enrolled three groups of participants to estimate orientation, motion speed, and point-light-walker direction. The results showed that the serial dependence bias explained over 20% of overall estimation errors in the three tasks, indicating that we could use the serial dependence bias to predict the overall estimation errors. The current study first demonstrated that the serial dependence bias was not independent from the overall estimation errors. This finding could inspire researchers to investigate the neural bases underlying the visual feature estimation and serial dependence.

Spatial correspondence in relative space regulates serial dependence

Our perception is often attracted to what we have seen before, a phenomenon called 'serial dependence.' Serial dependence can help maintain a stable perception of the world, given the statistical regularity in the environment. If serial dependence serves this presumed utility, it should be pronounced when consecutive elements share the same identity when multiple elements spatially shift across successive views. However, such preferential serial dependence between identity-matching elements in dynamic situations has never been empirically tested. Here, we hypothesized that serial dependence between consecutive elements is modulated more effectively by the spatial correspondence in relative space than by that in absolute space because spatial correspondence in relative coordinates can warrant identity matching invariantly to changes in absolute coordinates. To test this hypothesis, we developed a task where two targets change positions in unison between successive views. We found that serial dependence was substantially modulated by the correspondence in relative coordinates, but not by that in absolute coordinates. Moreover, such selective modulation by the correspondence in relative space was also observed even for the serial dependence defined by previous non-target elements. Our findings are consistent with the view that serial dependence subserves object-based perceptual stabilization over time in dynamic situations.

Tuning perception and decisions to temporal context

Recent work suggests that serial dependence, where perceptual decisions are biased toward previous stimuli, arises from the prior that sensory input is temporally correlated. However, existing studies have mostly used random stimulus sequences that do not involve such temporal consistencies. Here, we manipulated the temporal statistics of visual stimuli to examine the role of true temporal correlations in serial dependence. In two experiments, observers reproduced the orientation of the last stimulus in a sequence, while we varied temporal correlations in the stimulus features at two timescales: stimulus history within the trial and decision history across trials. We found a clear dissociation: increasing temporal correlation in the stimulus history led to adaptation-like repulsive biases, whereas increasing temporal correlation in the decision history reduced attractive biases. Thus, we suggest that temporal correlation enhances the discriminative ability of the visual system, revealing the fundamental role of the broader temporal context.

The effects of visual distractors on serial dependence

Attractive serial dependence occurs when perceptual decisions are attracted toward previous stimuli. This effect is mediated by spatial attention and is most likely to occur when similar stimuli are attended at nearby locations. Attention, however, also involves the suppression of distracting information and of spatial locations where distracting stimuli have frequently appeared. Although distractors form an integral part of our visual experience, how they affect the processing of subsequent stimuli is unknown. Here, in two experiments, we tested serial dependence from distractor stimuli during an orientation adjustment task. We interleaved adjustment trials with a discrimination task requiring observers to ignore a peripheral distractor randomly appearing on half of the trials. Distractors were either similar to the adjustment probe (Experiment 1) or differed in spatial frequency and contrast (Experiment 2) and were shown at predictable or random locations in separate blocks. The results showed that the distractor caused considerable attentional capture in the discrimination task, with observers likely using proactive strategies to anticipate distractors at predictable locations. However, there was no evidence that the distractors affected the perceptual stream leading to positive serial dependence. Instead, they left a weak repulsive trace in Experiment 1 and more generally interfered with the effect of the previous adjustment probe in the serial dependence task. We suggest that this repulsive bias may reflect the operation of mechanisms involved in attentional suppression.

Serial dependence in facial identity perception and visual working memory

Serial dependence (SD) refers to the effect in which a person's current perceptual judgment is attracted toward recent stimulus history. Perceptual and memory processes, as well as response and decisional biases, are thought to contribute to SD effects. The current study examined the processing stages of SD facial identity effects in the context of task-related decision processes and how such effects may differ from visual working memory (VWM) interactions. In two experiments, participants were shown a series of two sequentially presented face images. In Experiment 1, the two faces were separated by an interstimulus interval (ISI) of 1, 3, 6, or 10 s, and participants were instructed to reproduce the second face after a varying response delay of 0, 1, 3, 6, or 10 s. Results showed that SD effects occurred most consistently at ISI of 1 s and response delays of 1 and 6 s consistent with early and late stages of processing. In Experiment 2, the ISI was held constant at 1 s, and to separate SD from VWM interactions participants were post-cued to reproduce either the first or the second face. When the second face was the target, SD effects again occurred at response delays of 1 and 6 s, but not when the first face was the target. Together, the results demonstrates that SD facial identity effects occur independently of task-related processes in a distinct temporal fashion and suggest that SD and VWM interactions may rely on separate underlying mechanisms.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Serial dependence in perception across naturalistic generative adversarial network-generated mammogram

Purpose

Human perception and decisions are biased toward previously seen stimuli. This phenomenon is known as serial dependence and has been extensively studied for the last decade. Recent evidence suggests that clinicians' judgments of mammograms might also be impacted by serial dependence. However, the stimuli used in previous psychophysical experiments on this question, consisting of artificial geometric shapes and healthy tissue backgrounds, were unrealistic. We utilized realistic and controlled generative adversarial network (GAN)-generated radiographs to mimic images that clinicians typically encounter.

Approach

Mammograms from the digital database for screening mammography (DDSM) were utilized to train a GAN. This pretrained GAN was then adopted to generate a large set of authentic-looking simulated mammograms: 20 circular morph continuums, each with 147 images, for a total of 2940 images. Using these stimuli in a standard serial dependence experiment, participants viewed a random GAN-generated mammogram on each trial and subsequently matched the GAN-generated mammogram encountered using a continuous report. The characteristics of serial dependence from each continuum were analyzed.

Results

We found that serial dependence affected the perception of all naturalistic GAN-generated mammogram morph continuums. In all cases, the perceptual judgments of GAN-generated mammograms were biased toward previously encountered GAN-generated mammograms. On average, perceptual decisions had 7% categorization errors that were pulled in the direction of serial dependence.

Conclusions

Serial dependence was found even in the perception of naturalistic GAN-generated mammograms created by a GAN. This supports the idea that serial dependence could, in principle, contribute to decision errors in medical image perception tasks.

The Role of Visual Factors in Dyslexia

What are the causes of dyslexia? Decades of research reflect a determined search for a single cause where a common assumption is that dyslexia is a consequence of problems with converting phonological information into lexical codes. But reading is a highly complex activity requiring many well-functioning mechanisms, and several different visual problems have been documented in dyslexic readers. We critically review evidence from various sources for the role of visual factors in dyslexia, from magnocellular dysfunction through accounts based on abnormal eye movements and attentional processing, to recent proposals that problems with high-level vision contribute to dyslexia. We believe that the role of visual problems in dyslexia has been underestimated in the literature, to the detriment of the understanding and treatment of the disorder. We propose that rather than focusing on a single core cause, the role of visual factors in dyslexia fits well with risk and resilience models that assume that several variables interact throughout prenatal and postnatal development to either promote or hinder efficient reading.

Attractive and repulsive serial dependence: The role of task relevance, the passage of time, and the number of stimuli

Visual decisions are attracted toward features of previous stimuli. This phenomenon, termed serial dependence, has been related to a mechanism that integrates present visual input with stimuli seen up to 10 to 15 s in the past. It is believed that this mechanism is "temporally tuned" and the effect of prior stimuli fades with time. Here, we investigated whether the temporal window of serial dependence is influenced by the number of stimuli shown. Observers performed an orientation adjustment task where the interval between the past and present stimulus and the number of intervening stimuli varied. First, we found that the direction-repulsive or attractive-and duration of the effect of a past stimulus depends on whether the past stimulus was relevant to behavior. Second, we show that the number of stimuli, and not only the passage of time, plays a role: The effect of a stimulus at a fixed interval depends on the number of other stimuli shown after. Our results demonstrate that neither a single mechanism nor a general tuning window can fully capture the complexity of serial dependence.

The Representational Similarity between Visual Perception and Recent Perceptual History

From moment to moment, the visual properties of objects in the world fluctuate because of external factors like ambient lighting, occlusion and eye movements, and internal (proximal) noise. Despite this variability in the incoming information, our perception is stable. Serial dependence, the behavioral attraction of current perceptual responses toward previously seen stimuli, may reveal a mechanism underlying stability: a spatiotemporally tuned operator that smooths over spurious fluctuations. The current study examined the neural underpinnings of serial dependence by recording the electroencephalographic (EEG) brain response of female and male human observers to prototypical objects (faces, cars, and houses) and morphs that mixed properties of two prototypes. Behavior was biased toward previously seen objects. Representational similarity analysis (RSA) revealed that responses evoked by visual objects contained information about the previous stimulus. The trace of previous representations in the response to the current object occurred immediately on object appearance, suggesting that serial dependence arises from a brain state or set that precedes processing of new input. However, the brain response to current visual objects was not representationally similar to the trace they leave on subsequent object representations. These results reveal that while past stimulus history influences current representations, this influence does not imply a shared neural code between the previous trial (memory) and the current trial (perception).SIGNIFICANCE STATEMENT The perception of visual objects is pulled toward instances of that object seen in the recent past. The neural underpinnings of this serial dependence remain to be fully investigated. The present study examined electroencephalographic (EEG) responses to faces, cars, and houses, and ambiguous between-category morphs. With representational similarity analysis (RSA), we showed (1) object-specific neural patterns that differentiate the three categories; (2) that the response to the current object contains information about the previous object, mirroring behavioral serial dependence; (3) that the object-specific neural pattern about the past was different from that in the current response, revealing that while past stimulus history influences current representations, this does not imply a shared neural code between the previous trial (memory) and the current trial (perception).

The influence of the tested item on serial dependence in perceptual decisions

Serial dependence in vision reflects how perceptual decisions can be biased by what we have recently perceived. Serial dependence studies test single items' effects on perceptual decisions. However, our visual world contains multiple objects at any given moment, so it's important to understand how past experiences affect not only a single object but also perception in a more general sense. Here we asked the question: What effect does a single item have when there is more than one subsequently presented test item? We displayed a single line (inducer) at the screen center, then either a single test-line or two simultaneous test-lines, varying in orientation space to the inducer. Next, participants reported test-line orientation using a left or right located response circle (to indicate which test-line should be reported). The results demonstrated that the inducer influenced subsequent perceptual judgments of a test-line even when two test-lines were presented. Distant items caused repulsive serial dependence, while close items caused attractive serial dependence. This shows how a single inducer can influence test-line judgments, even when two test-lines are presented, and can produce attractive and repulsive serial dependence biases when the item to report is revealed after it has disappeared.

Perceptual comparisons modulate memory biases induced by new visual inputs

It is well-established that stimulus-specific information in visual working memory (VWM) can be systematically biased by new perceptual inputs. These memory biases are commonly attributed to interference that arises when perceptual inputs are physically similar to VWM contents. However, recent work has suggested that explicitly comparing the similarity between VWM contents and new perceptual inputs modulates the size of memory biases above and beyond stimulus-driven effects. Here, we sought to directly investigate this modulation hypothesis by comparing the size of memory biases following explicit comparisons to those induced when new perceptual inputs are ignored (Experiment 1) or maintained in VWM alongside target information (Experiment 2). We found that VWM reports showed larger attraction biases following explicit perceptual comparisons than when new perceptual inputs were ignored or maintained in VWM. An analysis of participants' perceptual comparisons revealed that memory biases were amplified after perceptual inputs were endorsed as similar-but not dissimilar-to one's VWM representation. These patterns were found to persist even after accounting for variability in the physical similarity between the target and perceptual stimuli across trials, as well as the baseline memory precision between the distinct task demands. Together, these findings illustrate a causal role of perceptual comparisons in modulating naturally-occurring memory biases.

Serial dependence in visual perception: A review

How does the visual system represent continuity in the constantly changing visual input? A recent proposal is that vision is serially dependent: Stimuli seen a moment ago influence what we perceive in the present. In line with this, recent frameworks suggest that the visual system anticipates whether an object seen at one moment is the same as the one seen a moment ago, binding visual representations across consecutive perceptual episodes. A growing body of work supports this view, revealing signatures of serial dependence in many diverse visual tasks. Yet, the variety of disparate findings and interpretations calls for a more general picture. Here, we survey the main paradigms and results over the past decade. We also focus on the challenge of finding a relationship between serial dependence and the concept of "object identity," taking centuries-long history of research into account. Among the seemingly contrasting findings on serial dependence, we highlight common patterns that may elucidate the nature of this phenomenon and attempt to identify questions that are unanswered.

Idiosyncratic biases in the perception of medical images

Introduction

Radiologists routinely make life-altering decisions. Optimizing these decisions has been an important goal for many years and has prompted a great deal of research on the basic perceptual mechanisms that underlie radiologists' decisions. Previous studies have found that there are substantial individual differences in radiologists' diagnostic performance (e.g., sensitivity) due to experience, training, or search strategies. In addition to variations in sensitivity, however, another possibility is that radiologists might have perceptual biases-systematic misperceptions of visual stimuli. Although a great deal of research has investigated radiologist sensitivity, very little has explored the presence of perceptual biases or the individual differences in these.

Methods

Here, we test whether radiologists' have perceptual biases using controlled artificial and Generative Adversarial Networks-generated realistic medical images. In Experiment 1, observers adjusted the appearance of simulated tumors to match the previously shown targets. In Experiment 2, observers were shown with a mix of real and GAN-generated CT lesion images and they rated the realness of each image.

Results

We show that every tested individual radiologist was characterized by unique and systematic perceptual biases; these perceptual biases cannot be simply explained by attentional differences, and they can be observed in different imaging modalities and task settings, suggesting that idiosyncratic biases in medical image perception may widely exist.

Discussion

Characterizing and understanding these biases could be important for many practical settings such as training, pairing readers, and career selection for radiologists. These results may have consequential implications for many other fields as well, where individual observers are the linchpins for life-altering perceptual decisions.

The test-retest reliability and spatial tuning of serial dependence in orientation perception

Humans perceive objects and scenes consistently, even in situations where visual input is noisy and unstable. One of the mechanisms that underlies this perceptual stability is serial dependence, whereby the perception of objects or features at any given moment is pulled toward what was previously seen. Although recent findings from several studies have reported large individual differences in serial dependence, it is not clear how stable the serial dependence is within an individual. Here, we investigated the stability of serial dependence in orientation perception over two different days within the same observers. In addition, we also examined the visual field location specificity of perceptual serial dependence. On each trial, observers viewed a Gabor patch and then reported its apparent orientation by adjusting the orientation of a bar. For each observer, the Gabor was located in the foveal or peripheral (10° right or left eccentricity) visual field on both days or changed location from day to day. The results showed a very high degree of test-retest reliability in serial dependence measured across days within individual observers. Interestingly, this high within-subject consistency was only found when serial dependence was measured at the same visual field location. These results suggest that individual differences in serial dependence are stable across days, and that the spatiotemporal range in which the previous stimulus assimilates the perception of the current stimulus (the continuity field) may vary across different visual field locations in an observer-specific manner.

Controllable Medical Image Generation via GAN

Medical image data is critically important for a range of disciplines, including medical image perception research, clinician training programs, and computer vision algorithms, among many other applications. Authentic medical image data, unfortunately, is relatively scarce for many of these uses. Because of this, researchers often collect their own data in nearby hospitals, which limits the generalizabilty of the data and findings. Moreover, even when larger datasets become available, they are of limited use because of the necessary data processing procedures such as de-identification, labeling, and categorizing, which requires significant time and effort. Thus, in some applications, including behavioral experiments on medical image perception, researchers have used naive artificial medical images (e.g., shapes or textures that are not realistic). These artificial medical images are easy to generate and manipulate, but the lack of authenticity inevitably raises questions about the applicability of the research to clinical practice. Recently, with the great progress in Generative Adversarial Networks (GAN), authentic images can be generated with high quality. In this paper, we propose to use GAN to generate authentic medical images for medical imaging studies. We also adopt a controllable method to manipulate the generated image attributes such that these images can satisfy any arbitrary experimenter goals, tasks, or stimulus settings. We have tested the proposed method on various medical image modalities, including mammogram, MRI, CT, and skin cancer images. The generated authentic medical images verify the success of the proposed method. The model and generated images could be employed in any medical image perception research.

Serial dependence in the perceptual judgments of radiologists

In radiological screening, clinicians scan myriads of radiographs with the intent of recognizing and differentiating lesions. Even though they are trained experts, radiologists’ human search engines are not perfect: average daily error rates are estimated around 3–5%. A main underlying assumption in radiological screening is that visual search on a current radiograph occurs independently of previously seen radiographs. However, recent studies have shown that human perception is biased by previously seen stimuli; the bias in our visual system to misperceive current stimuli towards previous stimuli is called serial dependence. Here, we tested whether serial dependence impacts radiologists’ recognition of simulated lesions embedded in actual radiographs. We found that serial dependence affected radiologists’ recognition of simulated lesions; perception on an average trial was pulled 13% toward the 1-back stimulus. Simulated lesions were perceived as biased towards the those seen in the previous 1 or 2 radiographs. Similar results were found when testing lesion recognition in a group of untrained observers. Taken together, these results suggest that perceptual judgements of radiologists are affected by previous visual experience, and thus some of the diagnostic errors exhibited by radiologists may be caused by serial dependence from previously seen radiographs.

You see what you look for: Targets and distractors in visual search can cause opposing serial dependencies

Visual perception is, at any given moment, strongly influenced by its temporal context-what stimuli have recently been perceived and in what surroundings. We have previously shown that to-be-ignored items produce a bias upon subsequent perceptual decisions that acts in parallel with other biases induced by attended items. However, our previous investigations were confined to biases upon the perceived orientation of a visual search target, and it is unclear whether these biases influence perceptual decisions in a more general sense. Here, we test whether the biases from visual search targets and distractors affect the perceived orientation of a neutral test line, one that is neither a target nor a distractor. To do so, we asked participants to search for an oddly oriented line among distractors and report its location for a few trials and next presented a test line irrelevant to the search task. Participants were asked to report the orientation of the test line. Our results indicate that in tasks involving visual search, targets induce a positive bias upon a neutral test line if their orientations are similar, whereas distractors produce an attractive bias for similar test lines and a repulsive bias if the orientations of the test line and the average orientation of the distractors are far apart in feature space. In sum, our results show that both attentional role and proximity in feature space between previous and current stimuli determine the direction of biases in perceptual decisions.

Serial dependence revealed in history-dependent perceptual templates

In any given perceptual task, the visual system selectively weighs or filters incoming information. The particular set of weights or filters form a kind of template, which reveals the regions or types of information that are particularly useful for a given perceptual decision.1,2 Unfortunately, sensory input is noisy and ever changing. To compensate for these fluctuations, the visual system could adopt a strategy of biasing the templates such that they reflect a temporal smoothing of input, which would be a form of serial dependence.3-5 Here, we demonstrate that perceptual templates are, in fact, altered by serial dependence. Using a simple orientation detection task and classification-image technique, we found that perceptual templates are systematically biased toward previously seen, task-irrelevant orientations. The results of an orientation discrimination task suggest that this shift in perceptual template derives from a change in the perceptual appearance of orientation. Our study reveals how serial dependence biases internal templates of orientation and suggests that the sensitivity of classification-image techniques in general could be improved by taking into account history-dependent fluctuations in templates.

Serial dependence in time and numerosity perception is dimension-specific

The perception of a visual event (e.g., a flock of birds) at the present moment can be biased by a previous perceptual experience (e.g., the perception of an earlier flock). Serial dependence is a perceptual bias whereby a current stimulus appears more similar to a previous one than it actually is. Whereas serial dependence emerges within several visual stimulus dimensions, whether it could simultaneously operate across different dimensions of the same stimulus (e.g., the numerosity and the duration of a visual pattern) remains unclear. Here we address this question by assessing the presence of serial dependence across duration and numerosity, two stimulus dimensions that are often associated and can bias each other. Participants performed either a duration or a numerosity discrimination task, in which they compared a constant reference with a variable test stimulus, varying along the task-relevant dimension (either duration or numerosity). Serial dependence was induced by a task-irrelevant inducer, that is, a stimulus presented before the reference and always varying in both duration and numerosity. The results show systematic serial dependencies only within the task-relevant stimulus dimension, that is, stimulus numerosity affects numerosity perception only, and duration affects duration perception only. Additionally, at least in the numerosity condition, the task-irrelevant dimension of the inducer (duration) had an opposite, repulsive effect. These findings thus show that attractive serial dependence operates in a highly specific fashion and does not transfer across different stimulus dimensions. Instead, the repulsive influence, possibly reflecting perceptual adaptation, can transfer from one dimension to another.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Optimizing perception: Attended and ignored stimuli create opposing perceptual biases

Humans have remarkable abilities to construct a stable visual world from continuously changing input. There is increasing evidence that momentary visual input blends with previous input to preserve perceptual continuity. Most studies have shown that such influences can be traced to characteristics of the attended object at a given moment. Little is known about the role of ignored stimuli in creating this continuity. This is important since while some input is selected for processing, other input must be actively ignored for efficient selection of the task-relevant stimuli. We asked whether attended targets and actively ignored distractor stimuli in an odd-one-out search task would bias observers' perception differently. Our observers searched for an oddly oriented line among distractors and were occasionally asked to report the orientation of the last visual search target they saw in an adjustment task. Our results show that at least two opposite biases from past stimuli influence current perception: A positive bias caused by serial dependence pulls perception of the target toward the previous target features, while a negative bias induced by the to-be-ignored distractor features pushes perception of the target away from the distractor distribution. Our results suggest that to-be-ignored items produce a perceptual bias that acts in parallel with other biases induced by attended items to optimize perception. Our results are the first to demonstrate how actively ignored information facilitates continuity in visual perception.

Controllable Medical Image Generation via Generative Adversarial Networks

Radiologists and pathologists frequently make highly consequential perceptual decisions. For example, visually searching for a tumor and recognizing whether it is malignant can have a life-changing impact on a patient. Unfortunately, all human perceivers-even radiologists-have perceptual biases. Because human perceivers (medical doctors) will, for the foreseeable future, be the final judges of whether a tumor is malignant, understanding and mitigating human perceptual biases is important. While there has been research on perceptual biases in medical image perception tasks, the stimuli used for these studies were highly artificial and often critiqued. Realistic stimuli have not been used because it has not been possible to generate or control them for psychophysical experiments. Here, we propose to use Generative Adversarial Networks (GAN) to create vivid and realistic medical image stimuli that can be used in psychophysical and computer vision studies of medical image perception. Our model can generate tumor-like stimuli with specified shapes and realistic textures in a controlled manner. Various experiments showed the authenticity of our GAN-generated stimuli and the controllability of our model.

Serial dependence alters perceived object appearance

The visual world as it presents itself to our eyes is constantly changing, in contrast with human perceptual experience, which is smooth and stable. One of the posited psychological mechanisms that may contribute to this constructed perceptual stability is the continuity field, a spatiotemporal integration window. The current study examined whether temporal integration, as quantified by serial dependence (SD) between perceived attributes of successive visual stimuli, influenced the subjective appearance of objects or decisional stages in response determination. To do so, an oddball task required participants to directly compare visual objects and decorrelated responses (present/absent) from the visual attribute on which SD may occur (orientation). Results showed that SD could cause a single visual object to appear different from surrounding distractors, leading to modulations of performance. These results argue in favor of a perceptual level of SD, and against decisional accounts.

Context information supports serial dependence of multiple visual objects across memory episodes

Serial dependence is thought to promote perceptual stability by compensating for small changes of an object’s appearance across memory episodes. So far, it has been studied in situations that comprised only a single object. The question of how we selectively create temporal stability of several objects remains unsolved. In a memory task, objects can be differentiated by their to-be-memorized feature (content) as well as accompanying discriminative features (context). We test whether congruent context features, in addition to content similarity, support serial dependence. In four experiments, we observe a stronger serial dependence between objects that share the same context features across trials. Apparently, the binding of content and context features is not erased but rather carried over to the subsequent memory episode. As this reflects temporal dependencies in natural settings, our findings reveal a mechanism that integrates corresponding content and context features to support stable representations of individualized objects over time.

Visual cognition compensates for small changes in an object’s appearance to ensure its perceived continuity. We show that in situations with multiple objects, context features like color, temporal or spatial position are used as anchors to selectively integrate corresponding objects over time.