Implicit processing during change blindness revealed with mouse-contingent and gaze-contingent displays

Top-Down Priors Disambiguate Target and Distractor Features in Simulated Covert Visual Search

Several models of visual search consider visual attention as part of a perceptual inference process, in which top-down priors disambiguate bottom-up sensory information. Many of these models have focused on gaze behavior, but there are relatively fewer models of covert spatial attention, in which attention is directed to a peripheral location in visual space without a shift in gaze direction. Here, we propose a biologically plausible model of covert attention during visual search that helps to bridge the gap between Bayesian modeling and neurophysiological modeling by using (1) top-down priors over target features that are acquired through Hebbian learning, and (2) spatial resampling of modeled cortical receptive fields to enhance local spatial resolution of image representations for downstream target classification. By training a simple generative model using a Hebbian update rule, top-down priors for target features naturally emerge without the need for hand-tuned or predetermined priors. Furthermore, the implementation of covert spatial attention in our model is based on a known neurobiological mechanism, providing a plausible process through which Bayesian priors could locally enhance the spatial resolution of image representations. We validate this model during simulated visual search for handwritten digits among nondigit distractors, demonstrating that top-down priors improve accuracy for estimation of target location and classification, relative to bottom-up signals alone. Our results support previous reports in the literature that demonstrated beneficial effects of top-down priors on visual search performance, while extending this literature to incorporate known neural mechanisms of covert spatial attention.

Memory representations during slow change blindness

Classic change blindness is the phenomenon where seemingly obvious changes that coincide with visual disruptions (such as blinks or brief blanks) go unnoticed by an attentive observer. Some early work into the causes of classic change blindness suggested that any pre-change stimulus representation is overwritten by a representation of the altered post-change stimulus, preventing change detection. However, recent work revealed that, even when observers do maintain memory representations of both the pre- and post-change stimulus states, they can still miss the change, suggesting that change blindness can also arise from a failure to compare the stored representations. Here, we studied slow change blindness, a related phenomenon that occurs even in the absence of visual disruptions when the change occurs sufficiently slowly, to determine whether it could be explained by conclusions from classic change blindness. Across three different slow change blindness experiments we found that observers who consistently failed to notice the change had access to at least two memory representations of the changing display. One representation was precise but short lived: a detailed representation of the more recent stimulus states, but fragile. The other representation lasted longer but was fairly general: stable but too coarse to differentiate the various stages of the change. These findings suggest that, although multiple representations are formed, the failure to compare hypotheses might not explain slow change blindness; even if a comparison were made, the representations would be too sparse (longer term stores) or too fragile (short-lived stores) for such comparison to inform about the change.

Between the Scenes

We constantly move our eyes to new information while inspecting a scene, but these patterns of eye movements change based on the task and goals of the observer. Inhibition of return (IOR) may facilitate visual search by reducing the likelihood of revisiting previously attended locations. However, IOR may present in any visual task, or it may be search-specific. We investigated the presence of IOR in foraging, memorization, change detection, and two versions of visual search. One version of search used a static search array that remained stable throughout the trial, but the second used a scene flickering paradigm similar to the change detection task. IOR was observed in both versions of visual search, memorization, and foraging, but not in change detection. Visual search and change detection both had temporal nonscene components, and we observed that IOR could be maintained despite the scene removal but only for search. Although IOR is maintained in scene coordinates, short disruptions to this scene are insufficient to completely remove the inhibitory tags. Finally, we compare return saccades in trials without a probe and observe fewer return saccades in tasks for which IOR was observed, providing further evidence that IOR might serve as a novelty drive.

Using eye-controlled highlighting techniques to support both serial and parallel processing in visual search

Recent research has developed two eye-controlled highlighting techniques, namely, block highlight display (BHD) and single highlight display (SHD), that enhance information presentation based on a user's current gaze position. The present research aimed to investigate how these techniques facilitate mental processing of users' visual search in high information-density visual environments. In Experiment 1, 60 participants performed 3-, 6-, 9-, and 12-icon visual search tasks. The search times significantly increased as the number of icons increased with the SHD but not with the BHD. In Experiment 2, 40 participants performed a 49-icon visual search task. The search time was faster, and the fixation spatial density was lower with the BHD than with the SHD. These results suggested that the BHD supported parallel processing in the highlighted area and serial processing in the broader display area; thus, the BHD improved search performance compared to the SHD, which primarily supported serial processing.

Sensing and seeing associated with overlapping occipitoparietal activation in simultaneous EEG-fMRI

The presence of a change in a visual scene can influence brain activity and behavior, even in the absence of full conscious report. It may be possible for us to sense that such a change has occurred, even if we cannot specify exactly where or what it was. Despite existing evidence from electroencephalogram (EEG) and eye-tracking data, it is still unclear how this partial level of awareness relates to functional magnetic resonance imaging (fMRI) blood oxygen level dependent (BOLD) activation. Using EEG, fMRI, and a change blindness paradigm, we found multi-modal evidence to suggest that sensing a change is distinguishable from being blind to it. Specifically, trials during which participants could detect the presence of a colour change but not identify the location of the change (sense trials), were compared to those where participants could both detect and localise the change (localise or see trials), as well as change blind trials. In EEG, late parietal positivity and N2 amplitudes were larger for localised changes only, when compared to change blindness. However, ERP-informed fMRI analysis found no voxels with activation that significantly co-varied with fluctuations in single-trial late positivity amplitudes. In fMRI, a range of visual (BA17,18), parietal (BA7,40), and mid-brain (anterior cingulate, BA24) areas showed increased fMRI BOLD activation when a change was sensed, compared to change blindness. These visual and parietal areas are commonly implicated as the storage sites of visual working memory, and we therefore argue that sensing may not be explained by a lack of stored representation of the visual display. Both seeing and sensing a change were associated with an overlapping occipitoparietal network of activation when compared to blind trials, suggesting that the quality of the visual representation, rather than the lack of one, may result in partial awareness during the change blindness paradigm.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Implicit Detection Observation in Different Features, Exposure Duration, and Delay During Change Blindness

To investigate whether implicit detection occurs uniformly during change blindness with single or combination feature stimuli, and whether implicit detection is affected by exposure duration and delay, two one-shot change detection experiments are designed. The implicit detection effect is measured by comparing the reaction times (RTs) of baseline trials, in which stimulus exhibits no change and participants report “same,” and change blindness trials, in which the stimulus exhibits a change but participants report “same.” If the RTs of blindness trials are longer than those of baseline trials, implicit detection has occurred. The strength of the implicit detection effect was measured by the difference in RTs between the baseline and change blindness trials, where the larger the difference, the stronger the implicit detection effect. In both Experiments 1 and 2, the results showed that the RTs of change blindness trials were significantly longer than those of baseline trials. Whether under set size 4, 6, or 8, the RTs of the change blindness trials were significantly longer than those in the baseline trials. In Experiment 1, the difference between the baseline trials’ RTs and change blindness trials’ RTs of the single features was significantly larger than that of the combination features. However, in Experiment 2, the difference between the baseline trials’ RTs and the change blindness trials’ RTs of single features was significantly smaller than that of the combination features. In Experiment 1a, when the exposure duration was shorter, the difference between the baseline and change blindness trials’ RTs was smaller. In Experiment 2, when the delay was longer, the difference between the two trials’ RTs was larger. These results suggest that regardless of whether the change occurs in a single or a combination of features and whether there is a long exposure duration or delay, implicit detection occurs uniformly during the change blindness period. Moreover, longer exposure durations and delays strengthen the implicit detection effect. Set sizes had no significant impact on implicit detection.

Is It Implicit Detection or Perception During Change Blindness?

Purpose

Implicit detection differs from implicit perception. The former includes implicit registration, localisation, identification and comparison of an object. Implicit comparison is not necessary for implicit perception, and should not involve the identification or localisation of objects. While many studies have reported evidence of implicit detection in change blindness, they may, in fact, have only observed implicit perception. In this study, we aimed to find out whether there is implicit detection or perception during the change blindness period.

Methods

In Experiments 1 and 2, we used a simple change detection paradigm, coupled with a speeded attribute discrimination task. Reaction times (RTs) and accuracy of the participants were measured for the speeded attribute discrimination task. We compared differences in RT and accuracy of the invalid and congruent cue trials to find evidence for implicit detection. Invalid trials referred to stimuli where the appearance of the cue does not change, whereas congruent trials involved cued objects with the same attributes as that of the change object. In Experiment 3, a one-shot change detection experiment was conducted, where subjects were required to report whether the objects were the same or different as quickly as possible. We compared the differences in RTs between trials in which the stimulus exhibited a change but participants reported “same” (change blindness trails) and trials in which the stimulus exhibited no change and participants reported “same” (baseline trials), to find evidence for implicit perception.

Results

In Experiments 1 and 2, the difference in accuracy and RTs under invalid and congruent conditions was not significant. We did not observe a validity effect as evidence for implicit localisation or a congruency effect as evidence for identification. In Experiment 3, the RTs were longer in the change blindness relative no-change trials, which indicated that there was implicit perception.

Conclusion

The results of this study showed that there was no evidence supporting implicit detection in colour or orientation as a single or a combination of features. However, we report evidence for implicit perception during the change blindness period. Change may be implicitly perceived, but not located or identified before there is conscious detection.

No Advantage for Separating Overt and Covert Attention in Visual Search

We move our eyes roughly three times every second while searching complex scenes, but covert attention helps to guide where we allocate those overt fixations. Covert attention may be allocated reflexively or voluntarily, and speeds the rate of information processing at the attended location. Reducing access to covert attention hinders performance, but it is not known to what degree the locus of covert attention is tied to the current gaze position. We compared visual search performance in a traditional gaze-contingent display, with a second task where a similarly sized contingent window is controlled with a mouse, allowing a covert aperture to be controlled independently by overt gaze. Larger apertures improved performance for both the mouse- and gaze-contingent trials, suggesting that covert attention was beneficial regardless of control type. We also found evidence that participants used the mouse-controlled aperture somewhat independently of gaze position, suggesting that participants attempted to untether their covert and overt attention when possible. This untethering manipulation, however, resulted in an overall cost to search performance, a result at odds with previous results in a change blindness paradigm. Untethering covert and overt attention may therefore have costs or benefits depending on the task demands in each case.

An EEG study of detection without localisation in change blindness

Previous studies of change blindness have suggested a distinction between detection and localisation of changes in a visual scene. Using a simple paradigm with an array of coloured squares, the present study aimed to further investigate differences in event-related potentials (ERPs) between trials in which participants could detect the presence of a colour change but not identify the location of the change (sense trials), versus those where participants could both detect and localise the change (localise trials). Individual differences in performance were controlled for by adjusting the difficulty of the task in real time. Behaviourally, reaction times for sense, blind, and false alarm trials were distinguishable when comparing across levels of participant certainty. In the EEG data, we found no significant differences in the visual awareness negativity ERP, contrary to previous findings. In the N2pc range, both awareness conditions (localise and sense) were significantly different to trials with no change detection (blind trials), suggesting that this ERP is not dependent on explicit awareness. Within the late positivity range, all conditions were significantly different. These results suggest that changes can be ‘sensed’ without knowledge of the location of the changing object, and that participant certainty scores can provide valuable information about the perception of changes in change blindness.