In sight, out of mind: The role of eye movements in the rapid resumption of visual search

Interruption in visual search: a systematic review

Visual search, the process of trying to find a target presented among distractors, is a much-studied cognitive task. Less well-studied is the condition in which the search task is interrupted before the target is found. The consequences of such interruptions in visual search have been investigated across various disciplines, which has resulted in diverse and at times contradictory findings. The aim of this systematic review is to provide a more cohesive understanding of the effects of interruptions in visual search. For this purpose, we identified 28 studies that met our inclusion criteria. To facilitate a more organized and comprehensive analysis, we grouped the studies based on three dimensions: the search environment, the interruption aftermath, and the type of the interrupting event. While interruptions in visual search are variable and manifest differently across studies, our review provides a foundational scheme for a more cohesive understanding of the subject. This categorization serves as a starting point for exploring potential future directions, which we delineate in our conclusions.

A target contrast signal theory of parallel processing in goal-directed search

Feature Integration Theory (FIT) set out the groundwork for much of the work in visual cognition since its publication. One of the most important legacies of this theory has been the emphasis on feature-specific processing. Nowadays, visual features are thought of as a sort of currency of visual attention (e.g., features can be attended, processing of attended features is enhanced), and attended features are thought to guide attention towards likely targets in a scene. Here we propose an alternative theory - the Target Contrast Signal Theory - based on the idea that when we search for a specific target, it is not the target-specific features that guide our attention towards the target; rather, what determines behavior is the result of an active comparison between the target template in mind and every element present in the scene. This comparison occurs in parallel and is aimed at rejecting from consideration items that peripheral vision can confidently reject as being non-targets. The speed at which each item is evaluated is determined by the overall contrast between that item and the target template. We present computational simulations to demonstrate the workings of the theory as well as eye-movement data that support core predictions of the theory. The theory is discussed in the context of FIT and other important theories of visual search.

Independence of long-term contextual memory and short-term perceptual hypotheses: Evidence from contextual cueing of interrupted search

Observers are able to resume an interrupted search trial faster relative to responding to a new, unseen display. This finding of rapid resumption is attributed to short-term perceptual hypotheses generated on the current look and confirmed upon subsequent looks at the same display. It has been suggested that the contents of perceptual hypotheses are similar to those of other forms of memory acquired long-term through repeated exposure to the same search displays over the course of several trials, that is, the memory supporting “contextual cueing.” In three experiments, we investigated the relationship between short-term perceptual hypotheses and long-term contextual memory. The results indicated that long-term, contextual memory of repeated displays neither affected the generation nor the confirmation of short-term perceptual hypotheses for these displays. Furthermore, the analysis of eye movements suggests that long-term memory provides an initial benefit in guiding attention to the target, whereas in subsequent looks guidance is entirely based on short-term perceptual hypotheses. Overall, the results reveal a picture of both long- and short-term memory contributing to reliable performance gains in interrupted search, while exerting their effects in an independent manner.

Fixations are not all created equal: An objection to mindless visual search

This call to revolution in theories of visual search does not go far enough. Treating fixations as uniform is an oversimplification that obscures the critical role of the mind. We remind readers that what happens during a fixation depends on mindset, as shown in studies of search strategy and of humans' ability to rapidly resume search following an interruption.

The Neural Mechanisms of Prediction in Visual Search

The speed of visual search depends on bottom-up stimulus features (e.g., we quickly locate a red item among blue distractors), but it is also facilitated by the presence of top-down perceptual predictions about the item. Here, we identify the nature, source, and neuronal substrate of the predictions that speed up resumed visual search. Human subjects were presented with a visual search array that was repeated up to 4 times, while brain activity was recorded using magnetoencephalography (MEG). Behaviorally, we observed a bimodal reaction time distribution for resumed visual search, indicating that subjects were extraordinarily rapid on a proportion of trials. MEG data demonstrated that these rapid-response trials were associated with a prediction of (1) target location, as reflected by alpha-band (8-12 Hz) lateralization; and (2) target identity, as reflected by beta-band (15-30 Hz) lateralization. Moreover, we show that these predictions are likely generated in a network consisting of medial superior frontal cortex and right temporo-parietal junction. These findings underscore the importance and nature of perceptual hypotheses for efficient visual search.

The role of prediction in perception: Evidence from interrupted visual search

Recent studies of rapid resumption-an observer's ability to quickly resume a visual search after an interruption-suggest that predictions underlie visual perception. Previous studies showed that when the search display changes unpredictably after the interruption, rapid resumption disappears. This conclusion is at odds with our everyday experience, where the visual system seems to be quite efficient despite continuous changes of the visual scene; however, in the real world, changes can typically be anticipated based on previous knowledge. The present study aimed to evaluate whether changes to the visual display can be incorporated into the perceptual hypotheses, if observers are allowed to anticipate such changes. Results strongly suggest that an interrupted visual search can be rapidly resumed even when information in the display has changed after the interruption, so long as participants not only can anticipate them, but also are aware that such changes might occur.

Rapid resumption of interrupted search is independent of age-related improvements in visual search

In this study, 7-19-year-olds performed an interrupted visual search task in two experiments. Our question was whether the tendency to respond within 500ms after a second glimpse of a display (the rapid resumption effect [Psychological Science, 16 (2005) 684-688]) would increase with age in the same way as overall search efficiency. The results indicated no correlation of rapid resumption with search speed either across age groups (7, 9, 11, and 19years) or at the level of individual participants. Moreover, relocating the target randomly between looks reduced the rate of rapid resumption in a very similar way at each age. These results imply that implicit perceptual prediction during search is invariant across this age range and is distinct from other critical processes such as feature integration and control over spatial attention.

The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search

Eye movements are now widely used to investigate cognitive processes during reading, scene perception, and visual search. In this article, research on the following topics is reviewed with respect to reading: (a) the perceptual span (or span of effective vision), (b) preview benefit, (c) eye movement control, and (d) models of eye movements. Related issues with respect to eye movements during scene perception and visual search are also reviewed. It is argued that research on eye movements during reading has been somewhat advanced over research on eye movements in scene perception and visual search and that some of the paradigms developed to study reading should be more widely adopted in the study of scene perception and visual search. Research dealing with “real-world” tasks and research utilizing the visual-world paradigm are also briefly discussed.

What's next? New evidence for prediction in human vision

Everyday visual experience involves making implicit predictions, as revealed by our surprise when something disturbs our expectations. Many theories of vision have been premised on the central role played by prediction. Yet, implicit prediction in human vision has been difficult to assess in the laboratory, and many results have not distinguished between the indisputably important role of memory and the future-oriented aspect of prediction. Now, a new and unexpected finding - that humans can resume an interrupted visual search much faster than they can start a new search - offers new hope, because the rapid resumption of a search seems to depend on participants forming an implicit prediction of what they will see after the interruption. These findings combined with results of recent neurophysiology studies provide a framework for studying implicit prediction in perception.