Visual search has memory

Inhibitory tagging both speeds and strengthens saccade target selection in the superior colliculus during visual search

It has been suggested that, during difficult visual search tasks involving time pressure and multiple saccades, inhibitory tagging helps to facilitate efficient saccade target selection by reducing responses to objects in the scene once they have been searched and rejected. The superior colliculus (SC) is a midbrain structure involved in target selection, and recent findings suggest an influence of inhibitory tagging on SC activity. Precisely how, and by how much, inhibitory tagging influences target selection by SC neurons, however, is unclear. The purpose of this study, therefore, was to characterize and quantify the influence of inhibitory tagging on target selection in the SC. Rhesus monkeys performed a visual search task involving time pressure and multiple saccades. Early in the fixation period between saccades, a subset of SC neurons reliably discriminated the stimulus selected as the next saccade goal, consistent with a role in target selection. Discrimination occurred earlier and was more robust, however, when unselected stimuli in the search array had been previously fixated on the same trial. This indicates that inhibitory tagging both speeds and strengthens saccade target selection in the SC during multisaccade search. The results provide constraints on models of target selection based on SC activity.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Cerebellar Transcranial Magnetic Stimulation (TMS) Impairs Visual Working Memory

An increasing body of evidence points to the involvement of the cerebellum in cognition. Specifically, previous studies have shown that the superior and inferior portions of the cerebellum are involved in different verbal working memory (WM) mechanisms as part of two separate cerebro-cerebellar loops for articulatory rehearsal and phonological storage mechanisms. In comparison, our understanding of the involvement of the cerebellum in visual WM remains limited. We have previously shown that performance in verbal WM is disrupted by single-pulse transcranial magnetic stimulation (TMS) of the right superior cerebellum. The present study aimed to expand on this notion by exploring whether the inferior cerebellum is similarly involved in visual WM. Here, we used fMRI-guided, double-pulse TMS to probe the necessity of left superior and left inferior cerebellum in visual WM. We first conducted an fMRI localizer using the Sternberg visual WM task, which yielded targets in left superior and inferior cerebellum. Subsequently, TMS stimulation of these regions at the end of the encoding phase resulted in decreased accuracy in the visual WM task. Differences in the visual WM deficits caused by stimulation of superior and inferior left cerebellum raise the possibility that these regions are involved in different stages of visual WM.

Evaluating the dual-task decrement within a simulated environment: Word recall and visual search

Simulated environments have become better able to replicate the real world and can be used for a variety of purposes, such as testing new technology without any of the costs or risks associated with working in the real world. Because of this, it is now possible to gain a better understanding of cognitive demands when working in operational environments, where individuals are often required to multitask. Multitasking often results in performance decrements, where adding more tasks can cause a decrease in performance in each of the individual tasks. However, little research investigated multitasking performance in simulated environments. In the current study we examined how multitasking affects performance in simulated environments. Forty-eight participants performed a dual visual search and word memory task where participants were navigated through a simulated environment while being presented with words. Performance was then compared to single-task performance (visual search and word memory alone). Results showed that participants experienced significant dual-task interference when comparing the dual-tasks to the single-tasks and subjective measures confirmed these findings. These results could provide useful insight for the design of technology in operational environments, but also serve as an evaluation of MRT in simulated environments.

The effects of dual-task interference on visual search and verbal memory

The operational costs of multitasking are more pressing given the increase in wearable technologies (head-up displays; HUDs) that facilitate multitasking. Often multitasking comes with performance costs, where the addition of more tasks impairs the performance of the tasks. The current study explored the extent to which multitasking interference can be characterised in simulated environments, as opposed to risky and harsh environments in real operational contexts. Forty-eight participants completed several trials where they performed a visual search task while navigating a simulated environment. There were three conditions: a standalone memory task, a standalone search task, and both tasks simultaneously. Results revealed significant dual-task interference when comparing the dual-task to each of the single-tasks. Results were corroborated by subjective workload and stress metrics. The results could prove useful for designing systems for individuals who routinely multitask in operational environments. Specifically, by furthering the understanding of their performance capabilities and trade-offs due to multitasking.Practitioner summary: Due to the demands of multitasking in operational environments, quantifying the degree of information lost on each task individually will aid in the understanding of the deficits of multitasking performance. This study shows that deficits in multi-tasking (via a HUD) can be understood in simulated environments to a similar degree as real-world tasks.

A global and local perspective of interruption frequency in a visual search task

We investigated the impact of frequency of interruptions in a simulated medical visual search task. Participants (N = 150) performed the visual search task during which they were interrupted by a number-classification task in 25, 50, or 75% of all trials, respectively, reflecting the frequency conditions (i.e., low, mid, high). Target presence (i.e., present vs. absent) and interruption (i.e., uninterrupted vs. interrupted) were varied within-subjects, and interruption frequency was varied between-subjects. Globally, on a frequency condition level, participants in the low frequency condition had longer mean response times (RT) for the primary visual search task than in the high condition, but there were no other performance differences between the three frequency conditions. Locally, on the level of specific interruption effects, accuracy decreased directly after interruptions for target present but not for target absent trials. Furthermore, interruptions caused significant interruption costs, reflected in slower overall RTs in interrupted than in uninterrupted trials. The combined findings show that especially for critical visual search tasks as in the medical field, interruptions—regardless of frequency—should be avoided.

Revealing Whole-Brain Causality Networks During Guided Visual Searching

In our daily lives, we use eye movements to actively sample visual information from our environment ("active vision"). However, little is known about how the underlying mechanisms are affected by goal-directed behavior. In a study of 31 participants, magnetoencephalography was combined with eye-tracking technology to investigate how interregional interactions in the brain change when engaged in two distinct forms of active vision: freely viewing natural images or performing a guided visual search. Regions of interest with significant fixation-related evoked activity (FRA) were identified with spatiotemporal cluster permutation testing. Using generalized partial directed coherence, we show that, in response to fixation onset, a bilateral cluster consisting of four regions (posterior insula, transverse temporal gyri, superior temporal gyrus, and supramarginal gyrus) formed a highly connected network during free viewing. A comparable network also emerged in the right hemisphere during the search task, with the right supramarginal gyrus acting as a central node for information exchange. The results suggest that all four regions are vital to visual processing and guiding attention. Furthermore, the right supramarginal gyrus was the only region where activity during fixations on the search target was significantly negatively correlated with search response times. Based on our findings, we hypothesize that, following a fixation, the right supramarginal gyrus supplies the right supplementary eye field (SEF) with new information to update the priority map guiding the eye movements during the search task.

Humans represent the precision and utility of information acquired across fixations

Our environment contains an abundance of objects which humans interact with daily, gathering visual information using sequences of eye-movements to choose which object is best-suited for a particular task. This process is not trivial, and requires a complex strategy where task affordance defines the search strategy, and the estimated precision of the visual information gathered from each object may be used to track perceptual confidence for object selection. This study addresses the fundamental problem of how such visual information is metacognitively represented and used for subsequent behaviour, and reveals a complex interplay between task affordance, visual information gathering, and metacogntive decision making. People fixate higher-utility objects, and most importantly retain metaknowledge about how much information they have gathered about these objects, which is used to guide perceptual report choices. These findings suggest that such metacognitive knowledge is important in situations where decisions are based on information acquired in a temporal sequence.

A sensory memory to preserve visual representations across eye movements

Saccadic eye movements (saccades) disrupt the continuous flow of visual information, yet our perception of the visual world remains uninterrupted. Here we assess the representation of the visual scene across saccades from single-trial spike trains of extrastriate visual areas, using a combined electrophysiology and statistical modeling approach. Using a model-based decoder we generate a high temporal resolution readout of visual information, and identify the specific changes in neurons' spatiotemporal sensitivity that underly an integrated perisaccadic representation of visual space. Our results show that by maintaining a memory of the visual scene, extrastriate neurons produce an uninterrupted representation of the visual world. Extrastriate neurons exhibit a late response enhancement close to the time of saccade onset, which preserves the latest pre-saccadic information until the post-saccadic flow of retinal information resumes. These results show how our brain exploits available information to maintain a representation of the scene while visual inputs are disrupted.

Sources of variation in search and foraging: A theoretical perspective

Search-the problem of exploring a space of alternatives to identify target goals-is a fundamental behaviour for many species. Although its foundation lies in foraging, most studies of human search behaviour have been directed towards understanding the attentional mechanisms that underlie the efficient visual exploration of two-dimensional (2D) scenes. With this review, we aim to characterise how search behaviour can be explained across a wide range of contexts, environments, spatial scales, and populations, both typical and atypical. We first consider the generality of search processes across psychological domains. We then review studies of interspecies differences in search. Finally, we explore in detail the individual and contextual variables that affect visual search and related behaviours in established experimental psychology paradigms. Despite the heterogeneity of the findings discussed, we identify that variations in control processes, along with the ability to regulate behaviour as a function of the structure of search space and the sampling processes adopted, to be central to explanations of variations in search behaviour. We propose a tentative theoretical model aimed at integrating these notions and close by exploring questions that remain unaddressed.

From "satisfaction of search" to "subsequent search misses": a review of multiple-target search errors across radiology and cognitive science

For over 50 years, the satisfaction of search effect has been studied within the field of radiology. Defined as a decrease in detection rates for a subsequent target when an initial target is found within the image, these multiple target errors are known to underlie errors of omission (e.g., a radiologist is more likely to miss an abnormality if another abnormality is identified). More recently, they have also been found to underlie lab-based search errors in cognitive science experiments (e.g., an observer is more likely to miss a target 'T' if a different target 'T' was detected). This phenomenon was renamed the subsequent search miss (SSM) effect in cognitive science. Here we review the SSM literature in both radiology and cognitive science and discuss: (1) the current SSM theories (i.e., satisfaction, perceptual set, and resource depletion theories), (2) the eye movement errors that underlie the SSM effect, (3) the existing efforts tested to alleviate SSM errors, and (4) the evolution of methodologies and analyses used when calculating the SSM effect. Finally, we present the attentional template theory, a novel mechanistic explanation for SSM errors, which ties together our current understanding of SSM errors and the attentional template literature.

Guided Search 6.0: An updated model of visual search

This paper describes Guided Search 6.0 (GS6), a revised model of visual search. When we encounter a scene, we can see something everywhere. However, we cannot recognize more than a few items at a time. Attention is used to select items so that their features can be "bound" into recognizable objects. Attention is "guided" so that items can be processed in an intelligent order. In GS6, this guidance comes from five sources of preattentive information: (1) top-down and (2) bottom-up feature guidance, (3) prior history (e.g., priming), (4) reward, and (5) scene syntax and semantics. These sources are combined into a spatial "priority map," a dynamic attentional landscape that evolves over the course of search. Selective attention is guided to the most active location in the priority map approximately 20 times per second. Guidance will not be uniform across the visual field. It will favor items near the point of fixation. Three types of functional visual field (FVFs) describe the nature of these foveal biases. There is a resolution FVF, an FVF governing exploratory eye movements, and an FVF governing covert deployments of attention. To be identified as targets or rejected as distractors, items must be compared to target templates held in memory. The binding and recognition of an attended object is modeled as a diffusion process taking > 150 ms/item. Since selection occurs more frequently than that, it follows that multiple items are undergoing recognition at the same time, though asynchronously, making GS6 a hybrid of serial and parallel processes. In GS6, if a target is not found, search terminates when an accumulating quitting signal reaches a threshold. Setting of that threshold is adaptive, allowing feedback about performance to shape subsequent searches. Simulation shows that the combination of asynchronous diffusion and a quitting signal can produce the basic patterns of response time and error data from a range of search experiments.

Avoiding potential pitfalls in visual search and eye-movement experiments: A tutorial review

Examining eye-movement behavior during visual search is an increasingly popular approach for gaining insights into the moment-to-moment processing that takes place when we look for targets in our environment. In this tutorial review, we describe a set of pitfalls and considerations that are important for researchers – both experienced and new to the field – when engaging in eye-movement and visual search experiments. We walk the reader through the research cycle of a visual search and eye-movement experiment, from choosing the right predictions, through to data collection, reporting of methodology, analytic approaches, the different dependent variables to analyze, and drawing conclusions from patterns of results. Overall, our hope is that this review can serve as a guide, a talking point, a reflection on the practices and potential problems with the current literature on this topic, and ultimately a first step towards standardizing research practices in the field.

Perceptual Load and Sex-Specific Personality Traits

The impact of sex-specific personality traits has often been investigated for visuospatial tasks such as mental rotation, but less is known about the influence of personality traits on visual search. We investigated whether the Big Five personality traits Extroversion (E), Openness (O), Agreeableness (A), Conscientiousness (C), and Neuroticism (N) and the Autism Quotient (AQ) influence visual search in a sample of N = 65 men and women. In three experiments, we varied stimulus complexity and predictability. As expected, latencies were longer when the target was absent. Pop-out search was faster than conjunction search. A large number of distracters slowed down reaction times (RTs). When stimulus complexity was not predictable in Experiment 3, this reduced search accuracy by about half. As could be predicted based on previous research on long RT tails, conjunction search in target absent trials revealed the impact of personality traits. The RT effect in visual search of the accelerating "less social" AQ score was specific to men, while the effects of the "more social" decelerating Big Five Inventory factors agreeableness and conscientiousness were specific to women. Thus, sex-specific personality traits could explain decision-making thresholds, while visual stimulus complexity yielded an impact of the classic personality traits neuroticism and extroversion.

Ketamine-Induced Alteration of Working Memory Utility during Oculomotor Foraging Task in Monkeys

Impairments of working memory (WM) are commonly observed in a variety of neurodegenerative disorders but they are difficult to quantitatively assess in clinical cases. Recent studies in experimental animals have used low-dose ketamine (an NMDA receptor antagonist) to disrupt WM, partly mimicking the pathophysiology of schizophrenia. Here, we developed a novel behavioral paradigm to assess multiple components of WM and applied it to monkeys with and without ketamine administration. In an oculomotor foraging task, the animals were presented with 15 identical objects on the screen. One of the objects was associated with a liquid reward, and monkeys were trained to search for the target by generating sequential saccades under a time constraint. We assumed that the occurrence of recursive movements to the same object might reflect WM dysfunction. We constructed a "foraging model" that incorporated (1) memory capacity, (2) memory decay, and (3) utility rate; this model was able to explain more than 92% of the variations in behavioral data obtained from three monkeys. Following systemic administration of low dosages of ketamine, the memory capacity and utility rate were dramatically reduced by 15% and 57%, respectively, while memory decay remained largely unchanged. These results suggested that the behavioral deficits during the blockade of NMDA receptors were mostly due to the decreased usage of short-term memory. Our oculomotor paradigm and foraging model appear to be useful for quantifying multiple components of WM and could be applicable to clinical cases in future studies.

Foraging behavior in visual search: A review of theoretical and mathematical models in humans and animals

Visual search (VS) is a fundamental task in daily life widely studied for over half a century. A variant of the classic paradigm-searching one target among distractors-requires the observer to look for several (undetermined) instances of a target (so-called foraging) or several targets that may appear an undefined number of times (recently named as hybrid foraging). In these searches, besides looking for targets, the observer must decide how much time is needed to exploit the area, and when to quit the search to eventually explore new search options. In fact, visual foraging is a very common search task in the real world, probably involving additional cognitive functions than typical VS. It has been widely studied in natural animal environments, for which several mathematical models have been proposed, and just recently applied to humans: Lévy processes, composite and area-restricted search models, marginal value theorem, and Bayesian learning (among others). We conducted a systematic search in the literature to understand those mathematical models and study its applicability in human visual foraging. The review suggests that these models might be the first step, but they seem to be limited to fully comprehend foraging in visual search. There are essential variables involving human visual foraging still to be established and understood. Indeed, a jointly theoretical interpretation based on the different models reviewed could better account for its understanding. In addition, some other relevant variables, such as certain individual differences or time perception might be crucial to understanding visual foraging in humans.

Maintaining rejected distractors in working memory during visual search depends on search stimuli: Evidence from contralateral delay activity

The presence of memory for rejected distractors during visual search has been heavily debated in the literature and has proven challenging to investigate behaviorally. In this research, we used an electrophysiological index of working memory (contralateral delay activity) to passively measure working memory activity during visual search. Participants were asked to indicate whether a novel target was present or absent in a lateralized search array with three visual set sizes (2, 4, or 6). If rejected distractors are maintained in working memory during search, working memory activity should increase with the number of distractors that need to be evaluated. Therefore, we predicted the amplitude of the contralateral delay activity would be larger for target-absent trials and would increase with visual set size until WM capacity was reached. In Experiment 1, we found no evidence for distractor maintenance in working memory during search for real-world stimuli. In Experiment 2, we found partial evidence in support of distractor maintenance during search for stimuli with high target/distractor similarity. In both experiments, working memory capacity did not appear to be a limiting factor during visual search. These results suggest the role of working memory during search may depend on the visual search task in question. Maintaining distractors in working memory appears to be unnecessary during search for realistic stimuli. However, there appears to be a limited role for distractor maintenance during search for artificial stimuli with a high degree of feature overlap.

Role of aging and working memory in performance on a naturalistic visual search task

Studying age-related changes in working memory (WM) and visual search can provide insights into mechanisms of visuospatial attention. In visual search, WM is used to remember previously inspected objects/locations and to maintain a mental representation of the target to guide the search. We sought to extend this work, using aging as a case of reduced WM capacity. The present study tested whether various domains of WM would predict visual search performance in both young (n = 47; aged 18-35 yrs) and older (n = 48; aged 55-78) adults. Participants completed executive and domain-specific WM measures, and a naturalistic visual search task with (single) feature and triple-conjunction (three-feature) search conditions. We also varied the WM load requirements of the search task by manipulating whether a reference picture of the target (i.e., target template) was displayed during the search, or whether participants needed to search from memory. In both age groups, participants with better visuospatial executive WM were faster to locate complex search targets. Working memory storage capacity predicted search performance regardless of target complexity; however, visuospatial storage capacity was more predictive for young adults, whereas verbal storage capacity was more predictive for older adults. Displaying a target template during search diminished the involvement of WM in search performance, but this effect was primarily observed in young adults. Age-specific interactions between WM and visual search abilities are discussed in the context of mechanisms of visuospatial attention and how they may vary across the lifespan.

Contextual Cueing Effect Under Rapid Presentation

In contextual cueing, previously encountered context tends to facilitate the detection of the target embedded in it than when the target appears in a novel context. In this study, we investigated whether the contextual cueing could develop at early time when the search display was presented briefly. In four experiments, participants searched for a target T in an array of distractor Ls. The results showed that with a rather short presentation time of the search display, participants were able to learn the spatial context and speeded up their response time overall, with the learning effect lasting for a long period. Specifically, the contextual cueing effect was observed either with or without a mask after a duration of 300-ms presentation of the search display. Such a context learning under rapid presentation could not operate only with the local context information repeated, thus suggesting that a global context was required to guide spatial attention when the viewing time of the search display was limited. Overall, these findings indicate that contextual cueing might arise at an "early," target selection stage and that the global context is necessary for the context learning under rapid presentation to function.

When Natural Behavior Engages Working Memory

Working memory (WM) enables temporary storage and manipulation of information,¹ supporting tasks that require bridging between perception and subsequent behavior. Its properties, such as its capacity, have been thoroughly investigated in highly controlled laboratory tasks.1, 2, 3, 4, 5, 6, 7, 8 Much less is known about the utilization and properties of WM in natural behavior,9, 10, 11 when reliance on WM emerges as a natural consequence of interactions with the environment. We measured the trade-off between reliance on WM and gathering information externally during immersive behavior in an adapted object-copying task.¹² By manipulating the locomotive demands required for task completion, we could investigate whether and how WM utilization changed as gathering information from the environment became more effortful. Reliance on WM was lower than WM capacity measures in typical laboratory tasks. A clear trade-off also occurred. As sampling information from the environment required increasing locomotion and time investment, participants relied more on their WM representations. This reliance on WM increased in a shallow and linear fashion and was associated with longer encoding durations. Participants’ avoidance of WM usage showcases a fundamental dependence on external information during ecological behavior, even if the potentially storable information is well within the capacity of the cognitive system. These foundational findings highlight the importance of using immersive tasks to understand how cognitive processes unfold within natural behavior. Our novel VR approach effectively combines the ecological validity, experimental rigor, and sensitive measures required to investigate the interplay between memory and perception in immersive behavior.

Video Abstract

•

Gaze provides a measure of working-memory (WM) usage during natural behavior

•

Natural reliance on WM is low even when searching for objects externally is effortful

•

WM utilization increases linearly as searching for objects requires more locomotion

•

The trade-off between using WM versus external sampling affects performance

Visual search strategies in children: A reflection of working memory processes?

Visual search is not only less accurate but also less organized in children than in adults. However, visual search strategies in children have not been extensively studied and they are not evaluated in clinical practice. Working memory could play a critical role for the execution and the maintaining of a visual search strategy. Few studies if any have explored the links between visual search organization and working memory in children. In the present study, 54 primary school children performed a cancellation task (Bells test) and working memory tests (span tasks). Our results suggested that, contrary to visual search accuracy, visual search organization was significantly linked to working memory and, more specifically, to the efficiency of the central executive component. There is, thus, a real need to better understand the visual search process and to improve its assessment with cancellation tests in clinical practice.

Neural correlates of task-related refixation behavior

Eye movement research has shown that attention shifts from the currently fixated location to the next before a saccade is executed. We investigated whether the cost of the attention shift depends on higher-order processing at the time of fixation, in particular on visual working memory load differences between fixations and refixations on task-relevant items. The attention shift is reflected in EEG activity in the saccade-related potential (SRP). In a free viewing task involving visual search and memorization of multiple targets amongst distractors, we compared the SRP in first fixations versus refixations on targets and distractors. The task-relevance of targets implies that more information will be loaded in memory (e.g. both identity and location) than for distractors (e.g. location only). First fixations will involve greater memory load than refixations, since first fixations involve loading of new items, while refixations involve rehearsal of previously visited items. The SRP in the interval preceding the saccade away from a target or distractor revealed that saccade preparation is affected by task-relevance and refixation behavior. For task-relevant items only, we found longer fixation duration and higher SRP amplitudes for first fixations than for refixations over the occipital region and the opposite effect over the frontal region. Our findings provide first neurophysiological evidence that working memory loading of task-relevant information at fixation affects saccade planning.

Long-term memory and hippocampal function support predictive gaze control during goal-directed search

Eye movements during visual search change with prior experience for search stimuli. Previous studies measured these gaze effects shortly after initial viewing, typically during free viewing; it remains open whether the effects are preserved across long delays and for goal-directed search, and which memory system guides gaze. In Experiment 1, we analyzed eye movements of healthy adults viewing novel and repeated scenes while searching for a scene-embedded target. The task was performed across different time points to examine the repetition effects in long-term memory, and memory types were grouped based on explicit recall of targets. In Experiment 2, an amnesic person with bilateral extended hippocampal damage and the age-matched control group performed the same task with shorter intervals to determine whether or not the repetition effects depend on hippocampal function. When healthy adults explicitly remembered repeated target-scene pairs, search time and fixation duration decreased, and gaze was directed closer to the target region, than when they forgot targets. These effects were seen even after a one-month delay from their initial viewing, suggesting the effects are associated with long-term, explicit memory. Saccadic amplitude was not strongly modulated by scene repetition or explicit recall of targets. The amnesic person did not show explicit recall or implicit repetition effects, whereas his control group showed similar patterns to those seen in Experiment 1. The results reveal several aspects of gaze control that are influenced by long-term memory. The dependence of gaze effects on medial temporal lobe integrity support a role for this region in predictive gaze control.

Visual working memory load does not affect the overall stimulus processing time in visual search

The dual-task paradigm is widely used in studying the interaction between visual search and working memory. A number of studies showed that holding items in working memory delays the overall response time (RT) in visual search, but it does not affect the efficiency of search (i.e., the slope of the RT × set size function). Why the memory load merely affects the overall RT? Some researchers proposed that this load-effect on overall RT may be caused by factors that only affect response selection processes, while others argued that it may reflect the effect of visual working memory load on visual search. This study investigated the two competing hypotheses by measuring the threshold stimulus exposure duration (TSED) for successfully fulfilling a search task. Experiment 1 replicated the large overall RT difference with the RT method but only found a small though reliable overall TSED difference with the TSED method. Experiment 2, with better controls, found no TSED difference by manipulating the visual working memory load. Experiment 3 showed that the TSED is not influenced by processes in the response selection stage. The present findings suggest that the overall stimulus processing time in visual search is not affected by visual working memory load and that the effect of memory load on overall RT is largely due to factors affecting response selection alone.

The confirmation and prevalence biases in visual search reflect separate underlying processes

Research by Rajsic, Wilson, and Pratt (2015, 2017) suggests that people are biased to use a target-confirming strategy when performing simple visual search. In 3 experiments, we sought to determine whether another stubborn phenomenon in visual search, the low-prevalence effect (Wolfe, Horowitz, & Kenner, 2005), would modulate this confirmatory bias. We varied the reliability of the initial cue: For some people, targets usually occurred in the cued color (high prevalence). For others, targets rarely matched the cues (low prevalence). High cue-target prevalence exacerbated the confirmation bias, indexed via search response times (RTs) and eye-tracking measures. Surprisingly, given low cue-target prevalence, people remained biased to examine cue-colored letters, even though cue-colored targets were exceedingly rare. At the same time, people were more fluent at detecting the more common, cue-mismatching targets. The findings suggest that attention is guided to "confirm" the more available cued target template, but prevalence learning over time determines how fluently objects are perceptually appreciated. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Visual Search May Not Require Target Representation in Working Memory or Long-Term Memory

We spend a lot of time searching for things. If we know what we are looking for in advance, a memory representation of the target will be created to guide search. But if the identity of the search target is revealed simultaneously with the presentation of the search array, is a similar memory representation formed? In the present study, 96 observers determined whether a central target was present in a peripheral search array. The results revealed that as long as the central target remained available for inspection (even if only in iconic memory), observers reinspected it after each distractor was checked, apparently forgoing consolidation of the target into working memory. The present findings challenged the assumption that evaluating items in a search array must involve comparison with a template in working memory.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Failures in top-down control in schizophrenia revealed by patterns of saccadic eye movements

Successful execution of many behavioral goals relies on well-organized patterns of saccadic eye movements, and in complex tasks, these patterns can reveal the component processes underlying task performance. The present study examined the pattern of eye movements in a visual search task to provide evidence of attentional control impairments in people with schizophrenia (PSZ). We tested PSZ(N = 38) and nonpsychiatric control subjects (NCS, N = 35) in a task that was designed to stress top-down control by pitting task goals against bottom-up salience. Participants searched for either a low-contrast (nonsalient) or a high-contrast (salient) target among low- and high-contrast distractors. By examining fixations of the low- and high-contrast items, we evaluated the ability of PSZ and NCS to focus on low-salience targets and filter out high-salience distractors (or vice versa). When participants searched for a salient target, both groups successfully focused on relevant, high-contrast stimuli and filtered out target-mismatched, low-contrast stimuli. However, when searching for a nonsalient target, PSZ were impaired at efficiently suppressing high-contrast (salient) distractors. Specifically, PSZ were more likely than NCS to fixate and revisit salient distractors, and they dwelled on these items longer than did NCS. The results provide direct evidence that PSZ are impaired in their ability to utilize top-down goals to overcome the prepotent tendency to focus attention on irrelevant but highly salient information. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Post-search IOR: Searching for inhibition of return after search

Previous research has indicated that Inhibition of return (IOR) supports visual search by discouraging the re-inspection of recently inspected items during search. However, it is not clear whether IOR persists after a search is completed or whether this depends on the presence of a further search in the same display. To investigate this issue, we had participants search consecutively twice in the same display (Experiment 1). Immediately after the end of the first search and after the end of the second search we probed an item which had been recently inspected or not in the previous search. The results showed that IOR as measured by the saccadic latency to the probed items was absent after the end of each of the two successive searches. In Experiment 2, we measured both saccadic latencies and manual responses in a single-search paradigm. We found that IOR during and after the search was present for saccadic responses but absent for manual responses. This suggests that IOR during and after a visual search depends on the modality of the response and the number of required searches.

Refixation patterns reveal memory-encoding strategies in free viewing

We investigated visual working memory encoding across saccadic eye movements, focusing our analysis on refixation behavior. Over 10-s periods, participants performed a visual search for three, four, or five targets and remembered their orientations for a subsequent change-detection task. In 50% of the trials, one of the targets had its orientation changed. From the visual search period, we scored three types of refixations and applied measures for quantifying eye-fixation recurrence patterns. Repeated fixations on the same regions as well as repeated fixation patterns increased with memory load. Correct change detection was associated with more refixations on targets and less on distractors, with increased frequency of recurrence, and with longer intervals between refixations. The results are in accordance with the view that patterns of eye movement are an integral part of visual working memory representation.

Effects of magnification modes and location cues on visual inspection performance

Image magnification often results in disorientation through loss of orientation and location during inspection. This study investigated the effects of three different magnification modes viz. full screen, circular, and fixed-area on visual inspection performance. Also, to improve participants’ global orientation with respect to the original product, location cues in the form of halftone landmarks were introduced as a job aid and their effectiveness on inspection performance was examined. Twenty-eight undergraduates participated in the experiment. Significant magnification mode effect was found, but the location cue effect was found non-significant. The results suggested that the presentation of content/contextual information on one single screen should be considered together with the nature of the visual task and participants’ search behaviors, and that the aid of location cues might be useful when the visual task demanded a high level of search memory and/or an unsystematic search strategy was employed by inspectors.

Neural Processing of Repeated Search Targets Depends Upon the Stimuli: Real World Stimuli Engage Semantic Processing and Recognition Memory

Recent evidence has suggested that visual working memory (VWM) plays an important role in representing the target prior to initiating a visual search. The more familiar we are with the search target, the more refined the representation of the target (or "target template") becomes. This sharpening of the target template is thought to underlie the reduced response time (RT) and increased accuracy associated with repeatedly searching for the same target. Perhaps target representations transition from limited-capacity VWM to Long-Term Memory (LTM) as targets repeat. In prior work, amplitude of an event-related potential (ERP) component associated with VWM representation decreased with target repetition, broadly supporting this notion. However, previous research has focused on artificial stimuli (Landolt Cs) that are far removed from search targets in the real world. The current study extends this work by directly comparing target representations for artificial stimuli and common object images. We found VWM representation follows the same pattern for real and artificial stimuli. However, the initial selection of the real world objects follows a much different pattern than more typical artificial stimuli. Further, the morphology of nonlateralized waveforms was substantially different for the two stimulus categories. This suggests that the two types of stimuli were processed in fundamentally different ways. We conclude that object type strongly influences how we deploy attentional and mnemonic resources prior to search. Early attentional selection of familiar objects may facilitate additional LTM processes that lead to behavioral benefits not seen with more simplistic stimuli.

Refixation control in free viewing: a specialized mechanism divulged by eye-movement-related brain activity

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eye-movement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. NEW & NOTEWORTHY A substantial proportion of eye fixations in human natural viewing behavior are revisits of recently visited locations, i.e., refixations. Our recently developed methods enabled us to study refixations in a free viewing visual search task, using combined eye movement and EEG recording. We identified in the EEG a distinctive refixation-related signal, signifying a control mechanism specific to refixations as opposed to ordinary eye fixations.

Long-Term Visual Memory and Its Role in Learning Suppression

Long-term memory is a core aspect of human learning that permits a wide range of skills and behaviors often important for survival. While this core ability has been broadly observed for procedural and declarative memory, whether similar mechanisms subserve basic sensory or perceptual processes remains unclear. Here, we use a visual learning paradigm to show that training humans to search for common visual features in the environment leads to a persistent improvement in performance over consecutive days but, surprisingly, suppresses the subsequent ability to learn similar visual features. This suppression is reversed if the memory is prevented from consolidating, while still permitting the ability to learn multiple visual features simultaneously. These findings reveal a memory mechanism that may enable salient sensory patterns to persist in memory over prolonged durations, but which also functions to prevent false-positive detection by proactively suppressing new learning.

The effects of array structure and secondary cognitive task demand on processes of visual search

Many aspects of our everyday behaviour require that we search for objects. However, in real situations search is often conducted while internal and external factors compete for our attention resources. Cognitive distraction interferes with our ability to search for targets, increasing search times. Here we consider whether effects of cognitive distraction interfere differentially with three distinct phases of search: initiating search, overtly scanning through items in the display, and verifying that the object is indeed the target of search once it has been fixated. Furthermore, we consider whether strategic components of visual search that emerge when searching items organized into structured arrays are susceptible to cognitive distraction or not. We used Gilchrist & Harvey's (2006) structured and unstructured visual search paradigm with the addition of Savage, Potter, and Tatler's (2013) secondary puzzle task. Cognitive load influenced two phases of search: 1) scanning times and 2) verification times. Under high load, fixation durations were longer and re-fixations of distracters were more common. In terms of scanning strategy, we replicated Gilchrist and Harvey's (2006) findings of more systematic search for structured arrays than unstructured ones. We also found an effect of cognitive load on this aspect of search but only in structured arrays. Our findings suggest that our eyes, by default, produce an autonomous scanning pattern that is modulated but not completely eliminated by secondary cognitive load.

What Does It Take to Search Organized? The Cognitive Correlates of Search Organization During Cancellation After Stroke

OBJECTIVES

Stroke could lead to deficits in organization of visual search. Cancellation tests are frequently used in standard neuropsychological assessment and appear suitable to measure search organization. The current aim was to evaluate which cognitive functions are associated with cancellation organization measures after stroke.

METHODS

Stroke patients admitted to inpatient rehabilitation were included in this retrospective study. We performed exploratory factor analyses to explore cognitive domains. A digital shape cancellation test (SC) was administered, and measures of search organization (intersections rate and best r) were computed. The following cognitive functions were measured by neuropsychological testing: neglect (SC, line bisection; LB, Catherine Bergego Scale; CBS, and Balloons Test), visuospatial perception and construction (Rey Complex Figure Test, RCFT), psychomotor speed (Trail Making Test; TMT-A), executive functioning/working memory (TMT-B), spatial planning (Tower Test), rule learning (Brixton Test), short-term auditory memory (Digit Span Forward; DSF), and verbal working memory (Digit Span Backward; DSB).

RESULTS

In total, 439 stroke patients were included in our analyses. Four clusters were separated: "Executive functioning" (TMT-A, TMT-B, Brixton Test, and Tower Test), "Verbal memory" (DSF and DSB), "Search organization" (intersections rate and best r), and "Neglect" (CBS, RCFT copy, Balloons Test, SC, and LB).

CONCLUSIONS

Search organization during cancellation, as measured with intersections rate and best r, seems a distinct cognitive construct compared to existing cognitive domains that are tested during neuropsychological assessment. Administering cancellation tests and analyzing measures of search organization could provide useful additional insights into the visuospatial processes of stroke patients. (JINS, 2018, 24, 424-436).

Visual search for changes in scenes creates long-term, incidental memory traces

Humans are very good at remembering large numbers of scenes over substantial periods of time. But how good are they at remembering changes to scenes? In this study, we tested scene memory and change detection two weeks after initial scene learning. In Experiments 1-3, scenes were learned incidentally during visual search for change. In Experiment 4, observers explicitly memorized scenes. At test, after two weeks observers were asked to discriminate old from new scenes, to recall a change that they had detected in the study phase, or to detect a newly introduced change in the memorization experiment. Next, they performed a change detection task, usually looking for the same change as in the study period. Scene recognition memory was found to be similar in all experiments, regardless of the study task. In Experiment 1, more difficult change detection produced better scene memory. Experiments 2 and 3 supported a "depth-of-processing" account for the effects of initial search and change detection on incidental memory for scenes. Of most interest, change detection was faster during the test phase than during the study phase, even when the observer had no explicit memory of having found that change previously. This result was replicated in two of our three change detection experiments. We conclude that scenes can be encoded incidentally as well as explicitly and that changes in those scenes can leave measurable traces even if they are not explicitly recalled.

Effort during visual search and counting: Insights from pupillometry

We investigated the processing effort during visual search and counting tasks using a pupil dilation measure. Search difficulty was manipulated by varying the number of distractors as well as the heterogeneity of the distractors. More difficult visual search resulted in more pupil dilation than did less difficult search. These results confirm a link between effort and increased pupil dilation.

The pupil dilated more during the counting task than during target-absent search, even though the displays were identical, and the two tasks were matched for reaction time. The moment-to-moment dilation pattern during search suggests little effort in the early stages, but increasingly more effort towards response, whereas the counting task involved an increased initial effort, which was sustained throughout the trial.

These patterns can be interpreted in terms of the differential memory load for item locations in each task. In an additional experiment, increasing the spatial memory requirements of the search evoked a corresponding increase in pupil dilation. These results support the view that search tasks involve some, but limited, memory for item locations, and the effort associated with this memory load increases during the trials. In contrast, counting involves a heavy locational memory component from the start.

What do eye movements reveal about the role of memory in visual search?

Horowitz and Wolfe (1998, 2003) have challenged the view that serial visual search involves memory processes that keep track of already inspected locations. The present study used a search paradigm similar to Horowitz and Wolfe's (1998), comparing a standard static search condition with a dynamic condition in which display elements changed locations randomly every 111 ms. In addition to measuring search reaction times, observers’ eye movements were recorded. For target-present trials, the search rates were near-identical in the two search conditions, replicating Horowitz and Wolfe's findings. However, the number of fixations and saccade amplitude were larger in the static than in the dynamic condition, whereas fixation duration and the latency of the first saccade were longer in the dynamic condition. These results indicate that an active, memory-guided search strategy was adopted in the static condition, and a passive “sit-and-wait” strategy in the dynamic condition.

Effects of Age on Searching for and Enumerating Targets that Cannot be Detected Efficiently

We investigated the effects of old age on search, subitizing, and counting of difficult-to-find targets. In Experiment 1, young and older adults enumerated targets (Os) with and without distractors (Qs). Without distractors, the usual subitization-counting function occurred in both groups, with the same subitization span of 3.3 items. Subitization disappeared with distractors; older adults were slowed more overall by their presence but enumeration rates were not slowed by ageing either with or without distractors. In contrast, search rates for a single target (O among Qs) were twice as slow for older as for young adults. Experiment 2 tested and ruled out one account of age-equivalent serial enumeration based on the need to subvocalize numbers as items are enumerated. Alternative explanations based on the specific task differences between detecting and enumerating stimuli are discussed.

Simple eye-movement feedback during visual search is not helpful

Searching for targets in the visual world, or visual search, is something we all do every day. We frequently make 'false-negative' errors, wherein we erroneously conclude a target was absent when one was, in fact, present. These sorts of errors can have tremendous costs, as when signs of cancers are missed in diagnostic radiology. Prior research has characterized the cause of many of these errors as being due to failure to completely search the area where targets may be present; indeed, roughly one-third of chest nodules missed in lung cancer screening are never fixated (Drew, Võ, Olwal, Jacobson, Seltzer and Wolfe, Journal of Vision 13:3, 2013). This suggests that observers do not have a good representation of what areas have and have not been searched prior to declaring an area target free. Therefore, in six experiments, we sought to examine the utility of reducing the uncertainty with respect to what areas had been examined via online eye-tracking feedback. We hypothesized that providing information about what areas had or had not been examined would lead to lower rates of false negatives or more efficient search, namely faster response times with no cost on target detection accuracy. Neither of these predictions held true. Over six experiments, online eye-tracking feedback did not yield any reliable performance benefits.

Working memory load predicts visual search efficiency: Evidence from a novel pupillary response paradigm

An observer's pupil dilates and constricts in response to variables such as ambient and focal luminance, cognitive effort, the emotional stimulus content, and working memory load. The pupil's memory load response is of particular interest, as it might be used for estimating observers' memory load while they are performing a complex task, without adding an interruptive and confounding memory test to the protocol. One important task in which working memory's involvement is still being debated is visual search, and indeed a previous experiment by Porter, Troscianko, and Gilchrist (Quarterly Journal of Experimental Psychology, 60, 211-229, 2007) analyzed observers' pupil sizes during search to study this issue. These authors found that pupil size increased over the course of the search, and they attributed this finding to accumulating working memory load. However, since the pupil response is slow and does not depend on memory load alone, this conclusion is rather speculative. In the present study, we estimated working memory load in visual search during the presentation of intermittent fixation screens, thought to induce a low, stable level of arousal and cognitive effort. Using standard visual search and control tasks, we showed that this paradigm reduces the influence of non-memory-related factors on pupil size. Furthermore, we found an early increase in working memory load to be associated with more efficient search, indicating a significant role of working memory in the search process.