Memory load affects visual search processes without influencing search efficiency

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Look twice: A generalist computational model predicts return fixations across tasks and species

Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.

The Effect of Cognitive Load on the Retrieval of Long-Term Memory: An fMRI Study

One of the less well-understood aspects of memory function is the mechanism by which the brain responds to an increasing load of memory, either during encoding or retrieval. Identifying the brain structures which manage this increasing cognitive demand would enhance our knowledge of human memory. Despite numerous studies about the effect of cognitive loads on working memory processes, whether these can be applied to long-term memory processes is unclear. We asked 32 healthy young volunteers to memorize all possible details of 24 images over a 12-day period ending 2 days before the fMRI scan. The images were of 12 categories relevant to daily events, with each category including a high and a low load image. Behavioral assessments on a separate group of participants (#22) provided the average loads of the images. The participants had to retrieve these previously memorized images during the fMRI scan in 15 s, with their eyes closed. We observed seven brain structures showing the highest activation with increasing load of the retrieved images, viz. parahippocampus, cerebellum, superior lateral occipital, fusiform and lingual gyri, precuneus, and posterior cingulate gyrus. Some structures showed reduced activation when retrieving higher load images, such as the anterior cingulate, insula, and supramarginal and postcentral gyri. The findings of this study revealed that the mechanism by which a difficult-to-retrieve memory is handled is mainly by elevating the activation of the responsible brain areas and not by getting other brain regions involved, which is a help to better understand the LTM retrieval process in the human brain.

Distracting Objects Induce Early Quitting in Visual Search

Task-irrelevant objects can sometimes capture attention and increase the time it takes an observer to find a target. However, less is known about how these distractors impact visual search strategies. Here, I found that salient distractors reduced rather than increased response times on target-absent trials (Experiment 1; N = 200). Combined with higher error rates on target-present trials, these results indicate that distractors can induce observers to quit search earlier than they otherwise would. These effects were replicated when target prevalence was low (Experiment 2; N = 200) and with different stimuli that elicited shallower search slopes (Experiment 3; N = 75). These results demonstrate that salient distractors can produce at least two consequences in visual search: They can capture attention, and they can cause observers to quit searching early. This novel finding has implications both for understanding visual attention and for examining distraction in real-world domains where targets are often absent, such as medical image screening.

Refixation control in free viewing: a specialized mechanism divulged by eye-movement-related brain activity

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eye-movement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. NEW & NOTEWORTHY A substantial proportion of eye fixations in human natural viewing behavior are revisits of recently visited locations, i.e., refixations. Our recently developed methods enabled us to study refixations in a free viewing visual search task, using combined eye movement and EEG recording. We identified in the EEG a distinctive refixation-related signal, signifying a control mechanism specific to refixations as opposed to ordinary eye fixations.

The effects of array structure and secondary cognitive task demand on processes of visual search

Many aspects of our everyday behaviour require that we search for objects. However, in real situations search is often conducted while internal and external factors compete for our attention resources. Cognitive distraction interferes with our ability to search for targets, increasing search times. Here we consider whether effects of cognitive distraction interfere differentially with three distinct phases of search: initiating search, overtly scanning through items in the display, and verifying that the object is indeed the target of search once it has been fixated. Furthermore, we consider whether strategic components of visual search that emerge when searching items organized into structured arrays are susceptible to cognitive distraction or not. We used Gilchrist & Harvey's (2006) structured and unstructured visual search paradigm with the addition of Savage, Potter, and Tatler's (2013) secondary puzzle task. Cognitive load influenced two phases of search: 1) scanning times and 2) verification times. Under high load, fixation durations were longer and re-fixations of distracters were more common. In terms of scanning strategy, we replicated Gilchrist and Harvey's (2006) findings of more systematic search for structured arrays than unstructured ones. We also found an effect of cognitive load on this aspect of search but only in structured arrays. Our findings suggest that our eyes, by default, produce an autonomous scanning pattern that is modulated but not completely eliminated by secondary cognitive load.

Negative cues lead to more inefficient search than positive cues even at later stages of visual search

Observers can focus their attention on task-relevant items in visual search when they have prior knowledge about the target's properties (i.e., positive cues). However, little is known about how negative cues, which specify the features of task-irrelevant items, can be used to guide attention away from distractors and how their effects differ from those of positive cues. It has been proposed that when a distractor color is cued, people would first select the to-be-ignored items early in search and then inhibit them later. The present study investigated how the effects of positive and negative cues differ throughout the visual search process. The results showed that positive cues sped up the early stage of visual search and that negative cues led to initial selection for inhibition. We further found that visual search with negative cues was more inefficient than that with positive cues even at later stages, suggesting that sustained inhibition is needed throughout the visual search process. Taken together, the results indicate that positive and negative cues have different functions: prior knowledge about target features can weight task-relevant information at early stages of visual search, and negative cues are used more inefficiently even at later stages of visual search.

Balancing energetic and cognitive resources: memory use during search depends on the orienting effector

Search outside the laboratory involves tradeoffs among a variety of internal and external exploratory processes. Here we examine the conditions under which item specific memory from prior exposures to a search array is used to guide attention during search. We extend the hypothesis that memory use increases as perceptual search becomes more difficult by turning to an ecologically important type of search difficulty - energetic cost. Using optical motion tracking, we introduce a novel head-contingent display system, which enables the direct comparison of search using head movements and search using eye movements. Consistent with the increased energetic cost of turning the head to orient attention, we discover greater use of memory in head-contingent versus eye-contingent search, as reflected in both timing and orienting metrics. Our results extend theories of memory use in search to encompass embodied factors, and highlight the importance of accounting for the costs and constraints of the specific motor groups used in a given task when evaluating cognitive effects.

Words, shape, visual search and visual working memory in 3-year-old children

Do words cue children's visual attention, and if so, what are the relevant mechanisms? Across four experiments, 3-year-old children (N = 163) were tested in visual search tasks in which targets were cued with only a visual preview versus a visual preview and a spoken name. The experiments were designed to determine whether labels facilitated search times and to examine one route through which labels could have their effect: By influencing the visual working memory representation of the target. The targets and distractors were pictures of instances of basic-level known categories and the labels were the common name for the target category. We predicted that the label would enhance the visual working memory representation of the target object, guiding attention to objects that better matched the target representation. Experiments 1 and 2 used conjunctive search tasks, and Experiment 3 varied shape discriminability between targets and distractors. Experiment 4 compared the effects of labels to repeated presentations of the visual target, which should also influence the working memory representation of the target. The overall pattern fits contemporary theories of how the contents of visual working memory interact with visual search and attention, and shows that even in very young children heard words affect the processing of visual information.

Attentional control via parallel target-templates in dual-target search

Simultaneous search for two targets has been shown to be slower and less accurate than independent searches for the same two targets. Recent research suggests this 'dual-target cost' may be attributable to a limit in the number of target-templates than can guide search at any one time. The current study investigated this possibility by comparing behavioural responses during single- and dual-target searches for targets defined by their orientation. The results revealed an increase in reaction times for dual- compared to single-target searches that was largely independent of the number of items in the display. Response accuracy also decreased on dual- compared to single-target searches: dual-target accuracy was higher than predicted by a model restricting search guidance to a single target-template and lower than predicted by a model simulating two independent single-target searches. These results are consistent with a parallel model of dual-target search in which attentional control is exerted by more than one target-template at a time. The requirement to maintain two target-templates simultaneously, however, appears to impose a reduction in the specificity of the memory representation that guides search for each target.

The Role of Search Speed in the Contextual Cueing of Children's Attention

The contextual cueing effect is a robust phenomenon in which repeated exposure to the same arrangement of random elements guides attention to relevant information by constraining search. The effect is measured using an object search task in which a target (e.g., the letter T) is located within repeated or nonrepeated visual contexts (e.g., configurations of the letter L). Decreasing response times for the repeated configurations indicates that contextual information has facilitated search. Although the effect is robust among adult participants, recent attempts to document the effect in children have yielded mixed results. We examined the effect of search speed on contextual cueing with school-aged children, comparing three types of stimuli that promote different search times in order to observe how speed modulates this effect. Reliable effects of search time were found, suggesting that visual search speed uniquely constrains the role of attention toward contextually cued information.

In Manually-Assisted Search, Perception Supervises Rather Than Directs Action

During manually-assisted search, where participants must actively manipulate search items, it has been reported that participants will often select and move the target of search itself without recognizing it (Solman et al., 2012a). In two experiments we explore the hypothesis that this error results from a naturally-arising strategy that decouples perception and action during search, enabling motor interactions with items to outpace the speed of perceptual analysis. In Experiment 1, we report that the error is prevalent for both mouse and touch-screen interaction modes, and is uninfluenced by speeding or slowing instructions – ruling out these task-specific details as causes of the error. In Experiment 2 we manipulate motor speed, and show that reducing the speed of individual movements during search leads to a reduction in error rates. These findings support the conclusion that the error results from incoordination between motor and perceptual processes, with motor processes outpacing perceptual abilities.