Visual search

Cue relevance drives early quitting in visual search

Irrelevant salient distractors can trigger early quitting in visual search, causing observers to miss targets they might otherwise find. Here, we asked whether task-relevant salient cues can produce a similar early quitting effect on the subset of trials where those cues fail to highlight the target. We presented participants with a difficult visual search task and used two cueing conditions. In the high-predictive condition, a salient cue in the form of a red circle highlighted the target most of the time a target was present. In the low-predictive condition, the cue was far less accurate and did not reliably predict the target (i.e., the cue was often a false positive). These were contrasted against a control condition in which no cues were presented. In the high-predictive condition, we found clear evidence of early quitting on trials where the cue was a false positive, as evidenced by both increased miss errors and shorter response times on target absent trials. No such effects were observed with low-predictive cues. Together, these results suggest that salient cues which are false positives can trigger early quitting, though perhaps only when the cues have a high-predictive value. These results have implications for real-world searches, such as medical image screening, where salient cues (referred to as computer-aided detection or CAD) may be used to highlight potentially relevant areas of images but are sometimes inaccurate.

Towards optimized methodological parameters for maximizing the behavioral effects of transcranial direct current stimulation

Introduction

Transcranial direct current stimulation (tDCS) administers low-intensity direct current electrical stimulation to brain regions via electrodes arranged on the surface of the scalp. The core promise of tDCS is its ability to modulate brain activity and affect performance on diverse cognitive functions (affording causal inferences regarding regional brain activity and behavior), but the optimal methodological parameters for maximizing behavioral effects remain to be elucidated. Here we sought to examine the effects of 10 stimulation and experimental design factors across a series of five cognitive domains: motor performance, visual search, working memory, vigilance, and response inhibition. The objective was to identify a set of optimal parameter settings that consistently and reliably maximized the behavioral effects of tDCS within each cognitive domain.

Methods

We surveyed tDCS effects on these various cognitive functions in healthy young adults, ultimately resulting in 721 effects across 106 published reports. Hierarchical Bayesian meta-regression models were fit to characterize how (and to what extent) these design parameters differentially predict the likelihood of positive/negative behavioral outcomes.

Results

Consistent with many previous meta-analyses of tDCS effects, extensive variability was observed across tasks and measured outcomes. Consequently, most design parameters did not confer consistent advantages or disadvantages to behavioral effects-a domain-general model suggested an advantage to using within-subjects designs (versus between-subjects) and the tendency for cathodal stimulation (relative to anodal stimulation) to produce reduced behavioral effects, but these associations were scarcely-evident in domain-specific models.

Discussion

These findings highlight the urgent need for tDCS studies to more systematically probe the effects of these parameters on behavior to fulfill the promise of identifying causal links between brain function and cognition.

Using a flashlight-contingent window paradigm to investigate visual search and object memory in virtual reality and on computer screens

A popular technique to modulate visual input during search is to use gaze-contingent windows. However, these are often rather discomforting, providing the impression of visual impairment. To counteract this, we asked participants in this study to search through illuminated as well as dark three-dimensional scenes using a more naturalistic flashlight with which they could illuminate the rooms. In a surprise incidental memory task, we tested the identities and locations of objects encountered during search. Importantly, we tested this study design in both immersive virtual reality (VR; Experiment 1) and on a desktop-computer screen (Experiment 2). As hypothesized, searching with a flashlight increased search difficulty and memory usage during search. We found a memory benefit for identities of distractors in the flashlight condition in VR but not in the computer screen experiment. Surprisingly, location memory was comparable across search conditions despite the enormous difference in visual input. Subtle differences across experiments only appeared in VR after accounting for previous recognition performance, hinting at a benefit of flashlight search in VR. Our findings highlight that removing visual information does not necessarily impair location memory, and that screen experiments using virtual environments can elicit the same major effects as VR setups.

Effects of object working memory load on visual search in basketball players: an eye movement study

BACKGROUND

Working memory may affect the athletes' visual search ability.

OBJECTIVE

This study aimed to examine the differences in the performance of visual search tasks among basketball players of varying sport levels, considering the influence of different object working memory loads.

METHOD

This study recruited forty-two participants who were divided into three groups based on the classification of elite athletes: competitive elite, semi-elite, and novice.

RESULTS

Objective working memory load significantly impacts the accuracy of visual search, reaction time, and gaze fixation in basketball players. In the visual search task of the basketball sports scene, the inclusion of object working memory load led to a significant decrease in the accuracy of visual search, a significant increase in reaction time, a significant increase in the number of fixation points, and a more complex gaze trajectory. In a visual search task with object working memory load, the difference in reaction time between basketball players of different sport levels was observed during the search initiation time and scanning time, with higher sport levels associated with shorter reaction times. The effect of object working memory load on the eye movement phase of visual search varied among basketball players of different sport levels. For the novice group, the effect was on the reaction time during the verification phase, while for the semi-elite and competitive elite groups, the effect was on the reaction time during the scanning phase.

CONCLUSION

The effect of object working memory load on visual search varied among basketball players of different sport levels.

The Efficiency of Question-Asking Strategies in a Real-World Visual Search Task

In recent years, a multitude of datasets of human-human conversations has been released for the main purpose of training conversational agents based on data-hungry artificial neural networks. In this paper, we argue that datasets of this sort represent a useful and underexplored source to validate, complement, and enhance cognitive studies on human behavior and language use. We present a method that leverages the recent development of powerful computational models to obtain the fine-grained annotation required to apply metrics and techniques from Cognitive Science to large datasets. Previous work in Cognitive Science has investigated the question-asking strategies of human participants by employing different variants of the so-called 20-question-game setting and proposing several evaluation methods. In our work, we focus on GuessWhat, a task proposed within the Computer Vision and Natural Language Processing communities that is similar in structure to the 20-question-game setting. Crucially, the GuessWhat dataset contains tens of thousands of dialogues based on real-world images, making it a suitable setting to investigate the question-asking strategies of human players on a large scale and in a natural setting. Our results demonstrate the effectiveness of computational tools to automatically code how the hypothesis space changes throughout the dialogue in complex visual scenes. On the one hand, we confirm findings from previous work on smaller and more controlled settings. On the other hand, our analyses allow us to highlight the presence of "uninformative" questions (in terms of Expected Information Gain) at specific rounds of the dialogue. We hypothesize that these questions fulfill pragmatic constraints that are exploited by human players to solve visual tasks in complex scenes successfully. Our work illustrates a method that brings together efforts and findings from different disciplines to gain a better understanding of human question-asking strategies on large-scale datasets, while at the same time posing new questions about the development of conversational systems.

A color-related bias in offside judgments in professional soccer: A matter of figure-background contrast?

We investigated the impact of outfit colors on the frequency of offside judgments in soccer. In a recent laboratory study, observers made more offside judgments against forwards wearing the outfit of Schalke 04 (blue shirts, white shorts) than against forwards wearing the outfit of Borussia Dortmund (yellow shirts, black shorts), when figure-background luminance contrast was higher for the former team. Here, we investigated whether a similar effect is present in real matches of the German Bundesliga. Study 1 revealed a higher offside score for Schalke 04 than for Borussia Dortmund in matches between these clubs. Studies 2-4 showed higher offside scores for teams wearing a blue/white outfit, and lower offside scores for teams wearing a yellow/black outfit, in their matches against all other Bundesliga teams. Together, results suggest that more offside judgments are made against teams of higher salience, possibly induced by differences in figure-background contrast. Notably, this color-related bias occurred in our study even though a Video-Assistant Referee (VAR) supervised the (offside) decisions of the Assistant Referees.

Match me if you can: Evidence for a domain-general visual comparison ability

Visual comparison-comparing visual stimuli (e.g., fingerprints) side by side and determining whether they originate from the same or different source (i.e., "match")-is a complex discrimination task involving many cognitive and perceptual processes. Despite the real-world consequences of this task, which is often conducted by forensic scientists, little is understood about the psychological processes underpinning this ability. There are substantial individual differences in visual comparison accuracy amongst both professionals and novices. The source of this variation is unknown, but may reflect a domain-general and naturally varying perceptual ability. Here, we investigate this by comparing individual differences (N = 248 across two studies) in four visual comparison domains: faces, fingerprints, firearms, and artificial prints. Accuracy on all comparison tasks was significantly correlated and accounted for a substantial portion of variance (e.g., 42% in Exp. 1) in performance across all tasks. Importantly, this relationship cannot be attributed to participants' intrinsic motivation or skill in other visual-perceptual tasks (visual search and visual statistical learning). This paper provides novel evidence of a reliable, domain-general visual comparison ability.

Individual differences in crowding predict visual search performance

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

Great expectations: minor differences in initial instructions have a major impact on visual search in the absence of feedback

Professions such as radiology and aviation security screening that rely on visual search-the act of looking for targets among distractors-often cannot provide operators immediate feedback, which can create situations where performance may be largely driven by the searchers' own expectations. For example, if searchers do not expect relatively hard-to-spot targets to be present in a given search, they may find easy-to-spot targets but systematically quit searching before finding more difficult ones. Without feedback, searchers can create self-fulfilling prophecies where they incorrectly reinforce initial biases (e.g., first assuming and then, perhaps wrongly, concluding hard-to-spot targets are rare). In the current study, two groups of searchers completed an identical visual search task but with just a single difference in their initial task instructions before the experiment started; those in the "high-expectation" condition were told that each trial could have one or two targets present (i.e., correctly implying no target-absent trials) and those in the "low-expectation" condition were told that each trial would have up to two targets (i.e., incorrectly implying there could be target-absent trials). Compared to the high-expectation group, the low-expectation group had a lower hit rate, lower false alarm rate and quit trials more quickly, consistent with a lower quitting threshold (i.e., performing less exhaustive searches) and a potentially higher target-present decision criterion. The expectation effect was present from the start and remained across the experiment-despite exposure to the same true distribution of targets, the groups' performances remained divergent, primarily driven by the different subjective experiences caused by each groups' self-fulfilling prophecies. The effects were limited to the single-targets trials, which provides insights into the mechanisms affected by the initial expectations set by the instructions. In sum, initial expectations can have dramatic influences-searchers who do not expect to find a target, are less likely to find a target as they are more likely to quit searching earlier.

Saliency-Aware Subtle Augmentation Improves Human Visual Search Performance in VR

Visual search becomes challenging when the time to find the target is limited. Here we focus on how performance in visual search can be improved via a subtle saliency-aware modulation of the scene. Specifically, we investigate whether blurring salient regions of the scene can improve participant's ability to find the target faster when the target is located in non-salient areas. A set of real-world omnidirectional images were displayed in virtual reality with a search target overlaid on the visual scene at a pseudorandom location. Participants performed a visual search task in three conditions defined by blur strength, where the task was to find the target as fast as possible. The mean search time, and the proportion of trials where participants failed to find the target, were compared across different conditions. Furthermore, the number and duration of fixations were evaluated. A significant effect of blur on behavioral and fixation metrics was found using linear mixed models. This study shows that it is possible to improve the performance by a saliency-aware subtle scene modulation in a challenging realistic visual search scenario. The current work provides an insight into potential visual augmentation designs aiming to improve user's performance in everyday visual search tasks.

Effects of repeated testing in a pen-and-paper test of selective attention (FAIR-2)

The FAIR-2 (‘Frankfurter Aufmerksamkeitsinventar’) is a pen-and-paper test of visual attention in which participants have to search for targets among distractors. For similar pen-and-paper tests of attention (e.g., d2), the repetition of the test causes large improvements in performance that threaten both its (retest) reliability and validity. We investigated the size and possible sources of practice effects in the FAIR-2 in three experiments. In Experiments 1 and 2, participants were tested twice using the original FAIR-2. We compared how performance changed after 2 weeks (Experiment 1) or 3 months (Experiment 2), when the test was repeated (complete repetition), or when targets and distractors changed their roles (test reversal). For Experiment 3, we used self-constructed versions of the FAIR that allowed for a third neutral condition (complete alternation) without any stimulus overlap between the two tests. The complete repetition condition produced strong performance gains (25–35%) that persisted for 3 months. For the complete-alternation condition, we observed small to moderate improvements, suggesting that stimulus-independent learning had occurred in session 1. Finally, performance did not differ between test reversal and complete alternation, therefore, suggesting that improvements in target processing had caused the large improvements in the complete-repetition condition.

Cultural differences in food detection

The ability to detect food plays an indispensable role in our survival and wellbeing. Previous psychological studies have revealed that food is detected more rapidly than non-food items. However, whether the detection of food could be modulated by cultural factors remains unknown. We investigated this issue in the present study using a visual search paradigm with Polish and Japanese participants. Photographs of international fast food, domestic Japanese food, or kitchen tools were presented alongside images of non-food distractors (cars). Participants were asked to judge whether the stimuli were all identical or not. The reaction time data showed that participants from both cultures detected food more rapidly than kitchen tools. Japanese participants detected fast food more rapidly than Japanese food, whereas Polish participants did not display such differences between food types. These results suggest that rapid detection of food is universal, but is modulated by cultural experiences.

Search and Match Task: Development of a Taskified Match-3 Puzzle Game to Assess and Practice Visual Search

Background

Visual search declines with aging, dementia, and brain injury and is linked to limitations in everyday activities. Recent studies suggest that visual search can be improved with practice using computerized visual search tasks and puzzle video games. For practical use, it is important that visual search ability can be assessed and practiced in a controlled and adaptive way. However, commercial puzzle video games make it hard to control task difficulty, and there are little means to collect performance data.

Objective

The aim of this study was to develop and initially validate the search and match task (SMT) that combines an enjoyable tile-matching match-3 puzzle video game with features of the visual search paradigm (taskified game). The SMT was designed as a single-target visual search task that allows control over task difficulty variables and collection of performance data.

Methods

The SMT is played on a grid-based (width × height) puzzle board, filled with different types of colored polygons. A wide range of difficulty levels was generated by combinations of 3 task variables over a range from 4 to 8 including height and width of the puzzle board (set size) and the numbers of tile types (distractor heterogeneity). For each difficulty level, large numbers of playable trials were pregenerated using Python. Each trial consists of 4 consecutive puzzle boards, where the goal of the task is to find a target tile configuration (search) on the puzzle board and swap 2 adjacent tiles to create a line of 3 identical tiles (match). For each puzzle board, there is exactly 1 possible match (single target search). In a user study with 28 young adults (aged 18 to 31 years), 13 older (aged 64 to 79 years) and 11 oldest (aged 86 to 98 years) adults played the long (young and older adults) or short version (oldest adults) of the difficulty levels of the SMT. Participants rated their perception and the usability of the task and completed neuropsychological tests that measure cognitive domains engaged by the puzzle game.

Results

Results from the user study indicate that the target search time is associated with set size, distractor heterogeneity, and age. Results further indicate that search performance is associated with general cognitive ability, selective and divided attention, visual search, and visuospatial and pattern recognition ability.

Conclusions

Overall, this study shows that an everyday puzzle game–based task can be experimentally controlled, is enjoyable and user-friendly, and permits data collection to assess visual search and cognitive abilities. Further research is needed to evaluate the potential of the SMT game to assess and practice visual search ability in an enjoyable and adaptive way. A PsychoPy version of the SMT is freely available for researchers.

Target-Specific Learning Contributes to Practice Effects in Paper-and-Pencil Tests of Attention

Abstract. Previous studies revealed that performance in paper-and-pencil tests of attention, such as the d2-R test, improves with practice, though the sources of these practice effects are yet unknown. Practice effects in psychometric tests are a serious problem because they impede the evaluation of test performance and constrain the utility of these tests. This study addresses the role of target-specific learning for practice in a d2-like paper-and-pencil test of attention. Two variants of a d2-like test, which exclusively differed in the set of target stimuli, were constructed. Participants were tested on two days separated by one week. Participants in the control condition searched for the same targets (among the same distractors) in each session, whereas participants in the test group searched for different targets (among the same distractors) in each session. Results showed practice benefits in both groups, but benefits were larger in the control group than in the test group. The results suggest that practice improves the processing of target features in paper-and-pencil tests of attention. Hence, using two versions with different sets of targets may effectively reduce practice effects in d2-like tests of attention. Further implications of the findings are discussed.

Traditional Visual Search vs. X-Ray Image Inspection in Students and Professionals: Are the Same Visual-Cognitive Abilities Needed?

The act of looking for targets amongst an array of distractors is a cognitive task that has been studied extensively over many decades and has many real-world applications. Research shows that specific visual-cognitive abilities are needed to efficiently and effectively locate a target among distractors. It is, however, not always clear whether the results from traditional, simplified visual search tasks conducted by students will extrapolate to an applied inspection tasks in which professionals search for targets that are more complex, ambiguous, and less salient. More concretely, there are several potential challenges when interpreting traditional visual search results in terms of their implications for the X-ray image inspection task. In this study, we tested whether a theoretical intelligence model with known facets of visual-cognitive abilities (visual processing Gv, short-term memory Gsm, and processing speed Gs) can predict performance in both a traditional visual search task and an X-ray image inspection task in both students and professionals. Results showed that visual search ability as measured with a traditional visual search task is not comparable to an applied X-ray image inspection task. Even though both tasks require aspects of the same visual-cognitive abilities, the overlap between the tasks was small. We concluded that different aspects of visual-cognitive abilities predict performance on the measured tasks. Furthermore, although our tested populations were comparable in terms of performance predictors based on visual-cognitive abilities, professionals outperformed students on an applied X-ray image inspection task. Hence, inferences from our research questions have to be treated with caution, because the comparability of the two populations depends on the task.

When is it time to move to the next map? Optimal foraging in guided visual search

Suppose that you are looking for visual targets in a set of images, each containing an unknown number of targets. How do you perform that search, and how do you decide when to move from the current image to the next? Optimal foraging theory predicts that foragers should leave the current image when the expected value from staying falls below the expected value from leaving. Here, we describe how to apply these models to more complex tasks, like search for objects in natural scenes where people have prior beliefs about the number and locations of targets in each image, and search is guided by target features and scene context. We model these factors in a guided search task and predict the optimal time to quit search. The data come from a satellite image search task. Participants searched for small gas stations in large satellite images. We model quitting times with a Bayesian model that incorporates prior beliefs about the number of targets in each map, average search efficiency (guidance), and actual search history in the image. Clicks deploying local magnification were used as surrogates for deployments of attention and, thus, for time. Leaving times (measured in mouse clicks) were well-predicted by the model. People terminated search when their expected rate of target collection fell to the average rate for the task. Apparently, people follow a rate-optimizing strategy in this task and use both their prior knowledge and search history in the image to decide when to quit searching.

HOW DO RADIOLOGISTS USE THE HUMAN SEARCH ENGINE?

Radiologists perform many 'visual search tasks' in which they look for one or more instances of one or more types of target item in a medical image (e.g. cancer screening). To understand and improve how radiologists do such tasks, it must be understood how the human 'search engine' works. This article briefly reviews some of the relevant work into this aspect of medical image perception. Questions include how attention and the eyes are guided in radiologic search? How is global (image-wide) information used in search? How might properties of human vision and human cognition lead to errors in radiologic search?

Using multidimensional scaling to quantify similarity in visual search and beyond

Visual search is one of the most widely studied topics in vision science, both as an independent topic of interest, and as a tool for studying attention and visual cognition. A wide literature exists that seeks to understand how people find things under varying conditions of difficulty and complexity, and in situations ranging from the mundane (e.g., looking for one's keys) to those with significant societal importance (e.g., baggage or medical screening). A primary determinant of the ease and probability of success during search are the similarity relationships that exist in the search environment, such as the similarity between the background and the target, or the likeness of the non-targets to one another. A sense of similarity is often intuitive, but it is seldom quantified directly. This presents a problem in that similarity relationships are imprecisely specified, limiting the capacity of the researcher to examine adequately their influence. In this article, we present a novel approach to overcoming this problem that combines multi-dimensional scaling (MDS) analyses with behavioral and eye-tracking measurements. We propose a method whereby MDS can be repurposed to successfully quantify the similarity of experimental stimuli, thereby opening up theoretical questions in visual search and attention that cannot currently be addressed. These quantifications, in conjunction with behavioral and oculomotor measures, allow for critical observations about how similarity affects performance, information selection, and information processing. We provide a demonstration and tutorial of the approach, identify documented examples of its use, discuss how complementary computer vision methods could also be adopted, and close with a discussion of potential avenues for future application of this technique.

The impending demise of the item in visual search

The way the cognitive system scans the visual environment for relevant information - visual search in short - has been a long-standing central topic in vision science. From its inception as a research topic, and despite a number of promising alternative perspectives, the study of visual search has been governed by the assumption that a search proceeds on the basis of individual items (whether processed in parallel or not). This has led to the additional assumptions that shallow search slopes (at most a few tens of milliseconds per item for target-present trials) are most informative about the underlying process, and that eye movements are an epiphenomenon that can be safely ignored. We argue that the evidence now overwhelmingly favours an approach that takes fixations, not individual items, as its central unit. Within fixations, items are processed in parallel, and the functional field of view determines how many fixations are needed. In this type of theoretical framework, there is a direct connection between target discrimination difficulty, fixations, and reaction time (RT) measures. It therefore promises a more fundamental understanding of visual search by offering a unified account of both eye movement and manual response behaviour across the entire range of observed search efficiency, and provides new directions for research. A high-level conceptual simulation with just one free and four fixed parameters shows the viability of this approach.