Visual similarity is stronger than semantic similarity in guiding visual search for numbers

Numerical values modulate size perception

The link between various codes of magnitude and their interactions has been studied extensively for many years. In the current study, we examined how the physical and numerical magnitudes of digits are mapped into a combined mental representation. In two psychophysical experiments, participants reported the physically larger digit among two digits. In the identical condition, participants compared digits of an identical value (e.g., "2" and "2"); in the different condition, participants compared digits of distinct numerical values (i.e., "2" and "5"). As anticipated, participants overestimated the physical size of a numerically larger digit and underestimated the physical size of a numerically smaller digit. Our results extend the shared-representation account of physical and numerical magnitudes.

Visual Search Asymmetry Due to the Relative Magnitude Represented by Number Symbols

In visual search tasks, physically large target stimuli are more easily identified among small distractors than are small targets among large distractors. The present study extends this finding by presenting preliminary evidence of a new search asymmetry: stimuli that symbolically represent larger magnitude are identified more easily among featurally equivalent distractors that represent smaller magnitude. Participants performed a visual search task using line-segment digits representing the numbers 2 and 5, and the numbers 6 and 9, as well as comparable non-numeric control stimuli. In three experiments, we found that search times are faster when the target is a digit that represents a larger magnitude than the distractor, although this pattern was not evident in one additional experiment. The results provide suggestive evidence that the magnitude of a number symbol can affect perceptual comparisons between number symbols, and that the semantic meaning of a target stimulus can systematically affect visual search.

The detail is in the difficulty: Challenging search facilitates rich incidental object encoding

When searching for objects in the environment, observers necessarily encounter other, nontarget, objects. Despite their irrelevance for search, observers often incidentally encode the details of these objects, an effect that is exaggerated as the search task becomes more challenging. Although it is well established that searchers create incidental memories for targets, less is known about the fidelity with which nontargets are remembered. Do observers store richly detailed representations of nontargets, or are these memories characterized by gist-level detail, containing only the information necessary to reject the item as a nontarget? We addressed this question across two experiments in which observers completed multiple-target (one to four potential targets) searches, followed by surprise alternative forced-choice (AFC) recognition tests for all encountered objects. To assess the detail of incidentally stored memories, we used similarity rankings derived from multidimensional scaling to manipulate the perceptual similarity across objects in 4-AFC (Experiment 1a) and 16-AFC (Experiments 1b and 2) tests. Replicating prior work, observers recognized more nontarget objects encountered during challenging, relative to easier, searches. More importantly, AFC results revealed that observers stored more than gist-level detail: When search objects were not recognized, observers systematically chose lures with higher perceptual similarity, reflecting partial encoding of the search object's perceptual features. Further, similarity effects increased with search difficulty, revealing that incidental memories for visual search objects are sharpened when the search task requires greater attentional processing.

Avoiding potential pitfalls in visual search and eye-movement experiments: A tutorial review

Examining eye-movement behavior during visual search is an increasingly popular approach for gaining insights into the moment-to-moment processing that takes place when we look for targets in our environment. In this tutorial review, we describe a set of pitfalls and considerations that are important for researchers – both experienced and new to the field – when engaging in eye-movement and visual search experiments. We walk the reader through the research cycle of a visual search and eye-movement experiment, from choosing the right predictions, through to data collection, reporting of methodology, analytic approaches, the different dependent variables to analyze, and drawing conclusions from patterns of results. Overall, our hope is that this review can serve as a guide, a talking point, a reflection on the practices and potential problems with the current literature on this topic, and ultimately a first step towards standardizing research practices in the field.

Improved X-ray baggage screening sensitivity with 'targetless' search training

When searching for a known target, mental representations of target features, or templates, guide attention towards matching objects and facilitate recognition. When only distractor features are known, distractor templates allow irrelevant objects to be recognised and attention to be shifted away. This is particularly true in X-ray baggage search, a challenging real-world visual search task with implications for public safety, where targets may be unknown, difficult to predict and concealed by an adversary, but distractors are typically benign and easier to identify. In the present study, we draw on basic principles of distractor suppression and rejection to investigate a counterintuitive ‘targetless’ approach to training baggage search. In a simulated X-ray baggage search task, we observed significant benefits to target detection sensitivity (d′) for targetless relative to target-based training, but no effects of performance-contingent rewards or the inclusion of superordinate semantic categories during training. The benefits of targetless search training were most apparent for stimuli involving less spatial overlap (occlusion), which likely represents the difficulty and greater individual variation involved in searching more visually complex images. Together, these results demonstrate the effectiveness of a counterintuitive targetless approach to training target detection in X-ray baggage search, based on basic principles of distractor suppression and rejection, with potential for use as a real-world training tool.

Nonsymbolic and symbolic representations of null numerosity

Previous research has shown that null numerosity can be processed as a numerical entity that is represented together with non-null numerosities on the same magnitude system. The present study examined which conditions enable perceiving nonsymbolic (i.e., an empty set) and symbolic (i.e., 0) representations of null numerosity as a numerical entity, using distance and end effects. In Experiment 1, participants performed magnitude comparisons of notation homogeneous pairs (both numerosities appeared in nonsymbolic or symbolic format), as well as heterogeneous pairs (a nonsymbolic numerosity versus a symbolic one). Comparisons to 0 resulted in faster responses and an attenuated distance effect in all conditions, whereas comparisons to an empty set produced such effects only in the nonsymbolic and symbolic homogeneous conditions. In Experiments 2 and 3, participants performed same/different numerosity judgments with heterogeneous pairs. A distance effect emerged for "different" judgments of 0 and sets of 1 to 9 dots, but not for those with an empty set versus digits 1-9. These findings indicate that perceiving an empty set, but not 0, as a numerical entity is determined by notation homogeneity and task requirements.

Serial and syntactic processing in the visual analysis of multi-digit numbers

The visual analysis of letter strings and digit strings is done by two separate cognitive processes. Recent studies have hypothesized that these processes are not only separate but also qualitatively different, in that they may encode information specific to numbers or to words. To examine this hypothesis and to shed further light on the visual analysis of numbers, we asked adults to read aloud multi-digit strings presented to them for brief durations. Their performance was better in digits on the number's left side than in digits farther to the right, with better performance in the two outer digits than their neighbors. This indicates the digits were processed serially, from left to right. Visual similarity of digits increased the likelihood of errors, and when a digit migrated to an incorrect position, it was most often to an adjacent location. Interestingly, the positions of 0 and 1 were encoded better than the positions of 2-9, and 2-9 were identified better when they were next to 0 or 1. To accommodate these findings, we propose a detailed model for the visual analysis of digit strings. The model assumes imperfect digit detectors in which a digit's visual information leaks to adjacent locations, and a compensation mechanism that inhibits this leakage. Crucially, the compensating inhibition is stronger for 0 and 1 than for the digits 2-9, presumably because of the importance of 0 and 1 in the number system. This sensitivity to 0 and 1 makes the visual analyzer specifically adapted to numbers, not words, and may be one of the brain's reasons to implement the visual analysis of numbers and words in two separate cognitive processes.

Incidental memory following rapid object processing: The role of attention allocation strategies

When observers search for multiple (rather than singular) targets, they are slower and less accurate, yet have better incidental memory for nontarget items encountered during the task (Hout & Goldinger, 2010). One explanation for this may be that observers titrate their attention allocation based on the expected difficulty suggested by search cues. Difficult search cues may implicitly encourage observers to narrow their attention, simultaneously enhancing distractor encoding and hindering peripheral processing. Across three experiments, we manipulated the difficulty of search cues preceding passive visual search for real-world objects, using a Rapid Serial Visual Presentation (RSVP) task to equate item exposure durations. In all experiments, incidental memory was enhanced for distractors encountered while participants monitored for difficult targets. Moreover, in key trials, peripheral shapes appeared at varying eccentricities off center, allowing us to infer the spread and precision of participants' attentional windows. Peripheral item detection and identification decreased when search cues were difficult, even when the peripheral items appeared before targets. These results were not an artifact of sustained vigilance in miss trials, but instead reflect top-down modulation of attention allocation based on task demands. Implications for individual differences are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Contingent capture during search for alphanumerical characters: A case of feature-based capture or of conceptual category membership?

To distinguish if search for alphanumerical characters is based on features or on conceptual category membership, we conducted two experiments where we presented upright and inverted characters as cues in a contingent-capture protocol. Here, only cues matching the top-down search template (e.g., a letter cue when searching for target letters) capture attention and lead to validity effects: shorter search times and fewer errors for validly than invalidly cued targets. Top-down nonmatching cues (e.g., a number cue when searching for target letters) do not capture attention. To tell a feature-based explanation from one based on conceptual category membership, we used both upright (canonical) and inverted characters as cues. These cues share the same features, but inverted cues cannot be conceptually categorized as easily as upright cues. Thus, we expected no difference between upright and inverted cues when search is feature-based, whereas inverted cues would elicit no or at least considerably weaker validity effects if search relies on conceptual category membership. Altogether, the results of both experiments (with overlapping and with separate sets of characters for cues and targets) provide evidence for search based on feature representations, as among other things, significant validity effects were found with upright and inverted characters as cues. However, an influence of category membership was also evident, as validity effects of inverted characters were diminished.

Eye-tracking reveals how observation chart design features affect the detection of patient deterioration: An experimental study

Particular design features intended to improve usability - including graphically displayed observations and integrated colour-based scoring-systems - have been shown to increase the speed and accuracy with which users of hospital observation charts detect abnormal patient observations. We used eye-tracking to evaluate two potential cognitive mechanisms underlying these effects. Novice chart-users completed a series of experimental trials in which they viewed patient data presented on one of three observation chart designs (varied within-subjects), and indicated which observation was abnormal (or that none were). A chart that incorporated both graphically displayed observations and an integrated colour-based scoring-system yielded faster, more accurate responses and fewer, shorter fixations than a graphical chart without a colour-based scoring-system. The latter, in turn, yielded the same advantages over a tabular chart (which incorporated neither design feature). These results suggest that both colour-based scoring-systems and graphically displayed observations improve search efficiency and reduce the cognitive resources required to process vital sign data.

The applicability of eye-controlled highlighting to the field of visual searching

Objective

With the increasing amount of information presented on current human-computer interfaces, eye-controlled highlighting has been proposed, as a new display technique, to optimise users' task performances. However, it is unknown to what extent the eye-controlled highlighting display facilitates visual search performance. The current study examined the facilitative effect of eye-controlled highlighting display technique on visual search with two major attributes of visual stimuli: stimulus type and the visual similarity between targets and distractors.

Method

In Experiment 1, we used digits and Chinese words as materials to explore the generalisation of the facilitative effect of the eye-controlled highlighting. In Experiment 2, we used Chinese words to examine the effect of target-distractor similarity on the facilitation of eye-controlled highlighting display.

Results

The eye-controlling highlighting display improved visual search performance when words were used as searching target and when the target-distractor similarity was high. No facilitative effect was found when digits were used as searching target or target-distractor similarity was low.

Conclusions

The effectiveness of the eye-controlled highlighting on a visual task was influenced by both stimulus type and target-distractor similarity. These findings provided guidelines for modern interface design with eye-based displays implemented.

Size congruity influences visual search via the target template

In numerical comparison experiments, participants are presented with two digits that vary in numerical and physical size, and they select the numerically (or physically) larger (or smaller) of the two digits. Response times are typically faster when numerical and physical size are congruent than when they are incongruent, which is called the size congruity effect (SCE). Although numerical size is unlikely to be a guiding feature in visual search, recent studies have nevertheless observed the SCE in the visual search paradigm. To explain this puzzling fact, we hypothesized that the incongruity between a target's numerical and physical size affects visual search primarily when an attended item is compared to the target template in visual short-term memory. In three experiments, participants searched for a target whose numerical and physical size were distinct from non-target distractors. The SCE and shallow search slopes in Experiment 1 suggest that the target's physical size captured attention, and only then did incongruent numerical size interfere with the response. Instructing participants to attend to physical size in Experiment 2 abolished the SCE, suggesting that participants did not analyze the target's numerical size when they could be confident that physical size was a reliable target cue. Presenting each of two possible target digits in blocks as in Experiment 3 enabled participants to load the visual features of shape and physical size into their target template, and once again the SCE was abolished. The three experiments show that the SCE in visual search can be reduced or eliminated by restricting the target template based on specific physical features and thus discouraging participants from analyzing the target's numerical size.

One visual search, many memory searches: An eye-tracking investigation of hybrid search

Suppose you go to the supermarket with a shopping list of 10 items held in memory. Your shopping expedition can be seen as a combination of visual search and memory search. This is known as "hybrid search." There is a growing interest in understanding how hybrid search tasks are accomplished. We used eye tracking to examine how manipulating the number of possible targets (the memory set size [MSS]) changes how observers (Os) search. We found that dwell time on each distractor increased with MSS, suggesting a memory search was being executed each time a new distractor was fixated. Meanwhile, although the rate of refixation increased with MSS, it was not nearly enough to suggest a strategy that involves repeatedly searching visual space for subgroups of the target set. These data provide a clear demonstration that hybrid search tasks are carried out via a "one visual search, many memory searches" heuristic in which Os examine items in the visual array once with a very low rate of refixations. For each item selected, Os activate a memory search that produces logarithmic response time increases with increased MSS. Furthermore, the percentage of distractors fixated was strongly modulated by the MSS: More items in the MSS led to a higher percentage of fixated distractors. Searching for more potential targets appears to significantly alter how Os approach the task, ultimately resulting in more eye movements and longer response times.

Five Factors that Guide Attention in Visual Search

How do we find what we are looking for? Fundamental limits on visual processing mean that even when the desired target is in our field of view, we often need to search, because it is impossible to recognize everything at once. Searching involves directing attention to objects that might be the target. This deployment of attention is not random. It is guided to the most promising items and locations by five factors discussed here: Bottom-up salience, top-down feature guidance, scene structure and meaning, the previous history of search over time scales from msec to years, and the relative value of the targets and distractors. Modern theories of search need to specify how all five factors combine to shape search behavior. An understanding of the rules of guidance can be used to improve the accuracy and efficiency of socially-important search tasks, from security screening to medical image perception.

Exploring the origin of the number-size congruency effect: Sensitivity or response bias?

Physical size modulates the efficiency of digit comparison, depending on whether the relation of numerical magnitude and physical size is congruent or incongruent (Besner & Coltheart, Neuropsychologia, 17, 467-472, 1979), the number-size congruency effect (NSCE). In addition, Henik and Tzelgov (Memory & Cognition, 10, 389-395, 1982) first reported an NSCE for the reverse task of comparing the physical size of digits such that the numerical magnitude of digits modulated the time required to compare their physical sizes. Does the NSCE in physical comparisons simply reflect a number-mediated bias mechanism related to making decisions and selecting responses about the digit's sizes? Alternatively, or in addition, the NSCE might indicate a true increase in the ability to discriminate small and large font sizes when these sizes are congruent with the digit's symbolic numerical meaning, over and above response bias effects. We present a new research design that permits us to apply signal detection theory to a task that required observers to judge the physical size of digits. Our results clearly demonstrate that the NSCE cannot be reduced to mere response bias effects, and that genuine sensitivity gains for congruent number-size pairings contribute to the NSCE.

Semantic and Syntactic Associations During Word Search Modulate the Relationship Between Attention and Subsequent Memory

Two experiments were conducted to investigate how linguistic information influences attention allocation in visual search and memory for words. In Experiment 1, participants searched for the synonym of a cue word among five words. The distractors included one antonym and three unrelated words. In Experiment 2, participants were asked to judge whether the five words presented on the screen comprise a valid sentence. The relationships among words were sentential, semantically related or unrelated. A memory recognition task followed. Results in both experiments showed that linguistically related words produced better memory performance. We also found that there were significant interactions between linguistic relation conditions and memorization on eye-movement measures, indicating that good memory for words relied on frequent and long fixations during search in the unrelated condition but to a much lesser extent in linguistically related conditions. We conclude that semantic and syntactic associations attenuate the link between overt attention allocation and subsequent memory performance, suggesting that linguistic relatedness can somewhat compensate for a relative lack of attention during word search.

Implicit object naming in visual search: Evidence from phonological competition

During visual search, people are distracted by objects that visually resemble search targets; search is impaired when targets and distractors share overlapping features. In this study, we examined whether a nonvisual form of similarity, overlapping object names, can also affect search performance. In three experiments, people searched for images of real-world objects (e.g., a beetle) among items whose names either all shared the same phonological onset (/bi/), or were phonologically varied. Participants either searched for 1 or 3 potential targets per trial, with search targets designated either visually or verbally. We examined standard visual search (Experiments 1 and 3) and a self-paced serial search task wherein participants manually rejected each distractor (Experiment 2). We hypothesized that people would maintain visual templates when searching for single targets, but would rely more on object names when searching for multiple items and when targets were verbally cued. This reliance on target names would make performance susceptible to interference from similar-sounding distractors. Experiments 1 and 2 showed the predicted interference effect in conditions with high memory load and verbal cues. In Experiment 3, eye-movement results showed that phonological interference resulted from small increases in dwell time to all distractors. The results suggest that distractor names are implicitly activated during search, slowing attention disengagement when targets and distractors share similar names.

Modeling Lag-2 Revisits to Understand Trade-Offs in Mixed Control of Fixation Termination During Visual Search

An important question about eye-movement behavior is when the decision is made to terminate a fixation and program the following saccade. Different approaches have found converging evidence in favor of a mixed-control account, in which there is some overlap between processing information at fixation and planning the following saccade. We examined one interesting instance of mixed control in visual search: lag-2 revisits, during which observers fixate a stimulus, move to a different stimulus, and then revisit the first stimulus on the next fixation. Results show that the probability of lag-2 revisits occurring increased with the number of target-similar stimuli, and revisits were preceded by a brief fixation on the intervening distractor stimulus. We developed the Efficient Visual Sampling (EVS) computational model to simulate our findings (fixation durations and fixation locations) and to provide insight into mixed control of fixations and the perceptual, cognitive, and motor processes that produce lag-2 revisits.

SpAM is convenient but also satisfying: Reply to Verheyen et al. (2016)

Hout, Goldinger, and Ferguson (2013) critically examined the spatial arrangement method (SpAM), originally proposed by Goldstone (1994), as a fast and efficient way to collect similarity data for multidimensional scaling. We found that SpAM produced high-quality data, making it an intuitive and user-friendly alternative to the classic "pairwise" method. Verheyen, Voorspoels, Vanpaemel, and Storms (2016) reexamined our data and raised 3 caveats regarding SpAM. In this reply, we suggest that Verheyen et al. mischaracterized our reported data as representing the entire range of potential SpAM data. SpAM results might appear more nuanced with modified instructions or stimuli. By contrast, the pairwise method is inherently limited because of its laborious, serial nature. We also demonstrate that, when the methods are equated in terms of required data-collection time, SpAM is clearly superior in terms of predicting classification data. We agree that caution is required when adopting a new method but suggest that fair assessment of SpAM requires a richer data set.

Interaction between numbers and size during visual search

The current study investigates an interaction between numbers and physical size (i.e. size congruity) in visual search. In three experiments, participants had to detect a physically large (or small) target item among physically small (or large) distractors in a search task comprising single-digit numbers. The relative numerical size of the digits was varied, such that the target item was either among the numerically large or small numbers in the search display and the relation between numerical and physical size was either congruent or incongruent. Perceptual differences of the stimuli were controlled by a condition in which participants had to search for a differently coloured target item with the same physical size and by the usage of LCD-style numbers that were matched in visual similarity by shape transformations. The results of all three experiments consistently revealed that detecting a physically large target item is significantly faster when the numerical size of the target item is large as well (congruent), compared to when it is small (incongruent). This novel finding of a size congruity effect in visual search demonstrates an interaction between numerical and physical size in an experimental setting beyond typically used binary comparison tasks, and provides important new evidence for the notion of shared cognitive codes for numbers and sensorimotor magnitudes. Theoretical consequences for recent models on attention, magnitude representation and their interactions are discussed.

Bottom-up and top-down attentional contributions to the size congruity effect

The size congruity effect refers to the interaction between the numerical and physical (i.e., font) sizes of digits in a numerical (or physical) magnitude selection task. Although various accounts of the size congruity effect have attributed this interaction to either an early representational stage or a late decision stage, only Risko, Maloney, and Fugelsang (Attention, Perception, & Psychophysics, 75, 1137-1147, 2013) have asserted a central role for attention. In the present study, we used a visual search paradigm to further study the role of attention in the size congruity effect. In Experiments 1 and 2, we showed that manipulating top-down attention (via the task instructions) had a significant impact on the size congruity effect. The interaction between numerical and physical size was larger for numerical size comparison (Exp. 1) than for physical size comparison (Exp. 2). In the remaining experiments, we boosted the feature salience by using a unique target color (Exp. 3) or by increasing the display density by using three-digit numerals (Exps. 4 and 5). As expected, a color singleton target abolished the size congruity effect. Searching for three-digit targets based on numerical size (Exp. 4) resulted in a large size congruity effect, but search based on physical size (Exp. 5) abolished the effect. Our results reveal a substantial role for top-down attention in the size congruity effect, which we interpreted as support for a shared-decision account.

Using multidimensional scaling to quantify similarity in visual search and beyond

Visual search is one of the most widely studied topics in vision science, both as an independent topic of interest, and as a tool for studying attention and visual cognition. A wide literature exists that seeks to understand how people find things under varying conditions of difficulty and complexity, and in situations ranging from the mundane (e.g., looking for one's keys) to those with significant societal importance (e.g., baggage or medical screening). A primary determinant of the ease and probability of success during search are the similarity relationships that exist in the search environment, such as the similarity between the background and the target, or the likeness of the non-targets to one another. A sense of similarity is often intuitive, but it is seldom quantified directly. This presents a problem in that similarity relationships are imprecisely specified, limiting the capacity of the researcher to examine adequately their influence. In this article, we present a novel approach to overcoming this problem that combines multi-dimensional scaling (MDS) analyses with behavioral and eye-tracking measurements. We propose a method whereby MDS can be repurposed to successfully quantify the similarity of experimental stimuli, thereby opening up theoretical questions in visual search and attention that cannot currently be addressed. These quantifications, in conjunction with behavioral and oculomotor measures, allow for critical observations about how similarity affects performance, information selection, and information processing. We provide a demonstration and tutorial of the approach, identify documented examples of its use, discuss how complementary computer vision methods could also be adopted, and close with a discussion of potential avenues for future application of this technique.

Faster than the speed of rejection: Object identification processes during visual search for multiple targets

When engaged in a visual search for two targets, participants are slower and less accurate in their responses, relative to their performance when searching for singular targets. Previous work on this "dual-target cost" has primarily focused on the breakdown of attentional guidance when looking for two items. Here, we investigated how object identification processes are affected by dual-target search. Our goal was to chart the speed at which distractors could be rejected, to assess whether dual-target search impairs object identification. To do so, we examined the capacity coefficient, which measures the speed at which decisions can be made, and provides a baseline of parallel performance against which to compare. We found that participants could search at or above this baseline, suggesting that dual-target search does not impair object identification abilities. We also found substantial differences in performance when participants were asked to search for simple versus complex images. Somewhat paradoxically, participants were able to reject complex images more rapidly than simple images. We suggest that this reflects the greater number of features that can be used to identify complex images, a finding that has important consequences for understanding object identification in visual search more generally.

Target templates: the precision of mental representations affects attentional guidance and decision-making in visual search

When people look for things in the environment, they use target templates-mental representations of the objects they are attempting to locate-to guide attention and to assess incoming visual input as potential targets. However, unlike laboratory participants, searchers in the real world rarely have perfect knowledge regarding the potential appearance of targets. In seven experiments, we examined how the precision of target templates affects the ability to conduct visual search. Specifically, we degraded template precision in two ways: 1) by contaminating searchers' templates with inaccurate features, and 2) by introducing extraneous features to the template that were unhelpful. We recorded eye movements to allow inferences regarding the relative extents to which attentional guidance and decision-making are hindered by template imprecision. Our findings support a dual-function theory of the target template and highlight the importance of examining template precision in visual search.