Visual similarity effects in categorical search

Eye can read your mind: decoding gaze fixations to reveal categorical search targets

Is it possible to infer a person's goal by decoding their fixations on objects? Two groups of participants categorically searched for either a teddy bear or butterfly among random category distractors, each rated as high, medium, or low in similarity to the target classes. Target-similar objects were preferentially fixated in both search tasks, demonstrating information about target category in looking behavior. Different participants then viewed the searchers' scanpaths, superimposed over the target-absent displays, and attempted to decode the target category (bear/butterfly). Bear searchers were classified perfectly; butterfly searchers were classified at 77%. Bear and butterfly Support Vector Machine (SVM) classifiers were also used to decode the same preferentially fixated objects and found to yield highly comparable classification rates. We conclude that information about a person's search goal exists in fixation behavior, and that this information can be behaviorally decoded to reveal a search target-essentially reading a person's mind by analyzing their fixations.

Modeling guidance and recognition in categorical search: bridging human and computer object detection

Search is commonly described as a repeating cycle of guidance to target-like objects, followed by the recognition of these objects as targets or distractors. Are these indeed separate processes using different visual features? We addressed this question by comparing observer behavior to that of support vector machine (SVM) models trained on guidance and recognition tasks. Observers searched for a categorically defined teddy bear target in four-object arrays. Target-absent trials consisted of random category distractors rated in their visual similarity to teddy bears. Guidance, quantified as first-fixated objects during search, was strongest for targets, followed by target-similar, medium-similarity, and target-dissimilar distractors. False positive errors to first-fixated distractors also decreased with increasing dissimilarity to the target category. To model guidance, nine teddy bear detectors, using features ranging in biological plausibility, were trained on unblurred bears then tested on blurred versions of the same objects appearing in each search display. Guidance estimates were based on target probabilities obtained from these detectors. To model recognition, nine bear/nonbear classifiers, trained and tested on unblurred objects, were used to classify the object that would be fixated first (based on the detector estimates) as a teddy bear or a distractor. Patterns of categorical guidance and recognition accuracy were modeled almost perfectly by an HMAX model in combination with a color histogram feature. We conclude that guidance and recognition in the context of search are not separate processes mediated by different features, and that what the literature knows as guidance is really recognition performed on blurred objects viewed in the visual periphery.

Modelling eye movements in a categorical search task

We introduce a model of eye movements during categorical search, the task of finding and recognizing categorically defined targets. It extends a previous model of eye movements during search (target acquisition model, TAM) by using distances from an support vector machine classification boundary to create probability maps indicating pixel-by-pixel evidence for the target category in search images. Other additions include functionality enabling target-absent searches, and a fixation-based blurring of the search images now based on a mapping between visual and collicular space. We tested this model on images from a previously conducted variable set-size (6/13/20) present/absent search experiment where participants searched for categorically defined teddy bear targets among random category distractors. The model not only captured target-present/absent set-size effects, but also accurately predicted for all conditions the numbers of fixations made prior to search judgements. It also predicted the percentages of first eye movements during search landing on targets, a conservative measure of search guidance. Effects of set size on false negative and false positive errors were also captured, but error rates in general were overestimated. We conclude that visual features discriminating a target category from non-targets can be learned and used to guide eye movements during categorical search.

Choosing colors for map display icons using models of visual search

OBJECTIVE

We show how to choose colors for icons on maps to minimize search time using predictions of a model of visual search.

BACKGROUND

The model analyzes digital images of a search target (an icon on a map) and a search display (the map containing the icon) and predicts search time as a function of target-distractor color distinctiveness and target eccentricity.

METHOD

We parameterized the model using data from a visual search task and performed a series of optimization tasks to test the model's ability to choose colors for icons to minimize search time across icons. Map display designs made by this procedure were tested experimentally. In a follow-up experiment, we examined the model's flexibility to assign colors in novel search situations.

RESULTS

The model fits human performance, performs well on the optimization tasks, and can choose colors for icons on maps with novel stimuli to minimize search time without requiring additional model parameter fitting.

CONCLUSION

Models of visual search can suggest color choices that produce search time reductions for display icons.

APPLICATION

Designers should consider constructing visual search models as a low-cost method of evaluating color assignments.

Searching for something familiar or novel: top-down attentional selection of specific items or object categories

Visual search is often guided by top-down attentional templates that specify target-defining features. But search can also occur at the level of object categories. We measured the N2pc component, a marker of attentional target selection, in two visual search experiments where targets were defined either categorically (e.g., any letter) or at the item level (e.g., the letter C) by a prime stimulus. In both experiments, an N2pc was elicited during category search, in both familiar and novel contexts (Experiment 1) and with symbolic primes (Experiment 2), indicating that, even when targets are only defined at the category level, they are selected at early sensory-perceptual stages. However, the N2pc emerged earlier and was larger during item-based search compared with category-based search, demonstrating the superiority of attentional guidance by item-specific templates. We discuss the implications of these findings for attentional control and category learning.

Evaluating the contribution of shape attributes to recognition using the minimal transient discrete cue protocol

Subjects were tested for their ability to identify objects that were represented by an array of dots that marked the major contours, usually only the outer boundary. Each dot was briefly flashed to make its position known, and a major variable was the time interval that was required to flash all the dots for a given shape. Recognition declined as the total time for display of the dot inventory was increased. Each shape was shown to a given subject only once and it was either recognized -- named - or not. Although the recorded response was binary, a large number of subjects was tested, which made it possible to derive regression functions and thus specify an intercept and slope for each shape. Shapes differed substantially in their slopes, which is likely due to the amount of redundant information provided by neighboring dots. Indices of shape attributes were also derived, specifically Attneave's indices of complexity, mean curvature, inflection count, and symmetry. Three of the four shape attributes were significantly related to intercept and slope levels, but none made a substantial contribution. This suggests that these attributes are not essential properties that define shapes and allow for recognition.

Search performance with discrete-cell stimulus arrays: filtered naturalistic images and probabilistic markers

In order to partially fill the gap between search studies with artificial and naturalistic stimuli, experiments with segmented and spatially filtered images of real stimuli in the discrete cells of search arrays were conducted. RT × set size functions obtained with the spatially filtered arrays were compared with those obtained with geometric and other types of arrays. With the aid of Fourier analysis, components of target certainty/uncertainty, target-distractor similarity/dissimilarity, and distractor homogeneity/heterogeneity were evaluated for their effects on search performance and efficiency. The comparative effects of valid/invalid and ambiguous/unambiguous markers of search items were included in the experiments to probe the potential contributions of top-down and bottom-up control to search with naturalistic stimuli. Visual coding in the form of spatial frequency passbands and top-down preferences for distractor search were used to explain the results.

Effects of part-based similarity on visual search: the Frankenbear experiment

Do the target-distractor and distractor-distractor similarity relationships known to exist for simple stimuli extend to real-world objects, and are these effects expressed in search guidance or target verification? Parts of photorealistic distractors were replaced with target parts to create four levels of target-distractor similarity under heterogenous and homogenous conditions. We found that increasing target-distractor similarity and decreasing distractor-distractor similarity impaired search guidance and target verification, but that target-distractor similarity and heterogeneity/homogeneity interacted only in measures of guidance; distractor homogeneity lessens effects of target-distractor similarity by causing gaze to fixate the target sooner, not by speeding target detection following its fixation.