Visual Search and the Collapse of Categorization

Explicit and implicit category learning in categorical visual search

Categorical search has been heavily investigated over the past decade, mostly using natural categories that leave the underlying category mental representation unknown. The categorization literature offers several theoretical accounts of category mental representations. One prominent account is that separate learning systems account for classification: an explicit learning system that relies on easily verbalized rules and an implicit learning system that relies on an associatively learned (nonverbalizable) information integration strategy. The current study assessed the contributions of these separate category learning systems in the context of categorical search using simple stimuli. Participants learned to classify sinusoidal grating stimuli according to explicit or implicit categorization strategies, followed by a categorical search task using these same stimulus categories. Computational modeling determined which participants used the appropriate classification strategy during training and search, and eye movements collected during categorical search were assessed. We found that the trained categorization strategies overwhelmingly transferred to the verification (classification response) phase of search. Implicit category learning led to faster search response and shorter target dwell times relative to explicit category learning, consistent with the notion that explicit rule classification relies on a more deliberative response strategy. Participants who transferred the correct category learning strategy to the search guidance phase produced stronger search guidance (defined as the proportion of trials on which the target was the first item fixated) with evidence of greater guidance in implicit-strategy learners. This demonstrates that both implicit and explicit categorization systems contribute to categorical search and produce dissociable patterns of data.

Can categorical knowledge be used in visual search?

Smith, Redford, Gent, and Washburn (2005) have proposed a new categorization paradigm called the visual-search categorization task to study how display size affects categorization performance. Their results show that, in a wide range of conditions, category knowledge collapses as soon as multiple stimuli are simultaneously displayed in a scene. This result is surprising and important considering that humans parse and categorize objects from complex scenes on a daily basis. However, Smith et al. only studied one kind of category structure. This article presents the results of three experiments exploring the effect of display size on perceptual categorization as a function of category structure. We show that rule-based and information-integration categories are differently affected by display size in the visual search categorization task. For rule-based structures, target-present and target-absent trials are not much affected by display size. However, the effect of display size is bigger for information-integration category structures, and much more pronounced for target-absent trials than for target-present trials. A follow-up experiment shows that target redundancy (i.e., having more than one target in the display) does not improve performance with information-integration category structures. These results suggest that categories may be learned differently depending on their underlying structure, and that the resulting category representation may influence performance in the visual search categorization task.

Occipitotemporal Category Representations Are Sensitive to Abstract Category Boundaries Defined by Generalization Demands

Categorization involves organizing perceptual information so as to maximize differences along dimensions that predict class membership while minimizing differences along dimensions that do not. In the current experiment, we investigated how neural representations reflecting learned category structure vary according to generalization demands. We asked male and female human participants to switch between two rules when determining whether stimuli should be considered members of a single known category. When categorizing according to the "strict" rule, participants were required to limit generalization to make fine-grained distinctions between stimuli and the category prototype. When categorizing according to the "lax" rule, participants were required to generalize category knowledge to highly atypical category members. As expected, frontoparietal regions were primarily sensitive to decisional demands (i.e., the distance of each stimulus from the active category boundary), whereas occipitotemporal representations were primarily sensitive to stimulus typicality (i.e., the similarity between each exemplar and the category prototype). Interestingly, occipitotemporal representations of stimulus typicality differed between rules. While decoding models were able to predict unseen data when trained and tested on the same rule, they were unable to do so when trained and tested on different rules. We additionally found that the discriminability of the multivariate signal negatively covaried with distance from the active category boundary. Thus, whereas many accounts of occipitotemporal cortex emphasize its important role in transforming visual information to accentuate learned category structure, our results highlight the flexible nature of these representations with regards to transient decisional demands.SIGNIFICANCE STATEMENT Occipitotemporal representations are known to reflect category structure and are often assumed to be largely invariant with regards to transient decisional demands. We found that representations of equivalent stimuli differed between strict and lax generalization rules, and that the discriminability of these representations increased as distance from abstract category boundaries decreased. Our results therefore indicate that occipitotemporal representations are flexibly modulated by abstract decisional factors.

The effect of integration masking on visual processing in perceptual categorization

Learning to recognize and categorize objects is an essential cognitive skill allowing animals to function in the world. However, animals rarely have access to a canonical view of an object in an uncluttered environment. Hence, it is essential to study categorization under noisy, degraded conditions. In this article, we explore how the brain processes categorization stimuli in low signal-to-noise conditions using multivariate pattern analysis. We used an integration masking paradigm with mask opacity of 50%, 60%, and 70% inside a magnetic resonance imaging scanner. The results show that mask opacity affects blood-oxygen-level dependent (BOLD) signal in visual processing areas (V1, V2, V3, and V4) but does not affect the BOLD signal in brain areas traditionally associated with categorization (prefrontal cortex, striatum, hippocampus). This suggests that when a stimulus is difficult to extract from its background (e.g., low signal-to-noise ratio), the visual system extracts the stimulus and that activity in areas typically associated with categorization are not affected by the difficulty level of the visual conditions. We conclude with implications of this result for research on visual attention, categorization, and the integration of these fields.

Change detection: training and transfer

Observers often fail to notice even dramatic changes to their environment, a phenomenon known as change blindness. If training could enhance change detection performance in general, then it might help to remedy some real-world consequences of change blindness (e.g. failing to detect hazards while driving). We examined whether adaptive training on a simple change detection task could improve the ability to detect changes in untrained tasks for young and older adults. Consistent with an effective training procedure, both young and older adults were better able to detect changes to trained objects following training. However, neither group showed differential improvement on untrained change detection tasks when compared to active control groups. Change detection training led to improvements on the trained task but did not generalize to other change detection tasks.

Searching Through the Hierarchy: How Level of Target Categorization Affects Visual Search

Does the same basic-level advantage commonly observed in the categorization literature also hold for targets in a search task? We answered this question by first conducting a category verification task to define a set of categories showing a standard basic-level advantage, which we then used as stimuli in a search experiment. Participants were cued with a picture preview of the target or its category name at either superordinate, basic, or subordinate levels, then shown a target-present/absent search display. Although search guidance and target verification was best using pictorial cues, the effectiveness of the categorical cues depended on the hierarchical level. Search guidance was best for the specific subordinate level cues, while target verification showed a standard basic-level advantage. These findings demonstrate different hierarchical advantages for guidance and verification in categorical search. We interpret these results as evidence for a common target representation underlying categorical search guidance and verification.

Effects of part-based similarity on visual search: the Frankenbear experiment

Do the target-distractor and distractor-distractor similarity relationships known to exist for simple stimuli extend to real-world objects, and are these effects expressed in search guidance or target verification? Parts of photorealistic distractors were replaced with target parts to create four levels of target-distractor similarity under heterogenous and homogenous conditions. We found that increasing target-distractor similarity and decreasing distractor-distractor similarity impaired search guidance and target verification, but that target-distractor similarity and heterogeneity/homogeneity interacted only in measures of guidance; distractor homogeneity lessens effects of target-distractor similarity by causing gaze to fixate the target sooner, not by speeding target detection following its fixation.

The impact of Relative Prevalence on dual-target search for threat items from airport X-ray screening

The probability of target presentation in visual search tasks influences target detection performance: this is known as the prevalence effect (Wolfe et al., 2005). Additionally, searching for several targets simultaneously reduces search performance: this is known as the dual-target cost (DTC: Menneer et al., 2007). The interaction between the DTC and prevalence effect was investigated in a single study by presenting one target in dual-target search at a higher level of prevalence than the other target (Target A: 45% Prevalence; Target B: 5% Prevalence). An overall DTC was found for both RTs and response accuracy. Furthermore, there was an effect of target prevalence in dual-target search, suggesting that, when one target is presented at a higher level of prevalence than the other, both the dual-target cost and the prevalence effect contribute to decrements in performance. The implications for airport X-ray screening are discussed.

Sustained activities and retrieval in a computational model of the perirhinal cortex

The perirhinal cortex is involved not only in object recognition and novelty detection but also in multimodal integration, reward association, and visual working memory. We propose a computational model that focuses on the role of the perirhinal cortex in working memory, particularly with respect to sustained activities and memory retrieval. This model describes how different partial informations are integrated into assemblies of neurons that represent the identity of an object. Through dopaminergic modulation, the resulting clusters can retrieve the global information with recurrent interactions between neurons. Dopamine leads to sustained activities after stimulus disappearance that form the basis of the involvement of the perirhinal cortex in visual working memory processes. The information carried by a cluster can also be retrieved by a partial thalamic or prefrontal stimulation. Thus, we suggest that areas involved in planning and memory coordination encode a pointer to access the detailed information encoded in the associative cortex such as the perirhinal cortex.

Exemplar-based inference in multi-attribute decision making: Contingent, not automatic, strategy shifts?

Several studies propose that exemplar retrieval contributes to multi-attribute decisions. The authors have proposed a process theory enabling a priori predictions of what cognitive representations people use as input to their judgment process (Sigma, for “summation”; P. Juslin, L. Karlsson, & H. Olsson, 2008). According to Sigma, exemplar retrieval is a back-up system when the task does not allow for additive and linear abstraction and integration of cue-criterion knowledge (e.g., when the task is non-additive). An important question is to what extent such shifts occur spontaneously as part of automatic procedures, such as error-minimization with the Delta rule, or if they are controlled strategy shifts contingent on the ability to identify a sufficiently successful judgment strategy. In this article data are reviewed that demonstrate a shift between exemplar memory and cue abstraction, as well as data where the expected shift does not occur. In contrast to a common assumption of previous models, these results suggest a controlled and contingent strategy shift.

Low target prevalence is a stubborn source of errors in visual search tasks

In visual search tasks, observers look for targets in displays containing distractors. Likelihood that targets will be missed varies with target prevalence, the frequency with which targets are presented across trials. Miss error rates are much higher at low target prevalence (1%-2%) than at high prevalence (50%). Unfortunately, low prevalence is characteristic of important search tasks such as airport security and medical screening where miss errors are dangerous. A series of experiments show this prevalence effect is very robust. In signal detection terms, the prevalence effect can be explained as a criterion shift and not a change in sensitivity. Several efforts to induce observers to adopt a better criterion fail. However, a regime of brief retraining periods with high prevalence and full feedback allows observers to hold a good criterion during periods of low prevalence with no feedback.

Specific-Token Effects in Screening Tasks: Possible Implications for Aviation Security

Screeners at airport security checkpoints perform an important categorization task in which they search for threat items in complex x-ray images. But little is known about how the processes of categorization stand up to visual complexity. The authors filled this research gap with screening tasks in which participants searched for members of target categories in visual displays. The authors found that when targets were sampled with replacement and repetition, participant screeners relied on recognizing familiar targets and had great difficulty using category-general knowledge. The authors observed a "heartbeat" in detection performance--it improved while test images repeated but dropped sharply when unfamiliar targets from the same categories appeared. This reliance on familiarity illuminates the processes of categorization under conditions of visual complexity and suggests limits on those processes. This reliance also has implications for the training and evaluation of screeners in the field.