Effects of target typicality on categorical search

The time course of category-based attentional template pre-activation depends on the category framework

When searching for a target defined by a set of objects, attention can be directed toward task-relevant objects by creating a category-based attentional template (CAT). Previous studies have found that CAT can be activated before the onset of the target. However, the time course of CAT pre-activation and whether the category framework (prototypical or semantic) can modulate it remain unclear. To explore the time course of CAT pre-activation, we employed a rapid serial probe presentation paradigm (RSPP) with event-related potentials (ERPs). To investigate the effect of the category framework on the time course of CAT pre-activation, the target category was defined as the prototypical category (Experiment 1) or the semantic category (Experiment 2). The results showed that the prototype-based CAT was pre-activated 300 ms prior to the target, whereas the semantics-based CAT was pre-activated 1500 ms before the onset of the target. The difference in the time course of pre-activation between the two CAT types indicates that the category framework can modulate the time course of CAT pre-activation. Additionally, during the attentional selection phase, an overall comparison of the target revealed that a larger N2pc was elicited by the prototype-based CAT than by the semantics-based CAT, suggesting that the prototype-based CAT could capture more attention than the semantics-based CAT. The findings on the difference between the two CAT frameworks in the preparatory and attentional selection phases provide more evidence for categorical information in visual search and extend our understanding of the mechanism of categorical attentional selection.

Explicit and implicit category learning in categorical visual search

Categorical search has been heavily investigated over the past decade, mostly using natural categories that leave the underlying category mental representation unknown. The categorization literature offers several theoretical accounts of category mental representations. One prominent account is that separate learning systems account for classification: an explicit learning system that relies on easily verbalized rules and an implicit learning system that relies on an associatively learned (nonverbalizable) information integration strategy. The current study assessed the contributions of these separate category learning systems in the context of categorical search using simple stimuli. Participants learned to classify sinusoidal grating stimuli according to explicit or implicit categorization strategies, followed by a categorical search task using these same stimulus categories. Computational modeling determined which participants used the appropriate classification strategy during training and search, and eye movements collected during categorical search were assessed. We found that the trained categorization strategies overwhelmingly transferred to the verification (classification response) phase of search. Implicit category learning led to faster search response and shorter target dwell times relative to explicit category learning, consistent with the notion that explicit rule classification relies on a more deliberative response strategy. Participants who transferred the correct category learning strategy to the search guidance phase produced stronger search guidance (defined as the proportion of trials on which the target was the first item fixated) with evidence of greater guidance in implicit-strategy learners. This demonstrates that both implicit and explicit categorization systems contribute to categorical search and produce dissociable patterns of data.

Assessing satisfaction of search in virtual mammograms for experienced and novice searchers

Purpose

Satisfaction of search (SOS) is a phenomenon where searchers are more likely to miss a lesion/target after detecting a first lesion/target. Here, we investigated SOS for masses and calcifications in virtual mammograms with experienced and novice searchers to determine the extent to which: (1) SOS affects breast lesion detection, (2) similarity between lesions impacts detection, and (3) experience impacts SOS rates.

Approach

The open virtual clinical trials framework was used to simulate the breast anatomy of patients, and up to two simulated masses and/or single-calcifications were inserted into the breast models. Experienced searchers (residents, fellows, and radiologists with breast imaging experience) and novice searchers (undergraduates who had no breast imaging experience) were instructed to search for up to two lesions (masses and calcifications) per image.

Results

2 × 2 mixed factors analysis of variances (ANOVAs) were run with: (1) single versus second lesion hit rates, (2) similar versus dissimilar second-lesion hit rates, and (3) similar versus dissimilar second-lesion response times as within-subject factors and experience as the between subject's factor. The ANOVAs demonstrated that: (1) experienced and novice searchers made a significant amount of SOS errors, (2) similarity had little impact on experienced searchers, but novice searchers were more likely to miss a dissimilar second lesion compared to when it was similar to a detected first lesion, (3) experienced and novice searchers were faster at finding similar compared to dissimilar second lesions.

Conclusions

We demonstrated that SOS is a significant cause of lesion misses in virtual mammograms and that reader experience impacts detection rates for similar compared to dissimilar abnormalities. These results suggest that experience may impact strategy and/or recognition with theoretical implications for determining why SOS occurs.

The influence of category representativeness on the low prevalence effect in visual search

Visual search is greatly affected by the appearance rate of given target types, such that low-prevalence items are harder to detect, which has consequences for real-world search tasks where target frequency cannot be balanced. However, targets that are highly representative of a categorically defined task set are also easier to find. We hypothesized that targets that are highly representative are less vulnerable to low-prevalence effects because an observer's attentional set prioritizes guidance toward them even when they are rare. We assessed this hypothesis by first determining the categorical structure of "prohibited carry-ons" via an exemplar-naming task, and used this structure to assess how category representativeness interacted with prevalence. Specifically, from the exemplar-naming task we selected a commonly named (knives) and rarely named (gas cans) target for a search task in which one of the targets was shown infrequently. As predicted, highly representative targets were found more easily than their less representative counterparts, but they also were less affected by prevalence manipulations. Experiment 1b replicated the results with targets matched for emotional valence (water bottles and fireworks). These findings demonstrate the powerful explanatory power of theories of attentional guidance that incorporate the dynamic influence of recent experience with the knowledge that comes from life experience to better predict behavioral outcomes associated with high-stakes search environments.

Well-designed medical pictograms accelerate search

Two types of newly designed pharmaceutical pictograms (with and without context) were compared with an existing type of certified pictograms regarding their search efficiency. Each of the 30 participants had to search a total of 1'090 "fictitious" medical shelves for a certain box defined by the amount and type of medical instructions given (memory size) and presented among a variable number of other boxes (set size). The boxes contained the different types of pictograms mentioned above. Calculated factorial analyses on reaction time data, among others, showed that the two newly designed pictogram types make search more efficient compared to existing types of pictograms (i.e., flatter reaction time x set size slopes). Furthermore, regardless of the type of pictogram, this set size effect became more pronounced with larger memory sizes. Overall, the newly designed pictograms need fewer attentional resources and therefore might help to increase patient adherence.

Negative cues minimize visual search specificity effects

Prior target knowledge (i.e., positive cues) improves visual search performance. However, there is considerable debate about whether distractor knowledge (i.e., negative cues) can guide search. Some studies suggest the active suppression of negatively cued search items, while others suggest the initial capture of attention by negatively cued items. Prior work has used pictorial or specific text cues but has not explicitly compared them. We build on that work by comparing positive and negative cues presented pictorially and as categorical text labels using photorealistic objects and eye movement measures. Search displays contained a target (cued on positive trials), a lure from the target category (cued on negative trials), and four categorically-unrelated distractors. Search performance with positive cues resulted in stronger attentional guidance and faster object recognition for pictorial relative to categorical cues (i.e., a pictorial advantage, suggesting specific visual details afforded by pictorial cues improved search). However, in most search performance metrics, negative cues mitigate the pictorial advantage. Given that the negatively cued items captured attention, generated target guidance but mitigated the pictorial advantage, these results are partly consistent with both existing theories. Specific visual details provided in positive cues produce a large pictorial advantage in all measures, whereas specific visual details in negative cues only produce a small pictorial advantage for object recognition but not for attentional guidance. This asymmetry in the pictorial advantage suggests that the down-weighting of specific negatively cued visual features is less efficient than the up-weighting of specific positively cued visual features.

Backward and forward neck tilt affects perceptual bias when interpreting ambiguous figures

The relationships between posture and perception have already been investigated in several studies. However, it is still unclear how perceptual bias and experiential contexts of human perception affect observers’ perception when posture is changed. In this study, we hypothesized that a change in the perceptual probability caused by perceptual bias also depends on posture. In order to verify this hypothesis, we used the Necker cube with two types of appearance, from above and below, although the input is constant, and investigated the change of the probability of perceptual content. Specifically, we asked observers their perception of the appearance of the Necker cube placed at any of the five angles in the space of virtual reality. There were two patterns of neck movement, vertical and horizontal. During the experiment, pupil diameter, one of the cognitive indices, was also measured. Results showed that during the condition of looking down vertically, the probability of the viewing-from-above perception of the Necker cube was significantly greater than during the condition of looking up. Interestingly, the pupillary results were also consistent with the probability of the perception. These results indicate that perception was modulated by the posture of the neck and suggest that neck posture is incorporated into ecological constraints.

Real-world object categories and scene contexts conjointly structure statistical learning for the guidance of visual search

We examined how object categories and scene contexts act in conjunction to structure the acquisition and use of statistical regularities to guide visual search. In an exposure session, participants viewed five object exemplars in each of two colors in each of 42 real-world categories. Objects were presented individually against scene context backgrounds. Exemplars within a category were presented with different contexts as a function of color (e.g., the five red staplers were presented with a classroom scene, and the five blue staplers with an office scene). Participants then completed a visual search task, in which they searched for novel exemplars matching a category label cue among arrays of eight objects superimposed over a scene background. In the context-match condition, the color of the target exemplar was consistent with the color associated with that combination of category and scene context from the exposure phase (e.g., a red stapler in a classroom scene). In the context-mismatch condition, the color of the target was not consistent with that association (e.g., a red stapler in an office scene). In two experiments, search response time was reliably lower in the context-match than in the context-mismatch condition, demonstrating that the learning of category-specific color regularities was itself structured by scene context. The results indicate that categorical templates retrieved from long-term memory are biased toward the properties of recent exemplars and that this learning is organized in a scene-specific manner.

Typicality modulates the visual awareness of objects

In the present study, we assessed whether typicality can influence the visual awareness of objects. Participants tracked moving images of objects and counted how often members of one category bounced off the edges of the display. On the last trial, an unexpected object moved across the display. In our first two experiments, this object could belong to the same category as the tracked or untracked objects. While participants were more likely to notice atypical members of the untracked category, this pattern of results reversed when participants tracked atypical objects. In our last two experiments, the unexpected object could belong to the same category as the tracked objects or a new category of objects. In this case, participants were more likely to notice typical members of both the tracked category and the new category. Together, these findings suggest that typicality can modulate the visual awareness of objects.

Interaction of prior category knowledge and novel statistical patterns during visual search for real-world objects

Two aspects of real-world visual search are typically studied in parallel: category knowledge (e.g., searching for food) and visual patterns (e.g., predicting an upcoming street sign from prior street signs). Previous visual search studies have shown that prior category knowledge hinders search when targets and distractors are from the same category. Other studies have shown that task-irrelevant patterns of non-target objects can enhance search when targets appear in locations that previously contained these irrelevant patterns. Combining EEG (N2pc ERP component, a neural marker of target selection) and behavioral measures, the present study investigated how search efficiency is simultaneously affected by prior knowledge of real-world objects (food and toys) and irrelevant visual patterns (sequences of runic symbols) within the same paradigm. We did not observe behavioral differences between locating items in patterned versus random locations. However, the N2pc components emerged sooner when search items appeared in the patterned location, compared to the random location, with a stronger effect when search items were targets, as opposed to non-targets categorically related to the target. A multivariate pattern analysis revealed that neural responses during search trials in the same time window reflected where the visual patterns appeared. Our finding contributes to our understanding of how knowledge acquired prior to the search task (e.g., category knowledge) interacts with new content within the search task.

Target templates in low target-distractor discriminability visual search have higher resolution, but the advantage they provide is short-lived

When you search repeatedly for a set of items among very similar distractors, does that make you more efficient in locating the targets? To address this, we had observers search for two categories of targets among the same set of distractors across trials. Visual and conceptual similarity of the stimuli were validated with a multidimensional scaling analysis, and separately using a deep neural network model. After a few blocks of visual search trials, the distractor set was replaced. In three experiments, we manipulated the level of discriminability between the targets and distractors before and after the distractors were replaced. Our results suggest that in the presence of repeated distractors, observers generally become more efficient. However, the difficulty of the search task does impact how efficient people are when the distractor set is replaced. Specifically, when the training is easy, people are more impaired in a difficult transfer test. We attribute this effect to the precision of the target template generated during training. In particular, a coarse target template is created when the target and distractors are easy to discriminate. These coarse target templates do not transfer well in a context with new distractors. This suggests that learning with more distinct targets and distractors can result in lower performance when context changes, but observers recover from this effect quickly (within a block of search trials).

COCO-Search18 fixation dataset for predicting goal-directed attention control

Attention control is a basic behavioral process that has been studied for decades. The currently best models of attention control are deep networks trained on free-viewing behavior to predict bottom-up attention control - saliency. We introduce COCO-Search18, the first dataset of laboratory-quality goal-directed behavior large enough to train deep-network models. We collected eye-movement behavior from 10 people searching for each of 18 target-object categories in 6202 natural-scene images, yielding [Formula: see text] 300,000 search fixations. We thoroughly characterize COCO-Search18, and benchmark it using three machine-learning methods: a ResNet50 object detector, a ResNet50 trained on fixation-density maps, and an inverse-reinforcement-learning model trained on behavioral search scanpaths. Models were also trained/tested on images transformed to approximate a foveated retina, a fundamental biological constraint. These models, each having a different reliance on behavioral training, collectively comprise the new state-of-the-art in predicting goal-directed search fixations. Our expectation is that future work using COCO-Search18 will far surpass these initial efforts, finding applications in domains ranging from human-computer interactive systems that can anticipate a person's intent and render assistance to the potentially early identification of attention-related clinical disorders (ADHD, PTSD, phobia) based on deviation from neurotypical fixation behavior.

Categorical cuing: Object categories structure the acquisition of statistical regularities to guide visual search

Recent statistical regularities have been demonstrated to influence visual search across a wide variety of learning mechanisms and search features. To function in the guidance of real-world search, however, such learning must be contingent on the context in which the search occurs and the object that is the target of search. The former has been studied extensively under the rubric of contextual cuing. Here, we examined, for the first time, categorical cuing: The role of object categories in structuring the acquisition of statistical regularities used to guide visual search. After an exposure session in which participants viewed six exemplars with the same general color in each of 40 different real-world categories, they completed a categorical search task, in which they searched for any member of a category based on a label cue. Targets that matched recent within-category regularities were found faster than targets that did not (Experiment 1). Such categorical cuing was also found to span multiple recent colors within a category (Experiment 2). It was observed to influence both the guidance of search to the target object (Experiment 3) and the basic operation of assigning single exemplars to categories (Experiment 4). Finally, the rapid acquisition of category-specific regularities was also quickly modified, with the benefit rapidly decreasing during the search session as participants were exposed equally to the two possible colors in each category. The results demonstrate that object categories organize the acquisition of perceptual regularities and that this learning exerts strong control over the instantiation of the category representation as a template for visual search. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

When more is more: redundant modifiers can facilitate visual search

According to the Gricean Maxim of Quantity, speakers provide the amount of information listeners require to correctly interpret an utterance, and no more (Grice in Logic and conversation, 1975). However, speakers do tend to violate the Maxim of Quantity often, especially when the redundant information improves reference precision (Degen et al. in Psychol Rev 127(4):591-621, 2020). Redundant (non-contrastive) information may facilitate real-world search if it narrows the spatial scope under consideration, or improves target template specificity. The current study investigated whether non-contrastive modifiers that improve reference precision facilitate visual search in real-world scenes. In two visual search experiments, we compared search performance when perceptually relevant, but non-contrastive modifiers were included in the search instruction. Participants (NExp. 1 = 48, NExp. 2 = 48) searched for a unique target object following a search instruction that contained either no modifier, a location modifier (Experiment 1: on the top left, Experiment 2: on the shelf), or a color modifier (the black lamp). In Experiment 1 only, the target was located faster when the verbal instruction included either modifier, and there was an overall benefit of color modifiers in a combined analysis for scenes and conditions common to both experiments. The results suggest that violations of the Maxim of Quantity can facilitate search when the violations include task-relevant information that either augments the target template or constrains the search space, and when at least one modifier provides a highly reliable cue. Consistent with Degen et al. (2020), we conclude that listeners benefit from non-contrastive information that improves reference precision, and engage in rational reference comprehension. SIGNIFICANCE STATEMENT: This study investigated whether providing more information than someone needs to find an object in a photograph helps them to find that object more easily, even though it means they need to interpret a more complicated sentence. Before searching a scene, participants were either given information about where the object would be located in the scene, what color the object was, or were only told what object to search for. The results showed that providing additional information helped participants locate an object in an image more easily only when at least one piece of information communicated what part of the scene the object was in, which suggests that more information can be beneficial as long as that information is specific and helps the recipient achieve a goal. We conclude that people will pay attention to redundant information when it supports their task. In practice, our results suggest that instructions in other contexts (e.g., real-world navigation, using a smartphone app, prescription instructions, etc.) can benefit from the inclusion of what appears to be redundant information.

Typicality modulates attentional capture by object categories

What we pay attention to in the visual environment is often driven by what we know about the world. For example, a number of studies have found that observers can adopt attentional sets for a particular semantic category. However, some objects are more typical members of a category than others. While previous evidence suggests that an object's typicality can influence the guidance of attention in visual search, it is unclear whether typicality can also influence the capture of attention. To test whether this is the case, participants were given a category of objects at the beginning of each trial. Then, a rapid serial visual presentation (RSVP) stream was presented at fixation, and participants had to indicate whether an object of the given category was present or absent from the stream. Importantly, a single flanker image also appeared above or below the central stream just before the target. This flanker could belong either to the same category as the target or a different category, and could be a typical or atypical exemplar of that category. Participants were less accurate at detecting the target when the flanker belonged to the same category as the target. Moreover, participants were even less accurate when the flanker was a typical exemplar of this category. Similar findings were observed when targets consisted of typical and atypical exemplars. Together, these findings indicate that the extent of attentional capture toward a distractor depends on whether the distractor matches the category and typicality of one's attentional set.

Typicality guides attention during categorical search, but not universally so

The degree to which an item is rated as being a typical member of its category influences an observer's ability to find that item during word-cued search. However, there are conflicting accounts as to whether or not typicality affects attentional guidance to categorical items, or whether it affects some other aspect of the search process. In this study, we employed word-cued search and eye tracking to disentangle typicality effects on attentional guidance and target verification across differing category cue specificities (i.e., superordinate or basic-level cues), while also varying the degree of similarity between targets and non-targets. We found that typicality influenced attentional guidance when searchers were cued at the superordinate level (e.g., clothing). When cues were provided at the basic level (e.g., pants), typicality did not influence attentional guidance, and only affected target verification when there was featural similarity between targets and non-targets. When a searcher uses a target template comprising features cued at the basic level, therefore, target/non-target similarity produces interference that affects attentional guidance, but we did not find evidence that it also affects target verification.

Modulating "Surprise" with Syntax: A Study on Negative Sentences and Eye-Movement Recording

This work focuses on a particular case of negative sentences, the Surprise Negation sentences (SNEGs). SNEGs belong to the class of expletive negation sentences, i.e., they are affirmative in meaning but involve a clausal negation. A clear example is offered by Italian: 'Enonmi è scesa dal treno Maria?!' (let. 'and not CLITIC.to_me is got off-the train Mary' = 'The surprise was that Maria got off the train!'). From a theoretical point of view, the interpretation of SNEGs as affirmative can be derived from their specific syntactic and semantic structure. Here we offer an implementation of the visual world paradigm to test how SNEGs are interpreted. Participants listened to affirmative, negative or expletive negative clauses while four objects (two relevant-either expected or unexpected-and two unrelated) were shown on the screen and their eye movements were recorded. Growth Curve Analysis showed that the fixation patterns to the relevant objects were very similar for affirmative and expletive negative sentences, while striking differences were observed between negative and affirmative sentences. These results showed that negation does play a different role in the mental representation of a sentence, depending on its syntactic derivation. Moreover, we also found that, compared to affirmative sentences, SNEGs require higher processing efforts due to both their syntactic complexity and pragmatic integration, with slower response time and lower accuracy.

Neural differences between chromatic- and luminance-driven attentional salience in visual search

Previous electroencephalographic research on attentional salience did not fully capture the complexities of low-level vision, which relies on both cone-opponent chromatic and cone-additive luminance mechanisms. We systematically varied color and luminance contrast using a visual search task for a higher contrast target to assess the degree to which the salience-computing attentional mechanisms are constrained by low-level visual inputs. In our first experiment, stimuli were defined by contrast that isolated chromatic or luminance mechanisms. In our second experiment, targets were defined by contrasts that isolated or combined achromatic and chromatic mechanisms. In both experiments, event-related potential waveforms contralateral and ipsilateral to the target were qualitatively different for chromatic- compared to luminance-defined stimuli. The same was true of the difference waves computed from these waveforms, with isoluminant stimuli eliciting a mid-latency posterior contralateral negativity (PCN) component and achromatic stimuli eliciting a complex of multiple components, including an early posterior contralateral positivity followed by a late-latency PCN. Combining color with luminance resulted in waveform and difference wave patterns equivalent to those of achromatic stimuli. When large levels of chromaticity contrast were added to targets with small levels of luminance contrast, PCN latency was speeded. In conclusion, the mechanisms underlying attentional salience are constrained by the low-level inputs they receive. Furthermore, speeded PCN latencies for stimuli that combine color and luminance signals compared to stimuli that contain luminance alone demonstrate that color and luminance channels are integrated during pre-attentive visual processing, before top-down allocation of attention is triggered.

Prediction and final temporal errors are used for trial-to-trial motor corrections

Many daily life situations (e.g. dodging an approaching object or hitting a moving target) require people to correct planning of future movements based on previous temporal errors. However, the actual temporal error can be difficult to perceive: imagine a baseball batter that swings and misses a fastball. Here we show that in such situations people can use an internal error signal to make corrections in the next trial. This signal is based on the discrepancy between the actual and the planned action onset time: the prediction error. In this study, we used three interception tasks: reaching movements, saccadic eye movements and a button press that released a cursor moving ballistically for a fixed time. We found that action onset depended on the previous temporal error in the arm movement experiment only and not in the saccadic and button press experiments. However, this dependency was modulated by the movement time: faster arm movements depended less on the previous actual temporal error. An analysis using a Kalman filter confirmed that people used the prediction error rather than the previous temporal error for trial-by-trial corrections in fast arm movements, saccades and button press.

Incidental memory following rapid object processing: The role of attention allocation strategies

When observers search for multiple (rather than singular) targets, they are slower and less accurate, yet have better incidental memory for nontarget items encountered during the task (Hout & Goldinger, 2010). One explanation for this may be that observers titrate their attention allocation based on the expected difficulty suggested by search cues. Difficult search cues may implicitly encourage observers to narrow their attention, simultaneously enhancing distractor encoding and hindering peripheral processing. Across three experiments, we manipulated the difficulty of search cues preceding passive visual search for real-world objects, using a Rapid Serial Visual Presentation (RSVP) task to equate item exposure durations. In all experiments, incidental memory was enhanced for distractors encountered while participants monitored for difficult targets. Moreover, in key trials, peripheral shapes appeared at varying eccentricities off center, allowing us to infer the spread and precision of participants' attentional windows. Peripheral item detection and identification decreased when search cues were difficult, even when the peripheral items appeared before targets. These results were not an artifact of sustained vigilance in miss trials, but instead reflect top-down modulation of attention allocation based on task demands. Implications for individual differences are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Whereof one cannot speak: How language and capture of visual attention interact

Our research addresses the important question whether language influences cognition by studying crosslinguistic differences in nonlinguistic visual search tasks. We investigated whether capture of visual attention is mediated by characteristics corresponding to concepts that are differently expressed across different languages. Korean grammatically distinguishes between tight- (kkita) and loose-fit (nehta) containment whereas German collapses them into a single semantic category (in). Although linguistic processing was neither instructed nor necessary to perform the visual search task, we found that Korean speakers showed attention capture by non-instructed but target-coincident (Experiment 1) or distractor-coincident (Experiments 4 and 5) spatial fitness of the stimuli, whereas German speakers were not sensitive to it. As the tight- versus loose-fit distinction is grammaticalized only in the Korean but not the German language, our results demonstrate that language influences which visual features capture attention even in non-linguistic tasks that do not require paying attention to these features. In separate control experiments (Experiments 2 and 3), we ruled out cultural or general cognitive group differences between Korean and German speaking participants as alternative explanations. We outline the mechanisms underlying these crosslinguistic differences in nonlinguistic visual search behaviors. This is the first study showing that linguistic spatial relational concepts held in long-term memory can affect attention capture in visual search tasks.

Contingent capture during search for alphanumerical characters: A case of feature-based capture or of conceptual category membership?

To distinguish if search for alphanumerical characters is based on features or on conceptual category membership, we conducted two experiments where we presented upright and inverted characters as cues in a contingent-capture protocol. Here, only cues matching the top-down search template (e.g., a letter cue when searching for target letters) capture attention and lead to validity effects: shorter search times and fewer errors for validly than invalidly cued targets. Top-down nonmatching cues (e.g., a number cue when searching for target letters) do not capture attention. To tell a feature-based explanation from one based on conceptual category membership, we used both upright (canonical) and inverted characters as cues. These cues share the same features, but inverted cues cannot be conceptually categorized as easily as upright cues. Thus, we expected no difference between upright and inverted cues when search is feature-based, whereas inverted cues would elicit no or at least considerably weaker validity effects if search relies on conceptual category membership. Altogether, the results of both experiments (with overlapping and with separate sets of characters for cues and targets) provide evidence for search based on feature representations, as among other things, significant validity effects were found with upright and inverted characters as cues. However, an influence of category membership was also evident, as validity effects of inverted characters were diminished.

Exploiting Typicality for Selecting Informative and Anomalous Samples in Videos

In this paper, we present a novel approach to find informative and anomalous samples in videos exploiting the concept of typicality from information theory. In most video analysis tasks, selection of the most informative samples from a huge pool of training data in order to learn a good recognition model is an important problem. Furthermore, it is also useful to reduce the annotation cost as it is time-consuming to annotate unlabeled samples. Typicality is a simple and powerful technique which can be applied to compress the training data to learn a good classification model. In a continuous video clip, an activity shares a strong correlation with its previous activities. We assume that the activity samples that appear in a video form a Markov chain. We explicitly show how typicality can be utilized in this scenario. We compute an atypical score for a sample using typicality and the Markovian property, which can be applied to two challenging vision problems-(a) sample selection for learning activity recognition models, and (b) anomaly detection. In the first case, our approach leads to a significant reduction of manual labeling cost while achieving similar or better recognition performance compared to a model trained with the entire training set. For the latter case, the atypical score has been exploited in identifying anomalous activities in videos where our results demonstrate the effectiveness of the proposed framework over other recent strategies.

Understanding Moment-to-Moment Processing of Visual Narratives

What role do moment‐to‐moment comprehension processes play in visual attentional selection in picture stories? The current work uniquely tested the role of bridging inference generation processes on eye movements while participants viewed picture stories. Specific components of the Scene Perception and Event Comprehension Theory (SPECT) were tested. Bridging inference generation was induced by manipulating the presence of highly inferable actions embedded in picture stories. When inferable actions are missing, participants have increased viewing times for the immediately following critical image (Magliano, Larson, Higgs, & Loschky, 2016). This study used eye‐tracking to test competing hypotheses about the increased viewing time: (a) Computational Load: inference generation processes increase overall computational load, producing longer fixation durations; (b) Visual Search: inference generation processes guide eye‐movements to pick up inference‐relevant information, producing more fixations. Participants had similar fixation durations, but they made more fixations while generating inferences, with that process starting from the fifth fixation. A follow‐up hypothesis predicted that when generating inferences, participants fixate scene regions important for generating the inference. A separate group of participants rated the inferential‐relevance of regions in the critical images, and results showed that these inferentially relevant regions predicted differences in other viewers’ eye movements. Thus, viewers’ event models in working memory affect visual attentional selection while viewing visual narratives.

Visual properties and memorising scenes: Effects of image-space sparseness and uniformity

Previous studies have demonstrated that humans have a remarkable capacity to memorise a large number of scenes. The research on memorability has shown that memory performance can be predicted by the content of an image. We explored how remembering an image is affected by the image properties within the context of the reference set, including the extent to which it is different from its neighbours (image-space sparseness) and if it belongs to the same category as its neighbours (uniformity). We used a reference set of 2,048 scenes (64 categories), evaluated pairwise scene similarity using deep features from a pretrained convolutional neural network (CNN), and calculated the image-space sparseness and uniformity for each image. We ran three memory experiments, varying the memory workload with experiment length and colour/greyscale presentation. We measured the sensitivity and criterion value changes as a function of image-space sparseness and uniformity. Across all three experiments, we found separate effects of 1) sparseness on memory sensitivity, and 2) uniformity on the recognition criterion. People better remembered (and correctly rejected) images that were more separated from others. People tended to make more false alarms and fewer miss errors in images from categorically uniform portions of the image-space. We propose that both image-space properties affect human decisions when recognising images. Additionally, we found that colour presentation did not yield better memory performance over grayscale images.

Look before you seek: Preview adds a fixed benefit to all searches

Knowing in advance what to look for speeds up search, but how this knowledge guides search is poorly understood. The little available evidence suggests that previewing the target produces larger reductions in search times for harder searches. To investigate this issue further, we performed seven experiments in which subjects searched for an oddball target after previewing the target, distracter, or an unrelated square. Consistent with previous studies, harder searches showed bigger reductions in search time for an informative preview. However, the same data replotted using the reciprocal of search time showed a remarkably different result: The informative preview showed a fixed additive increase in reciprocal search time across all searches regardless of difficulty. This is a nontrivial outcome because it cannot be explained using a simple relationship between search times in the informative and uninformative preview conditions. We interpret our findings by proposing that the reciprocal of search time reflects the strength of an underlying accumulating signal related to the distinctiveness or salience of the target over the distracters and that preview additively increases this signal for all searches. This in turn implies that the top-down signals related to target preview and bottom-up signals related to target-distracter salience sum linearly.

Rapid Parallel Attentional Selection Can Be Controlled by Shape and Alphanumerical Category

Previous research has shown that when two color-defined target objects appear in rapid succession at different locations, attention is deployed independently and in parallel to both targets. This study investigated whether this rapid simultaneous attentional target selection mechanism can also be employed in tasks where targets are defined by a different visual feature (shape) or when alphanumerical category is the target selection attribute. Two displays that both contained a target and a nontarget object on opposite sides were presented successively, and the SOA between the two displays was 100, 50, 20, or 10 msec in different blocks. N2pc components were recorded to both targets as a temporal marker of their attentional selection. When observers searched for shape-defined targets (Experiment 1), N2pc components to the two targets were equal in size and overlapped in time when the SOA between the two displays was short, reflecting two parallel shape-guided target selection processes with their own independent time course. Essentially the same temporal pattern of N2pc components was observed when alphanumerical category was the target-defining attribute (Experiment 2), demonstrating that the rapid parallel attentional selection of multiple target objects is not restricted to situations where the deployment of attention can be guided by elementary visual features but that these processes can even be employed in category-based attentional selection tasks. These findings have important implications for our understanding of the cognitive and neural basis of top–down attentional control.

Using space to represent categories: insights from gaze position

We investigated the boundaries among imagery, memory, and perception by measuring gaze during retrieved versus imagined visual information. Eye fixations during recall were bound to the location at which a specific stimulus was encoded. However, eye position information generalized to novel objects of the same category that had not been seen before. For example, encoding an image of a dog in a specific location enhanced the likelihood of looking at the same location during subsequent mental imagery of other mammals. The results suggest that eye movements can also be launched by abstract representations of categories and not exclusively by a single episode or a specific visual exemplar.

Searching for Category-Consistent Features: A Computational Approach to Understanding Visual Category Representation

This article introduces a generative model of category representation that uses computer vision methods to extract category-consistent features (CCFs) directly from images of category exemplars. The model was trained on 4,800 images of common objects, and CCFs were obtained for 68 categories spanning subordinate, basic, and superordinate levels in a category hierarchy. When participants searched for these same categories, targets cued at the subordinate level were preferentially fixated, but fixated targets were verified faster when they followed a basic-level cue. The subordinate-level advantage in guidance is explained by the number of target-category CCFs, a measure of category specificity that decreases with movement up the category hierarchy. The basic-level advantage in verification is explained by multiplying the number of CCFs by sibling distance, a measure of category distinctiveness. With this model, the visual representations of real-world object categories, each learned from the vast numbers of image exemplars accumulated throughout everyday experience, can finally be studied.

On the Hunt: Searching for Poorly Defined Camouflaged Targets

As camouflaged targets share visual characteristics with the environment within which they are embedded, searchers rarely have access to a perfect visual template of such targets. Instead, they must rely on less specific representations to guide search. Although search for camouflaged and non-specified targets have both received attention in the literature, to date they have not been explored in a combined context. Here we introduce a new paradigm for characterizing behavior during search for camouflaged targets in natural scenes, while also exploring how the fidelity of the target template affects search processes. Search scenes were created from forest images, with targets a distortion (varied size) of that image at a random location. In Experiment 1 a preview of the target was provided; in Experiment 2 there was no preview. No differences were found between experiments on nearly all measures. Generally, reaction times and accuracy improved with familiarity on the task (more so for small targets). Analysis of eye movements indicated that performance benefits were related to improvements in both Search and Target Verification time. Combined, our data suggest that search for camouflaged targets can be improved over a short time-scale, even when targets are poorly defined.

The anterior temporal cortex is a primary semantic source of top-down influences on object recognition

Perception emerges from a dynamic interplay between feed-forward sensory input and feedback modulation along the cascade of neural processing. Prior knowledge, a major form of top-down modulatory signal, benefits perception by enabling efficacious inference and resolving ambiguity, particularly under circumstances of degraded visual input. Despite semantic information being a potentially critical source of this top-down influence, to date, the core neural substrate of semantic knowledge (the anterolateral temporal lobe – ATL) has not been considered as a key component of the feedback system. Here we provide direct evidence of its significance for visual cognition – the ATL underpins the semantic aspect of object recognition, amalgamating sensory-based (amount of accumulated sensory input) and semantic-based (representational proximity between exemplars and typicality of appearance) influences. Using transcranial theta-burst stimulation combined with a novel visual identification paradigm, we demonstrate that the left ATL contributes to discrimination between visual objects. Crucially, its contribution is especially vital under situations where semantic knowledge is most needed for supplementing deficiency of input (brief visual exposure), discerning analogously-coded exemplars (close representational distance), and resolving discordance (target appearance violating the statistical typicality of its category). Our findings characterise functional properties of the ATL in object recognition: this neural structure is summoned to augment the visual system when the latter is overtaxed by challenging conditions (insufficient input, overlapped neural coding, and conflict between incoming signal and expected configuration). This suggests a need to revisit current theories of object recognition, incorporating the ATL that interfaces high-level vision with semantic knowledge.

Using multidimensional scaling to quantify similarity in visual search and beyond

Visual search is one of the most widely studied topics in vision science, both as an independent topic of interest, and as a tool for studying attention and visual cognition. A wide literature exists that seeks to understand how people find things under varying conditions of difficulty and complexity, and in situations ranging from the mundane (e.g., looking for one's keys) to those with significant societal importance (e.g., baggage or medical screening). A primary determinant of the ease and probability of success during search are the similarity relationships that exist in the search environment, such as the similarity between the background and the target, or the likeness of the non-targets to one another. A sense of similarity is often intuitive, but it is seldom quantified directly. This presents a problem in that similarity relationships are imprecisely specified, limiting the capacity of the researcher to examine adequately their influence. In this article, we present a novel approach to overcoming this problem that combines multi-dimensional scaling (MDS) analyses with behavioral and eye-tracking measurements. We propose a method whereby MDS can be repurposed to successfully quantify the similarity of experimental stimuli, thereby opening up theoretical questions in visual search and attention that cannot currently be addressed. These quantifications, in conjunction with behavioral and oculomotor measures, allow for critical observations about how similarity affects performance, information selection, and information processing. We provide a demonstration and tutorial of the approach, identify documented examples of its use, discuss how complementary computer vision methods could also be adopted, and close with a discussion of potential avenues for future application of this technique.

Components of Attention in Grapheme-Color Synesthesia: A Modeling Approach

Grapheme-color synesthesia is a condition where the perception of graphemes consistently and automatically evokes an experience of non-physical color. Many have studied how synesthesia affects the processing of achromatic graphemes, but less is known about the synesthetic processing of physically colored graphemes. Here, we investigated how the visual processing of colored letters is affected by the congruence or incongruence of synesthetic grapheme-color associations. We briefly presented graphemes (10-150 ms) to 9 grapheme-color synesthetes and to 9 control observers. Their task was to report as many letters (targets) as possible, while ignoring digit (distractors). Graphemes were either congruently or incongruently colored with the synesthetes' reported grapheme-color association. A mathematical model, based on Bundesen's (1990) Theory of Visual Attention (TVA), was fitted to each observer's data, allowing us to estimate discrete components of visual attention. The models suggested that the synesthetes processed congruent letters faster than incongruent ones, and that they were able to retain more congruent letters in visual short-term memory, while the control group's model parameters were not significantly affected by congruence. The increase in processing speed, when synesthetes process congruent letters, suggests that synesthesia affects the processing of letters at a perceptual level. To account for the benefit in processing speed, we propose that synesthetic associations become integrated into the categories of graphemes, and that letter colors are considered as evidence for making certain perceptual categorizations in the visual system. We also propose that enhanced visual short-term memory capacity for congruently colored graphemes can be explained by the synesthetes' expertise regarding their specific grapheme-color associations.

Activation of new attentional templates for real-world objects in visual search

Visual search is controlled by representations of target objects (attentional templates). Such templates are often activated in response to verbal descriptions of search targets, but it is unclear whether search can be guided effectively by such verbal cues. We measured ERPs to track the activation of attentional templates for new target objects defined by word cues. On each trial run, a word cue was followed by three search displays that contained the cued target object among three distractors. Targets were detected more slowly in the first display of each trial run, and the N2pc component (an ERP marker of attentional target selection) was attenuated and delayed for the first relative to the two successive presentations of a particular target object, demonstrating limitations in the ability of word cues to activate effective attentional templates. N2pc components to target objects in the first display were strongly affected by differences in object imageability (i.e., the ability of word cues to activate a target-matching visual representation). These differences were no longer present for the second presentation of the same target objects, indicating that a single perceptual encounter is sufficient to activate a precise attentional template. Our results demonstrate the superiority of visual over verbal target specifications in the control of visual search, highlight the fact that verbal descriptions are more effective for some objects than others, and suggest that the attentional templates that guide search for particular real-world target objects are analog visual representations.