Attentional Guidance and Match Decisions Rely on Different Template Information During Visual Search

Bayesian multi-level modelling for predicting single and double feature visual search

Performance in visual search tasks is frequently summarised by "search slopes" - the additional cost in reaction time for each additional distractor. While search tasks with a shallow search slopes are termed efficient (pop-out, parallel, feature), there is no clear dichotomy between efficient and inefficient (serial, conjunction) search. Indeed, a range of search slopes are observed in empirical data. The Target Contrast Signal (TCS) Theory is a rare example of quantitative model that attempts to predict search slopes for efficient visual search. One study using the TCS framework has shown that the search slope in a double-feature search (where the target differs in both colour and shape from the distractors) can be estimated from the slopes of the associated single-feature searches. This estimation is done using a contrast combination model, and a collinear contrast integration model was shown to outperform other options. In our work, we extend TCS to a Bayesian multi-level framework. We investigate modelling using normal and shifted-lognormal distributions, and show that the latter allows for a better fit to previously published data. We run a new fully within-subjects experiment to attempt to replicate the key original findings, and show that overall, TCS does a good job of predicting the data. However, we do not replicate the finding that the collinear combination model outperforms the other contrast combination models, instead finding that it may be difficult to conclusively distinguish between them.

Task-adaptive changes to the target template in response to distractor context: Separability versus similarity

Theories of attention hypothesize the existence of an attentional template that contains target features in working or long-term memory. It is frequently assumed that the template contains a veridical copy of the target, but recent studies suggest that this is not true when the distractors are linearly separable from the target. In such cases, target representations shift "off-veridical" in response to the distractor context, presumably because doing so is adaptive and increases the representational distinctiveness of targets from distractors. However, some have argued that the shifts may be entirely explained by perceptual biases created by simultaneous color contrast. Here we address this debate and test the more general hypothesis that the target template is adaptively shaped by elements of the distractor context needed to distinguish targets from distractors. We used a two-dimensional target and separately manipulated the linear separability of one dimension (color) and the visual similarity of the other (orientation). We found that target shifting along the linearly separable color dimension was dependent on the similarity of targets-to-distractors along the other dimension. The target representations were consistent with a postexperiment strategy questionnaire in which participants reported using color more when orientation was hard to use, and orientation more when it was easier to use. We conclude that the target template is task-adaptive and exploit features in the distractor context that most predictably distinguish targets from distractors to increase visual search efficiency. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Learned associations serve as target proxies during difficult but not easy visual search

The target template contains information in memory that is used to guide attention during visual search and is typically thought of as containing features of the actual target object. However, when targets are hard to find, it is advantageous to use other information in the visual environment that is predictive of the target's location to help guide attention. The purpose of these studies was to test if newly learned associations between face and scene category images lead observers to use scene information as a proxy for the face target. Our results showed that scene information was used as a proxy for the target to guide attention but only when the target face was difficult to discriminate from the distractor face; when the faces were easy to distinguish, attention was no longer guided by the scene unless the scene was presented earlier. The results suggest that attention is flexibly guided by both target features as well as features of objects that are predictive of the target location. The degree to which each contributes to guiding attention depends on the efficiency with which that information can be used to decode the location of the target in the current moment. The results contribute to the view that attentional guidance is highly flexible in its use of information to rapidly locate the target.

Little engagement of attention by salient distractors defined in a different dimension or modality to the visual search target

Singleton distractors may inadvertently capture attention, interfering with the task at hand. The underlying neural mechanisms of how we prevent or handle distractor interference remain elusive. Here, we varied the type of salient distractor introduced in a visual search task: the distractor could be defined in the same (shape) dimension as the target, a different (color) dimension, or a different (tactile) modality (intra-dimensional, cross-dimensional, and, respectively, cross-modal distractor, all matched for physical salience); and besides behavioral interference, we measured lateralized electrophysiological indicators of attentional selectivity (the N2pc, Ppc, PD , CCN/CCP, CDA, and cCDA). The results revealed the intra-dimensional distractor to produce the strongest reaction-time interference, associated with the smallest target-elicited N2pc. In contrast, the cross-dimensional and cross-modal distractors did not engender any significant interference, and the target-elicited N2pc was comparable to the condition in which the search display contained only the target singleton, thus ruling out early attentional capture. Moreover, the cross-modal distractor elicited a significant early CCN/CCP, but did not influence the target-elicited N2pc, suggesting that the tactile distractor is registered by the somatosensory system (rather than being proactively suppressed), without, however, engaging attention. Together, our findings indicate that, in contrast to distractors defined in the same dimension as the target, distractors singled out in a different dimension or modality can be effectively prevented to engage attention, consistent with dimension- or modality-weighting accounts of attentional priority computation.

Tuning to non-veridical features in attention and perceptual decision-making: An EEG study

When searching for a lost item, we tune attention to the known properties of the object. Previously, it was believed that attention is tuned to the veridical attributes of the search target (e.g., orange), or an attribute that is slightly shifted away from irrelevant features towards a value that can more optimally distinguish the target from the distractors (e.g., red-orange; optimal tuning). However, recent studies showed that attention is often tuned to the relative feature of the search target (e.g., redder), so that all items that match the relative features of the target equally attract attention (e.g., all redder items; relational account). Optimal tuning was shown to occur only at a later stage of identifying the target. However, the evidence for this division mainly relied on eye tracking studies that assessed the first eye movements. The present study tested whether this division can also be observed when the task is completed with covert attention and without moving the eyes. We used the N2pc in the EEG of participants to assess covert attention, and found comparable results: Attention was initially tuned to the relative colour of the target, as shown by a significantly larger N2pc to relatively matching distractors than a target-coloured distractor. However, in the response accuracies, a slightly shifted, "optimal" distractor interfered most strongly with target identification. These results confirm that early (covert) attention is tuned to the relative properties of an item, in line with the relational account, while later decision-making processes may be biased to optimal features.

Testing a relational account of search templates in visual foraging

Search templates guide human visual attention toward relevant targets. Templates are often seen as encoding exact target features, but recent studies suggest that templates rather contain "relational properties" (e.g., they facilitate "redder" stimuli instead of specific hues of red). Such relational guidance seems helpful in naturalistic searches where illumination or perspective renders exact feature values unreliable. So far relational guidance has only been demonstrated in rather artificial single-target search tasks with briefly flashed displays. Here, we investigate whether relational guidance also occurs when humans interact with the search environment for longer durations to collect multiple target elements. In a visual foraging task, participants searched for and collected multiple targets among distractors of different relationships to the target colour. Distractors whose colour differed from the environment in the same direction as the targets reduced foraging efficiency to the same amount as distractors whose colour matched the target colour. Distractors that differed by the same colour distance but in the opposite direction of the target colour did not reduce efficiency. These findings provide evidence that search templates encode relational target features in naturalistic search tasks and suggest that attention guidance based on relational features is a common mode in dynamic, real-world search environments.

Attentional selection is a sufficient cause for visual working memory interference

Visual attention and visual working memory (VWM) are intertwined processes that allow navigation of the visual world. These systems can compete for highly limited cognitive resources, creating interference effects when both operate in tandem. Performing an attentional task while maintaining a VWM load often leads to a loss of memory information. These losses are seen even with very simple visual search tasks. Previous research has argued that this may be due to the attentional selection process, of choosing the target item out of surrounding nontarget items. Over two experiments, the current study disentangles the roles of search and selection in visual search and their influence on a retained VWM load. Experiment 1 revealed that, when search stimuli were relatively simple, target-absent searches (which did not require attentional selection) did not provoke memory interference, whereas target-present search did. In Experiment 2, the number of potential targets was varied in the search displays. In one condition, participants were required to select any one of the items displayed, requiring an attentional selection but no need to search for a specific item. Importantly, this condition led to memory interference to the same extent as a condition where a single target was presented among nontargets. Together, these results show that the process of attentional selection is a sufficient cause for interference with a concurrently maintained VWM load.

Good-enough attentional guidance

Theories of attention posit that attentional guidance operates on information held in a target template within memory. The template is often thought to contain veridical target features, akin to a photograph, and to guide attention to objects that match the exact target features. However, recent evidence suggests that attentional guidance is highly flexible and often guided by non-veridical features, a subset of features, or only associated features. We integrate these findings and propose that attentional guidance maximizes search efficiency based on a 'good-enough' principle to rapidly localize candidate target objects. Candidates are then serially interrogated to make target-match decisions using more precise information. We suggest that good-enough guidance optimizes the speed-accuracy-effort trade-offs inherent in each stage of visual search.

Which processes dominate visual search: Bottom-up feature contrast, top-down tuning or trial history?

Previous research has identified three mechanisms that guide visual attention: bottom-up feature contrasts, top-down tuning, and the trial history (e.g., priming effects). However, only few studies have simultaneously examined all three mechanisms. Hence, it is currently unclear how they interact or which mechanisms dominate over others. With respect to local feature contrasts, it has been claimed that a pop-out target can only be selected immediately in dense displays when the target has a high local feature contrast, but not when the displays are sparse, which leads to an inverse set-size effect. The present study critically evaluated this view by systematically varying local feature contrasts (i.e., set size), top-down knowledge, and the trial history in pop-out search. We used eye tracking to distinguish between early selection and later identification-related processes. The results revealed that early visual selection was mainly dominated by top-down knowledge and the trial history: When attention was biased to the target feature, either by valid pre-cueing (top-down) or automatic priming, the target could be localised immediately, regardless of display density. Bottom-up feature contrasts only modulated selection when the target was unknown and attention was biased to the non-targets. We also replicated the often-reported finding of reliable feature contrast effects in the mean RTs, but showed that these were due to later, target identification processes (e.g., in the target dwell times). Thus, contrary to the prevalent view, bottom-up feature contrasts in dense displays do not seem to directly guide attention, but only facilitate nontarget rejection, probably by facilitating nontarget grouping.

Mirror blindness: Our failure to recognize the target in search for mirror-reversed shapes

It is well known that visual search for a mirror target (i.e., a horizontally flipped item) is more difficult than search for other-oriented items (e.g., vertically flipped items). Previous studies have typically attributed costs of mirror search to early, attention-guiding processes but could not rule out contributions from later processes. In the present study we used eye tracking to distinguish between early, attention-guiding processes and later target identification processes. The results of four experiments revealed a marked human weakness in identifying mirror targets: Observers appear to frequently fail to classify a mirror target as a target on first fixation and to continue with search even after having directly looked at the target. Awareness measures corroborated that the location of a mirror target could not be reported above chance level after it had been fixated once. This mirror blindness effect explained a large proportion (45-87%) of the overall costs of mirror search, suggesting that part of the difficulties with mirror search are rooted in later, object identification processes (not attentional guidance). Mirror blindness was significantly reduced but not completely eliminated when both the target and non-targets were held constant, which shows that perfect top-down knowledge can reduce mirror blindness, without completely eliminating it. The finding that non-target certainty reduced mirror blindness suggests that object identification is not solely achieved by comparing a selected item to a target template. These results demonstrate that templates that guide search toward targets are not identical to the templates used to conclusively identify those targets.

Negative and positive templates: Two forms of cued attentional control

Our ability to control our attention to focus on goal-relevant information is critical for functioning in daily life. In addition to the typical attentional control driven by target enhancement described in most theories of attention, recent research has focused on our ability to use information about distractions maintained in working memory to direct our attention away from known distractors. Using these negative templates can improve the efficiency of attention, much in the same way as enhancing information matching search targets. However, these effects only occur for specific tasks or in specific circumstances. In this review, I will focus on our emerging understanding of the relationship between distractor ignoring from negative templates and target enhancement from positive templates. I will also highlight key remaining questions for further study.

Attentional priority is determined by predicted feature distributions

Visual attention is often characterized as being guided by precise memories for target objects. However, real-world search targets have dynamic features that vary over time, meaning that observers must predict how the target could look based on how features are expected to change. Despite its importance, little is known about how target feature predictions influence feature-based attention, or how these predictions are represented in the target template. In Experiment 1 (N = 60 university students), we show observers readily track the statistics of target features over time and adapt attentional priority to predictions about the distribution of target features. In Experiments 2a and 2b (N = 480 university students), we show that these predictions are encoded into the target template as a distribution of likelihoods over possible target features, which are independent of memory precision for the cued item. These results provide a novel demonstration of how observers represent predicted feature distributions when target features are uncertain and show that these predictions are used to set attentional priority during visual search. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Benefits from negative templates in easy and difficult search depend on rapid distractor rejection and enhanced guidance

While most research on attentional templates focuses on how attention can be guided toward targets, a growing body of research has been examining our ability to guide attention away from distractors using negative templates. A recent report showed larger reaction time benefits from negative templates during a difficult search task compared to an easy search task. However, it remains unclear what shifts in attentional processing led to these differences in reaction time benefits. In the current study, we tested the predictions of enhanced guidance and rapid rejection as potential explanations for the differential impact of negative templates based on search difficulty. In Experiment 1, we replicated the larger benefits from negative templates in a within-subjects design. In Experiment 2, we use eye tracking to measure the proportion of fixations directed towards target-colored distractors as an index of enhanced guidance and the dwell time on distractors as a measure of rapid rejection. We found more attentional guidance from negative templates in the difficult search condition. In addition, we found larger benefits of rapid rejection from negative templates in the difficult search condition. While we typically focus on templates as a way of guiding attention, these results highlight another key role played by attentional templates: rapid distractor rejection.

Visual search guidance uses coarser template information than target-match decisions

When searching for an object, we use a target template in memory that contains task-relevant information to guide visual attention to potential targets and to determine the identity of attended objects. These processes in visual search have typically been assumed to rely on a common source of template information. However, our recent work (Yu et al., 2022) argued that attentional guidance and target-match decisions rely on different information during search, with guidance using a “fuzzier” version of the template compared with target decisions. However, that work was based on the special case of search for a target amongst linearly separable distractors (e.g., search for an orange target amongst yellower distractors). Real-world search targets, however, are infrequently linearly separable from distractors, and it remains unclear whether the differences between the precision of template information used for guidance compared with target decisions also applies under more typical conditions. In four experiments, we tested this question by varying distractor similarity during visual search and measuring the likelihood of attentional guidance to distractors and target misidentifications. We found that early attentional guidance is indeed less precise than that of subsequent match decisions under varying exposure durations and distractor set sizes. These results suggest that attentional guidance operates on a coarser code than decisions, perhaps because guidance is constrained by lower acuity in peripheral vision or the need to rapidly explore a wide region of space while decisions about selected objects are more precise to optimize decision accuracy.