A signal detection model predicts the effects of set size on visual search accuracy for feature, conjunction, triple conjunction, and disjunction displays

ATLAS: Mapping ATtention's Location And Size to probe five modes of serial and parallel search

Conventional visual search tasks do not address attention directly and their core manipulation of 'set size' - the number of displayed items - introduces stimulus confounds that hinder interpretation. However, alternative approaches have not been widely adopted, perhaps reflecting their complexity, assumptions, or indirect attention-sampling. Here, a new procedure, the ATtention Location And Size ('ATLAS') task used probe displays to track attention's location, breadth, and guidance during search. Though most probe displays comprised six items, participants reported only the single item they judged themselves to have perceived most clearly - indexing the attention 'peak'. By sampling peaks across variable 'choice sets', the size and position of the attention window during search was profiled. These indices appeared to distinguish narrow- from broad attention, signalled attention to pairs of items where it arose and tracked evolving attention-guidance over time. ATLAS is designed to discriminate five key search modes: serial-unguided, sequential-guided, unguided attention to 'clumps' with local guidance, and broad parallel-attention with or without guidance. This initial investigation used only an example set of highly regular stimuli, but its broader potential should be investigated.

Multiple visual items can be simultaneously compared with target templates in memory

When we search for something, we often rely on both what we see and what we remember. This process can be divided into three stages: selecting items, identifying those items, and comparing them with what we are trying to find in our memory. It has been suggested that we select items one by one, and we can identify several items at once. In the present study, we tested whether we need to finish comparing a selected item in the visual display with one or more target templates in memory before we can move on to the next selected item. In Experiment 1, observers looked for either one or two target types in a rapid serially presented stimuli stream. The time interval between the presentation onset of successive items in the stream was varied to get a threshold. For search for one target, the threshold was 89 ms. When look for either of two targets, it was 192 ms. This threshold difference offered a baseline. In Experiment 2, observers looked for one or two types of target in a search array. If they compared each identified item separately, we should expect a jump in the slope of the RT × Set Size function, on the order of the baseline obtained in Experiment 1. However, the slope difference was only 13 ms/item, suggesting that several identified items can be compared at once with target templates in memory. Experiment 3 showed that this slope difference was not just a memory-load cost.

Effects of target-amplitude and background-contrast uncertainty predicted by a normalized template-matching observer

When detecting targets under natural conditions, the visual system almost always faces multiple, simultaneous, dimensions of extrinsic uncertainty. This study focused on the simultaneous uncertainty about target amplitude and background contrast. These dimensions have a large effect on detection and vary greatly in natural scenes. We measured the human performance for detecting a sine-wave target in white noise and natural-scene backgrounds for two levels of prior probability of the target being present. We derived and tested the ideal observer for white-noise backgrounds, a special case of a template-matching observer that dynamically moves its criterion with the background contrast (the DTM observer) and two simpler models with a fixed criterion: the template-matching (TM) observer and the normalized template-matching (NTM) observer that normalizes template response by background contrast. Simulations show that, when the target prior is low, the performance of the NTM observer is near optimal and the TM observer is near chance, suggesting that manipulating the target prior is valuable for distinguishing among models. Surprisingly, we found that the NTM and DTM observers better explain human performance than the TM observer for both target priors in both background types. We argue that the visual system most likely exploits contrast normalization, rather than dynamic criterion adjustment, to deal with simultaneous background contrast and target amplitude uncertainty. Finally, our findings show that the data collected under high levels of uncertainty have a rich structure capable of discriminating between models, providing an alternative approach for studying high dimensions of uncertainty.

How do (perceptual) distracters distract?

When a target stimulus occurs in the presence of distracters, decisions are less accurate. But how exactly do distracters affect choices? Here, we explored this question using measurement of human behaviour, psychophysical reverse correlation and computational modelling. We contrasted two models: one in which targets and distracters had independent influence on choices (independent model) and one in which distracters modulated choices in a way that depended on their similarity to the target (interaction model). Across three experiments, participants were asked to make fine orientation judgments about the tilt of a target grating presented adjacent to an irrelevant distracter. We found strong evidence for the interaction model, in that decisions were more sensitive when target and distracter were consistent relative to when they were inconsistent. This consistency bias occurred in the frame of reference of the decision, that is, it operated on decision values rather than on sensory signals, and surprisingly, it was independent of spatial attention. A normalization framework, where target features are normalized by the expectation and variability of the local context, successfully captures the observed pattern of results.

In the real world, visual scenes usually contain many objects. As a consequence, we often have to make perceptual judgments about a specific ‘target’ stimulus in the presence of irrelevant ‘distracter’ stimuli. For instance, when hanging a picture frame, we want to discern whether it is hanging straight, ignoring the surrounding, potentially tilted frames. Laboratory experiments have shown that the presence of distracter stimuli (i.e. other picture frames) makes this type of perceptual judgment less accurate. However, the specific effect distracters have on judgments is controversial. Here, we conducted a series of experiments to compare two alternative theories of distracter influence: one in which distracters compete with targets to determine choices (independent model) and one in which distracters wield a more indirect influence on choices (interaction model). We found evidence for the latter account. Our results suggest distracters affect perceptual decisions by adjusting how sensitive decisions are to the target stimulus.

Temporal integration of feature probability distributions

Humans are surprisingly good at learning the statistical characteristics of their visual environment. Recent studies have revealed that not only can the visual system learn repeated features of visual search distractors, but also their actual probability distributions. Search times were determined by the frequency of distractor features over consecutive search trials. The search displays applied in these studies involved many exemplars of distractors on each trial and while there is clear evidence that feature distributions can be learned from large distractor sets, it is less clear if distributions are well learned for single targets presented on each trial. Here, we investigated potential learning of probability distributions of single targets during visual search. Over blocks of trials, observers searched for an oddly colored target that was drawn from either a Gaussian or a uniform distribution. Search times for the different target colors were clearly influenced by the probability of that feature within trial blocks. The same search targets, coming from the extremes of the two distributions were found significantly slower during the blocks where the targets were drawn from a Gaussian distribution than from a uniform distribution indicating that observers were sensitive to the target probability determined by the distribution shape. In Experiment 2, we replicated the effect using binned distributions and revealed the limitations of encoding complex target distributions. Our results demonstrate detailed internal representations of target feature distributions and that the visual system integrates probability distributions of target colors over surprisingly long trial sequences.

The Redesign of a Checklist for Evaluating Driver Impairment: A Human Factors and Ergonomics Approach

The Cantonal Police of Zurich, Switzerland, use a checklist to identify impaired drivers when conducting traffic stops. This checklist was developed by subject-matter experts and has been in use for eight years. The goal of this study was to redesign the checklist while considering human factors and ergonomics principles in combination with findings from a retrospective analysis of a set of 593 completed checklists. The checklist was amended in accordance with the results of the retrospective analysis by adding missing items and discarding superfluous ones. In addition, a hierarchical cluster analysis of the retrospective data suggested an improved spatial organization of checklist elements and the grouping of similar items of the checklist. Furthermore, aspects related to Fitts’s law, visual complexity, and an optimized direction of processing the checklist underpinned the design process. The results of an evaluation of the redesigned checklist by 11 laypeople and 13 police officers indicated an improved usability of the redesigned checklist over the original.

Could simplified stimuli change how the brain performs visual search tasks? A deep neural network study

Visual search is a complex behavior influenced by many factors. To control for these factors, many studies use highly simplified stimuli. However, the statistics of these stimuli are very different from the statistics of the natural images that the human visual system is optimized by evolution and experience to perceive. Could this difference change search behavior? If so, simplified stimuli may contribute to effects typically attributed to cognitive processes, such as selective attention. Here we use deep neural networks to test how optimizing models for the statistics of one distribution of images constrains performance on a task using images from a different distribution. We train four deep neural network architectures on one of three source datasets—natural images, faces, and x-ray images—and then adapt them to a visual search task using simplified stimuli. This adaptation produces models that exhibit performance limitations similar to humans, whereas models trained on the search task alone exhibit no such limitations. However, we also find that deep neural networks trained to classify natural images exhibit similar limitations when adapted to a search task that uses a different set of natural images. Therefore, the distribution of data alone cannot explain this effect. We discuss how future work might integrate an optimization-based approach into existing models of visual search behavior.

Behavior in a visual search task with moving dot stimuli

Understanding the neuronal mechanisms underlying the processing of visual attention requires a well-designed behavioral task that allows investigators to clearly describe the behavioral effects of attention. Here, we introduce a behavioral paradigm in which one, two or four moving dot stimuli are used in a visual search paradigm that includes two additional attentional conditions. Two animals were trained to make a saccade to a target (a dot patch with net rightward motion) and hold central fixation if no target was present. To implement covert visual attention, we included trials in which a 100% valid spatial cue appeared and trials in which the color of the fixation point indicated, with 100% validity, which of four colored dot patches the animals should attend to. We analyzed the behavior in terms of reaction times and signal detection theory metrics d-prime (representing sensitivity) and criteria. In both animals, we found that reaction times were greater for larger set-sizes and that the introduction of an attentional cue reduced the reaction times substantially. We also found that both animals showed increases in criteria, but no change in sensitivity, as set-size increased and the attentional cues led to an increase in sensitivity, with only some change in criteria. Our results illustrate how the animals perform this task and imply that both animals chose similar strategies. Importantly, this will allow future neurophysiological studies to probe not only the effects of attention, but will give the possibility of seeing whether different neural mechanisms drive changes in criteria and d-prime.

Rhythmic sampling revisited: Experimental paradigms and neural mechanisms

Sampling of information is thought to be an important aspect of explorative behaviour. Evidence for it has been gained in behavioural assessments of a variety of overt and covert cognitive domains, including sensation, attention, memory, eye movements and dexterity. A common aspect across many findings is that sampling tends to exhibit a rhythmicity at low frequencies (theta, 4-8 Hz; alpha, 9-12 Hz). Neurophysiological investigations in a wide range of species, including rodents, non-human primates and humans have demonstrated the presence of sampling related neural oscillations in a number of brain areas ranging from early sensory cortex, hippocampus to high-level cognitive areas. However, to assess whether rhythmic sampling represents a general aspect of exploratory behaviour one must critically evaluate the task parameters, and their potential link with neural oscillations. Here we focus on sampling during attentive vision to present an overview on the experimental conditions that are used to investigate rhythmic sampling and associated oscillatory brain activity in this domain. This review aims to (1) provide guidelines to efficiently quantify behavioural rhythms, (2) compare results from human and non-human primate studies and (3) argue that the underlying neural mechanisms of sampling can co-occur in both sensory and high-level areas.

Optimal attentional allocation in the presence of capacity constraints in uncued and cued visual search

The vision sciences literature contains a large diversity of experimental and theoretical approaches to the study of visual attention. We argue that this diversity arises, at least in part, from the field's inability to unify differing theoretical perspectives. In particular, the field has been hindered by a lack of a principled formal framework for simultaneously thinking about both optimal attentional processing and capacity-limited attentional processing, where capacity is limited in a general, task-independent manner. Here, we supply such a framework based on rate-distortion theory (RDT) and optimal lossy compression. Our approach defines Bayes-optimal performance when an upper limit on information processing rate is imposed. In this article, we compare Bayesian and RDT accounts in both uncued and cued visual search tasks. We start by highlighting a typical shortcoming of unlimited-capacity Bayesian models that is not shared by RDT models, namely, that they often overestimate task performance when information-processing demands are increased. Next, we reexamine data from two cued-search experiments that have previously been modeled as the result of unlimited-capacity Bayesian inference and demonstrate that they can just as easily be explained as the result of optimal lossy compression. To model cued visual search, we introduce the concept of a "conditional communication channel." This simple extension generalizes the lossy-compression framework such that it can, in principle, predict optimal attentional-shift behavior in any kind of perceptual task, even when inputs to the model are raw sensory data such as image pixels. To demonstrate this idea's viability, we compare our idealized model of cued search, which operates on a simplified abstraction of the stimulus, to a deep neural network version that performs approximately optimal lossy compression on the real (pixel-level) experimental stimuli.

Serial, self-terminating search can be distinguished from others: Evidence from multi-target search data

How do people find a target among multiple stimuli? The process of searching for a target among distractors has been a fundamental issue in human perception and cognition, evoking raging debates. Some researchers argued that search should be carried out by serially allocating focal attention to each item until the target is found. Others claimed that multiple stimuli, sharing a finite amount of processing resource, could be processed in parallel. This strict serial/parallel dichotomy in visual search has been challenged and many recent theories suggest that visual search tasks involve both serial and parallel processes. However, some search tasks should primarily depend on serial processing, while others would rely upon parallel processing to a greater extent. Here, by simple innovation of an experimental paradigm, we were able to identify a specific behavioral pattern associated with serial, self-terminating search and clarified which tasks depend on serial processing to a greater extent than others. Using this paradigm, we provide insights regarding under which condition the search becomes more serial or parallel. We also discuss several recent models of visual search that are capable of accommodating these findings and reconciling the extant controversy.

Visual attention in spatial cueing and visual search

To characterize internal processes of an observer conducting perceptual tasks, we developed an observer model that combines the perceptual template model (PTM), the attention mechanisms in the PTM framework (Lu & Dosher, 1998), and uncertainty of signal detection theory (Green & Swets, 1966). The model was evaluated with a visual search experiment conducted in a range of external noise, signal contrast, and target-distractor similarity conditions. In each trial, eight Gabor patches were shown in each of two brief intervals, with one target at a different orientation from the distractors in one of the presentations. Subjects were precued to a subset of the stimuli (1, 2, 4, or 8) and asked to report (a) which interval contained the target and (b) where the target was. Individual roles of uncertainty and of attention in visual search were investigated by comparing models with and without an attention component. The results showed that decision uncertainty alone was sufficient to account for the set-size effect, even in conditions with high target-distractor similarity. Our theoretical model and empirical results provide a coherent picture regarding how visual information is selected and processed during feature search.

Response of multiple demand network to visual search demands

Neuroimaging studies for human participants have shown that the activity in the multiple-demand (MD) network is associated with various kinds of cognitive demand. However, surprisingly, it remains unclear how this MD network is related to a core component of cognition, the process of searching for a target among distractors. This was because previous neuroimaging studies of visual search were confounded by task difficulty or time on task. To circumvent these limitations, we examined human brain activity while participants perform two different visual search tasks. The performance of a task was limited by increased attentional demand, while the other task was primarily limited by poor quality of input data or neural noise. Throughout the MD network, increased activity and strengthened functional connectivity among the MD regions were observed under the search task recruiting capacity-limited attentional resources. The present findings provide unequivocal evidence that the MD network mediates visual search, as well as other capacity-limited cognitive processes.

Explaining the effects of distractor statistics in visual search

Visual search, the task of detecting or locating target items among distractor items in a visual scene, is an important function for animals and humans. Different theoretical accounts make differing predictions for the effects of distractor statistics. Here we use a task in which we parametrically vary distractor items, allowing for a simultaneously fine-grained and comprehensive study of distractor statistics. We found effects of target-distractor similarity, distractor variability, and an interaction between the two, although the effect of the interaction on performance differed from the one expected. To explain these findings, we constructed computational process models that make trial-by-trial predictions for behavior based on the stimulus presented. These models, including a Bayesian observer model, provided excellent accounts of both the qualitative and quantitative effects of distractor statistics, as well as of the effect of changing the statistics of the environment (in the form of distractors being drawn from a different distribution). We conclude with a broader discussion of the role of computational process models in the understanding of visual search.

Attention as a Unitary Concept

In this paper, I discuss attention in terms of selecting visual information and acting on it. Selection has been taken as a bedrock concept in attention research since James (1890). Selective attention guides action by privileging some things at the expense of others. I formalize this notion with models which capture the relationship between input and output under the control of spatial and temporal attention, by attenuating or discarding certain inputs and by weighing energetic costs, speed, and accuracy in meeting pre-chosen goals. Examples are given from everyday visually guided actions, and from modeling data obtained from visual searches through temporal and spatial arrays and related research. The relation between selection, as defined here, and other forms of attention is discussed at the end.

A target contrast signal theory of parallel processing in goal-directed search

Feature Integration Theory (FIT) set out the groundwork for much of the work in visual cognition since its publication. One of the most important legacies of this theory has been the emphasis on feature-specific processing. Nowadays, visual features are thought of as a sort of currency of visual attention (e.g., features can be attended, processing of attended features is enhanced), and attended features are thought to guide attention towards likely targets in a scene. Here we propose an alternative theory - the Target Contrast Signal Theory - based on the idea that when we search for a specific target, it is not the target-specific features that guide our attention towards the target; rather, what determines behavior is the result of an active comparison between the target template in mind and every element present in the scene. This comparison occurs in parallel and is aimed at rejecting from consideration items that peripheral vision can confidently reject as being non-targets. The speed at which each item is evaluated is determined by the overall contrast between that item and the target template. We present computational simulations to demonstrate the workings of the theory as well as eye-movement data that support core predictions of the theory. The theory is discussed in the context of FIT and other important theories of visual search.

Does the standard search task predict performance in related tasks for Kanizsa-style illusory contours?

It is often assumed that results from standard visual search tasks will be replicated in related tasks but his idea is rarely tested. In a conceptual replication of Li, Cave, and Wolfe (2008), we investigated the attentional demands of Kanizsa-style illusory contours using orientation-based search, comparing performance for items defined by real- as compared to illusory contours. After confirming the initial findings in standard search, we tested the same manipulation in multiple-target search, Thornton and Gilden's (2007) hybrid standard/multiple-target search, and simple- and selective enumeration. The RT slope differences between real- and illusory contours did not replicate in Thornton and Gilden's task, though they did in multiple-target search and selective enumeration. In fact, absolute differences between real- and illusory contours in RT costs per distractor were 2 - 6 times larger than in standard search. To determine whether performance differences between real and illusory contours originated from shape-definition (necessary for distinguishing target shapes from distractors) or unit formation (grouping disconnected parts to define an item/unit), simple and selective enumeration were compared. The differences between real- and illusory-contours only emerged in selective enumeration (enumerating targets among distractors), which suggests the discrepancies between conditions originate from shape definition rather than unit formation processes. There was no evidence of subitizing in selective enumeration for illusory contour figures, but contrary to attention-based theories of enumeration, there was no subitizing for the real-contour controls either. This study contributes to research on illusory contours but it is especially important to the study of search and enumeration.

Temporal organization of color and shape processing during visual search

The mechanisms guiding visual attention are of great interest within cognitive and perceptual psychology. Many researchers have proposed models of these mechanisms, which serve to both formalize their theories and to guide further empirical investigations. The assumption that a number of basic features are processed in parallel early in the attentional process is common among most models of visual attention and visual search. To date, much of the evidence for parallel processing has been limited to set-size manipulations. Unfortunately, set-size manipulations have been shown to be insufficient evidence for parallel processing. We applied Systems Factorial Technology, a general nonparametric framework, to test this assumption, specifically whether color and shape are processed in parallel or in serial, in three experiments representative of feature search, conjunctive search, and odd-one-out search, respectively. Our results provide strong evidence that color and shape information guides search through parallel processes. Furthermore, we found evidence for facilitation between color and shape when the target was known in advance but performance consistent with unlimited capacity, independent parallel processing in odd-one-out search. These results confirm core assumptions about color and shape feature processing instantiated in most models of visual search and provide more detailed clues about the manner in which color and shape information is combined to guide search.

A model that adopts human fixations explains individual differences in multiple object tracking

In many settings "keep your eye on the ball" is good advice. People fixate important objects to obtain high quality information. Perhaps equally often, however, we engage with multiple important, moving, and unpredictable objects. Where should we fixate in these situations, and where do we? Do we for example appropriately center fixations to manage spatial non-uniformity in our visual system? And do we fixate empty space strategically to gain as much information as possible about multiple objects of interest? We explored these issues in the context of Multiple Object Tracking (MOT), wherein observers track several moving objects (targets) within a larger set of moving objects (nontargets), all the objects physically indistinguishable from one another. Among the features that make MOT an interesting paradigm is that it cannot be accommodated by continuous gaze to one important object, because there are multiple such objects in a given trial. Instead, it demands sustained processing of inputs from an entire display and iterated inferences about target versus nontarget identities. MOT therefore demands a strategic interaction between eye movements and cognition: the observer should seek fixation locations that minimize the aggregate probability of confusing any target with any nontarget. Individuals who meet this fixation challenge should perform the task better than those who meet the challenge less effectively. Here we describe a probabilistic model that implements the basic computations needed to do MOT, estimating the positions of targets, predicting their future positions, and inferring correspondences between new inputs and represented targets. The quality of the input received by the model depends on its fixation location at a given moment. We simulated a group of fifty participants who all performed the same MOT trials, with the model adopting each observer's fixation locations in the respective simulations. The model reliably predicted individual participant tracking performances and their relative rankings within the cohort. The results suggest that an individual's relative capability in this cognitively demanding task is in part determined by his/her utilization of eye fixations to control the quality and relevance of incoming visual input.

Use of unmanned aerial vehicles (UAVs) for mark-resight nesting population estimation of adult female green sea turtles at Raine Island

Nester abundance is a key measure of the performance of the world’s largest green turtle rookery at Raine Island, Australia, and has been estimated by mark-resight counts since 1984. Nesters are first marked by painting their carapace with a longitudinal white stripe. Painted and unpainted turtles are then counted by a surface observer on a small boat in waters adjacent to the reef. Unmanned aerial vehicles (UAV) and underwater video may provide more cost-effective and less biased alternatives to this approach, but estimates must be comparable with historical estimates. Here we compare and evaluate the three methods. We found comparatively little variation in resighting probabilities between consecutive days of sampling or time of day, which supports an underlying assumption of the method (i.e. demographic closure during sampling). This lack of bias in the location availability for detection of painted versus unpainted turtles and further supported by a parallel satellite tracking study of 40 turtles at Raine Island. Our results demonstrated that surface observers consistently reported higher proportions of marked turtles than either the UAV or underwater video method. This in turn yielded higher population estimates with UAV or underwater video compared to the historical surface observer method, which suggested correction factors of 1.53 and 1.73 respectively. We attributed this to observer search error because a white marked turtle is easier to spot than the non-marked turtle. In contrast, the UAV and underwater video methods allowed subsequent frame-by-frame review, thus reducing observer search error. UAVs were the most efficient in terms of survey time, personnel commitment and weather tolerance compared to the other methods. However, underwater video may also be a useful alternative for in-water mark-resight surveys of turtles.

A diffusion model analysis of target detection in near-threshold visual search

When searching for a target briefly presented among distractors how do people combine information across display locations to make a decision and how does the quality of the evidence entering the decision process vary with the type of items in the display? Research on accuracy in near-threshold visual search has had difficulty in distinguishing between models that make similar predictions about accuracy but make different assumptions about the underlying psychological processes. We used the diffusion model to analyse response times and accuracy data from four near-threshold search tasks which showed striking asymmetries between response-time distributions on target-present and target-absent trials. We found that performance was better explained by a model in which evidence was accumulated in parallel about each stimulus separately than one in which the evidence was pooled into a single decision process. We found that as contrast increased, the quality of the evidence entering the decision process about targets was markedly stronger than the evidence about nontargets. The overall pattern of evidence strength for stimuli on target-present and target-absent trials was consistent with a fixed-capacity memory system in which early visual processes assigned resources preferentially to targets over nontargets. The asymmetry was somewhat reduced in a letter-digit discrimination task that used heterogeneous targets and distractors, likely because heterogeneity reduces the efficiency of the preattentive filtering processes.

Seeing Beyond Salience and Guidance: The Role of Bias and Decision in Visual Search

Visual search is a popular tool for studying a range of questions about perception and attention, thanks to the ease with which the basic paradigm can be controlled and manipulated. While often thought of as a sub-field of vision science, search tasks are significantly more complex than most other perceptual tasks, with strategy and decision playing an essential, but neglected, role. In this review, we briefly describe some of the important theoretical advances about perception and attention that have been gained from studying visual search within the signal detection and guided search frameworks. Under most circumstances, search also involves executing a series of eye movements. We argue that understanding the contribution of biases, routines and strategies to visual search performance over multiple fixations will lead to new insights about these decision-related processes and how they interact with perception and attention. We also highlight the neglected potential for variability, both within and between searchers, to contribute to our understanding of visual search. The exciting challenge will be to account for variations in search performance caused by these numerous factors and their interactions. We conclude the review with some recommendations for ways future research can tackle these challenges to move the field forward.

Concurrent evaluation of independently cued features during perceptual decisions and saccadic targeting in visual search

Simultaneous search for one of two targets is slower and less accurate than search for a single target. Within the Signal Detection Theoretic (SDT) framework, this can be attributed to the division of resources during the comparison of visual input against independently cued targets. The current study used one or two cues to elicit single- and dual-target searches for orientation targets among similar and dissimilar distractors. In Experiment 1, the accuracy of target discrimination in brief displays was compared at setsizes of 1, 2 and 4. Results revealed a reduction in accuracy that scaled with the product of set size and the number of cued targets. In Experiment 2, the accuracy and latency of observers’ saccadic targeting were compared. Fixations on single-target searches were highly selective towards the target. On dual-target searches, the requirement to detect one of two targets produced a significant reduction in target fixations and equivalent rates of fixations to distractors with opposite orientations. For most observers, the dual-target cost was predicted by an SDT model that simulated increases in decision-noise and the distribution of capacity-limited resources during the comparison of selected input against independently cued targets. For others, search accuracy was consistent with a single-item limit on perceptual decisions and saccadic targeting during search. These findings support a flexible account of the dual-target cost based on different strategies to resolve competition between independently cued targets.

What limits search for conjunctions of simple visual features?

Despite of decades of research, we still do not know for sure the roles of internal noise, attention, and crowding in search for conjunctions of simple visual features. In this study, we tried several modifications to the classic design of conjunction-search experiments. In order to match exactly the proportions of simple features, two different targets were presented in target-present trials-vertical red and horizontal blue bars among vertical blue and horizontal red distractors. Both the length of the bars and the number of objects in a display were varied. Positions of objects were selected for minimal crowding effects. Exposure duration was 60 ms, and proportion correct was used as the measure of performance. For conjunction search, the results rejected the unlimited-capacity model and were consistent with limited-capacity attentional processing, and with the Naka-Rushton transform of the target-distractor difference. Qualitatively the same results were obtained when bar length was fixed, and fine orientation difference was used to manipulate target-distractor discriminability. An experiment of feature (orientation) search produced results close to the unlimited-capacity model.

Visual search does not always predict performance in tasks that require finding targets among distractors: The case of line-ending illusory contours

The standard visual search task is integral to the study of selective attention and in search tasks target present slopes are the primary index of attentional demand. However, there are times when similarities in slopes may obscure important differences between conditions. To demonstrate this point, we used the case of line-ending illusory contours, building on a study by Li, Cave, and Wolfe (2008) where orientation-based search for figures defined by line-ending illusory contours was compared to that for the corresponding real-contour controls. Consistent with Li et al. (2008), we found search to be efficient for both illusory contour figures and the corresponding real-contour controls, with no significant differences between them. However, major differences between illusory contours and the real-contour controls emerged in selective enumeration, a task where participants enumerated targets in a display of distractors, with the number of targets and distractors manipulated. When looking at the distractor slopes, the increase in RT to enumerate a single target as a function of the number of distractors (a direct analogue to target present trials, with identical displays), we found distractor costs for illusory contour figures to be over 100 ms/distractor higher than for the corresponding real-contour controls. Furthermore, the discrepancies in RT slope between 1-3 and 6-8 targets associated with subitizing were only seen in the real-contour controls. These results show that similarities in RT slopes in search may mask important differences between conditions that emerge in other tasks.

The face-in-the-crowd effect: Threat detection versus iso-feature suppression and collinear facilitation

Are people biologically prepared for the rapid detection of threat posed by an angry facial expression, even when it is conveyed in the form of a schematic line drawing? Based on visual search times, the current literature would suggest that the answer is yes. But are there low-level explanations for this effect? Here, we present visual search results for schematic faces using current best practice, based on a concentric search array and set size manipulation. Using this approach, we replicate the classic search advantage for angry over happy faces. However, we also report a comparable effect when abstract plus- and square-shaped stimuli-derived from the angry and happy schematic faces respectively-are used within the same paradigm. We then go on to demonstrate that, while reduced, the effect remains after removal of the circular surround, bringing us closer to the source of the effect. We explore the possibility that the source of this search asymmetry could be the iso-feature suppression and collinear facilitation model proposed in Li's (1999a, 1999b, and 2002) bottom-up model of saliency. Simulations with this model using the abstract stimuli align with the corresponding behavioral results (i.e., the plus shape was found to be more salient than the square). Given the deliberate similarities between these abstract shapes and the respective face stimuli, we propose that the underlying cause for the asymmetries typically found using schematic faces, may be more related to early visual processing of line orientation than threat detection.

Visual search for complex objects: Set-size effects for faces, words and cars

To compare visual processing for different object types, we developed visual search tests that generated accuracy and response time parameters, including an object set-size effect that indexes perceptual processing load. Our goal was to compare visual search for two expert object types, faces and visual words, as well as a less expert type, cars. We first asked if faces and words showed greater inversion effects in search. Second, we determined whether search with upright stimuli correlated with other perceptual indices. Last we assessed for correlations between tests within a single orientation, and between orientations for a single object type. Object set-size effects were smaller for faces and words than cars. All accuracy and temporal measures showed an inversion effect for faces and words, but not cars. Face-search accuracy measures correlated with accuracy on the Cambridge Face Memory Test and word-search temporal measures correlated with single-word reading times, but car search did not correlate with semantic car knowledge. There were cross-orientation correlations for all object types, as well as cross-object correlations in the inverted orientation, while in the upright orientation face search did not correlate with word or car search. We conclude that object search shows effects of expertise. Compared to cars, words and faces showed smaller object set-size effects, greater inversion effects, and their search results correlated with other indices of perceptual expertise. The correlation analyses provide preliminary evidence supporting contributions from common processes in the case of inverted stimuli, object-specific processes that operate in both orientations, and distinct processing for upright faces.

Unraveling Causal Mechanisms of Top-Down and Bottom-Up Visuospatial Attention with Non-invasive Brain Stimulation

Attention is a process of selection that allows us to intelligently navigate the abundance of information in our world. Attention can be either directed voluntarily based on internal goals-"top-down" or goal-directed attention-or captured automatically, by salient stimuli-"bottom-up" or stimulus-driven attention. Do these two modes of attention control arise from same or different brain circuits? Do they share similar or distinct neural mechanisms? In this review, we explore this dichotomy between the neural bases of top-down and bottom-up attention control, with a special emphasis on insights gained from non-invasive neurostimulation techniques, specifically, transcranial magnetic stimulation (TMS). TMS enables spatially focal and temporally precise manipulation of brain activity. We explore a significant literature devoted to investigating the role of fronto-parietal brain regions in top-down and bottom-up attention with TMS, and highlight key areas of convergence and debate. We also discuss recent advances in combinatorial paradigms that combine TMS with other imaging modalities, such as functional magnetic resonance imaging or electroencephalography. These paradigms are beginning to bridge essential gaps in our understanding of the neural pathways by which TMS affects behavior, and will prove invaluable for unraveling mechanisms of attention control, both in health and in disease.

Speeded multielement decision-making as diffusion in a hypersphere: Theory and application to double-target detection

We generalize the circular 2D diffusion model of Smith (Psychological Review, 123, 425-451: 2016) to provide a new model of speeded decision-making in multielement visual displays. We model decision-making in tasks with multielement displays as evidence accumulation by a vector-valued diffusion process in a hypersphere, whose radius represents the decision criterion for the task. We show that the methods used to derive response time and accuracy predictions for the 2D model can be applied, with only minor changes, to predict performance in higher-dimensional spaces as well. We apply the model to the double-target deficit paradigm of Duncan (Psychological Review, 87, 272-300: 1980) in which participants judge whether briefly presented four-element displays contain one- or two-digit targets among letter distractors. A 4D version of the hyperspherical diffusion model correctly predicted distributions of response times and response accuracy as a function of task difficulty in single-target and double-target versions of the task. The estimated drift rate parameters from the model imply that the mental representation of the decision alternatives, which we term the "decision template" for the task, encodes configural stimulus properties that reflect the number of targets in the display. Along with its application to multielement decision-making, the model has the potential to characterize the speed and accuracy of multiattribute decisions in studies of cognitive categorization, visual attention, and other areas.

Natural image clutter degrades overt search performance independently of set size

Although studies of visual search have repeatedly demonstrated that visual clutter impairs search performance in natural scenes, these studies have not attempted to disentangle the effects of search set size from those of clutter per se. Here, we investigate the effect of natural image clutter on performance in an overt search for categorical targets when the search set size is controlled. Observers completed a search task that required detecting and localizing common objects in a set of natural images. The images were sorted into high- and low-clutter conditions based on the clutter metric by Bravo and Farid (2008). The search set size was varied independently by fixing the number and positions of potential targets across set size conditions within a block of trials. Within each fixed set size condition, search times increased as a function of increasing clutter, suggesting that clutter degrades overt search performance independently of set size.

Dissociating the impact of attention and expectation on early sensory processing

Most studies that focus on understanding how top-down knowledge influences behavior attempt to manipulate either 'attention' or 'expectation' and often use the terms interchangeably. However, having expectations about statistical regularities in the environment and the act of willfully allocating attention to a subset of relevant sensory inputs are logically distinct processes that could, in principle, rely on similar neural mechanisms and influence information processing at the same stages. In support of this framework, several recent studies attempted to isolate expectation from attention, and advanced the idea that expectation and attention both modulate early sensory processing. Here, we argue that there is currently insufficient empirical evidence to support this conclusion, because previous studies have not fully isolated the effects of expectation and attention. Instead, most prior studies manipulated the relevance of different sensory features, and as a result, few existing findings speak directly to the potentially separable influences of expectation and attention on early sensory processing. Indeed, recent studies that attempt to more strictly isolate expectation and attention suggest that expectation has little influence on early sensory responses and primarily influences later 'decisional' stages of information processing.

On the brink: The demise of the item in visual search moves closer

We proposed to abandon the item as conceptual unit in visual search and adopt a fixation-based framework instead. We treat various themes raised by our commentators, including the nature of the Functional Visual Field and existing similar ideas, alongside the importance of items, covert attention, and top-down/contextual influences. We reflect on the current state of, and future directions for, visual search.

Multi-step planning of eye movements in visual search

The capability of directing gaze to relevant parts in the environment is crucial for our survival. Computational models have proposed quantitative accounts of human gaze selection in a range of visual search tasks. Initially, models suggested that gaze is directed to the locations in a visual scene at which some criterion such as the probability of target location, the reduction of uncertainty or the maximization of reward appear to be maximal. But subsequent studies established, that in some tasks humans instead direct their gaze to locations, such that after the single next look the criterion is expected to become maximal. However, in tasks going beyond a single action, the entire action sequence may determine future rewards thereby necessitating planning beyond a single next gaze shift. While previous empirical studies have suggested that human gaze sequences are planned, quantitative evidence for whether the human visual system is capable of finding optimal eye movement sequences according to probabilistic planning is missing. Here we employ a series of computational models to investigate whether humans are capable of looking ahead more than the next single eye movement. We found clear evidence that subjects' behavior was better explained by the model of a planning observer compared to a myopic, greedy observer, which selects only a single saccade at a time. In particular, the location of our subjects' first fixation differed depending on the stimulus and the time available for the search, which was well predicted quantitatively by a probabilistic planning model. Overall, our results are the first evidence that the human visual system's gaze selection agrees with optimal planning under uncertainty.

Beer? Over here! Examining attentional bias towards alcoholic and appetitive stimuli in a visual search eye-tracking task

RATIONALE

Experimental tasks that demonstrate alcohol-related attentional bias typically expose participants to single-stimulus targets (e.g. addiction Stroop, visual probe, anti-saccade task), which may not correspond fully with real-world contexts where alcoholic and non-alcoholic cues simultaneously compete for attention. Moreover, alcoholic stimuli are rarely matched to other appetitive non-alcoholic stimuli.

OBJECTIVES

To address these limitations by utilising a conjunction search eye-tracking task and matched stimuli to examine alcohol-related attentional bias.

METHODS

Thirty social drinkers (Mage = 19.87, SD = 1.74) were asked to detect whether alcoholic (beer), non-alcoholic (water) or non-appetitive (detergent) targets were present or absent amongst a visual array of matching and non-matching distractors. Both behavioural response times and eye-movement dwell time were measured.

RESULTS

Social drinkers were significantly quicker to detect alcoholic and non-alcoholic appetitive targets relative to non-appetitive targets in an array of matching and mismatching distractors. Similarly, proportional dwell time was lower for both alcoholic and non-alcoholic appetitive distractors relative to non-appetitive distractors, suggesting that appetitive targets were relatively easier to detect.

CONCLUSIONS

Social drinkers may exhibit generalised attentional bias towards alcoholic and non-alcoholic appetitive cues. This adds to emergent research suggesting that the mechanisms driving these individual's attention towards alcoholic cues might 'spill over' to other appetitive cues, possibly due to associative learning.

Having More Choices Changes How Human Observers Weight Stable Sensory Evidence

Decision-making becomes slower when more choices are available. Existing models attribute this slowing to poor sensory processing, to attenuated rates of sensory evidence accumulation, or to increases in the amount of evidence required before committing to a decision (a higher decision threshold). However, studies have not isolated the effects of having more choices on sensory and decision-related processes from changes in task difficulty and divided attention. Here, we controlled task difficulty while independently manipulating the distribution of attention and the number of choices available to male and female human observers. We used EEG to measure steady-state visually evoked potentials (SSVEPs) and a frontal late positive deflection (LPD), EEG markers of sensory and postsensory decision-related processes, respectively. We found that dividing attention decreased SSVEP and LPD amplitudes, consistent with dampened sensory responses and slower rates of evidence accumulation, respectively. In contrast, having more choices did not alter SSVEP amplitude and led to a larger LPD. These results suggest that having more options largely spares early sensory processing and slows down decision-making via a selective increase in decision thresholds.SIGNIFICANCE STATEMENT When more choices are available, decision-making becomes slower. We tested whether this phenomenon is due to poor sensory processing, to reduced rates of evidence accumulation, or to increases in the amount of evidence required before committing to a decision (a higher decision threshold). We measured choice modulations of sensory and decision-related neural responses using EEG. We also minimized potential confounds from changes in the distribution of attention and task difficulty, which often covary with having more choices. Dividing attention reduced the activity levels of both sensory and decision-related responses. However, having more choices did not change sensory processing and led to larger decision-related responses. These results suggest that having more choices spares sensory processing and selectively increases decision thresholds.

Theoretical vs. empirical discriminability: the application of ROC methods to eyewitness identification

Receiver operating characteristic (ROC) analysis was introduced to the field of eyewitness identification 5 years ago. Since that time, it has been both influential and controversial, and the debate has raised an issue about measuring discriminability that is rarely considered. The issue concerns the distinction between empirical discriminability (measured by area under the ROC curve) vs. underlying/theoretical discriminability (measured by d’ or variants of it). Under most circumstances, the two measures will agree about a difference between two conditions in terms of discriminability. However, it is possible for them to disagree, and that fact can lead to confusion about which condition actually yields higher discriminability. For example, if the two conditions have implications for real-world practice (e.g., a comparison of competing lineup formats), should a policymaker rely on the area-under-the-curve measure or the theory-based measure? Here, we illustrate the fact that a given empirical ROC yields as many underlying discriminability measures as there are theories that one is willing to take seriously. No matter which theory is correct, for practical purposes, the singular area-under-the-curve measure best identifies the diagnostically superior procedure. For that reason, area under the ROC curve informs policy in a way that underlying theoretical discriminability never can. At the same time, theoretical measures of discriminability are equally important, but for a different reason. Without an adequate theoretical understanding of the relevant task, the field will be in no position to enhance empirical discriminability.

The bandwidth of VWM consolidation varies with the stimulus feature: Evidence from event-related potentials

Our previous work suggests that 2 colors can be consolidated into visual short-term memory (VSTM) in parallel without a loss of memory precision, whereas consolidation of 2 orientations is performed in a strictly serial manner. Those experiments compared VSTM performance for simultaneously and sequentially presented stimuli. However, there is still controversy about whether the bandwidth for consolidation is determined by the type of information. To further investigate this issue, here we measured electroencephalography while participants attempted to consolidate 1, 2 or 4 simultaneously presented colors (Experiment 1) or orientations (Experiment 2) under limited presentation times. We used the contralateral delay activity (CDA) as an electrophysiological marker of the number of items that were consolidated. For colored stimuli, the CDA amplitude increased between set-size 1 and 2 but did not further increase for set size 4. By contrast, for orientation, the CDA amplitude remained at the set size 1 amplitude as set size increased to 2 or 4 items. Furthermore, in a long exposure duration (300 ms) condition that did not limit the consolidation process, the CDA amplitude pattern indicated that VSTM capacity is limited to about 3 colored items and about 2 orientation items in our paradigm. Thus, the CDA effects observed in the short presentation time was not limited by VSTM storage, but rather by consolidation. These results are consistent with our previous behavioral research and suggest that the bandwidth of VSTM consolidation is determined by the stimulus feature. (PsycINFO Database Record

Distinct perceptual rhythms for feature and conjunction searches

Feature and conjunction searches are widely used to study attentional deployment. However, the spatiotemporal behavior of attention integration in these tasks remains under debate. Are multiple search stimuli processed in parallel or sequentially? Does sampling of visual information and attentional deployment differ between these two types of search? If so, how? We used an innovative methodology to estimate the distribution of attention on a single-trial basis for feature and conjunction searches. Observers performed feature- and conjunction-search tasks. They had to detect and discriminate a tilted low-spatial-frequency grating among three low-spatial-frequency vertical gratings (feature search) or low-spatial-frequency vertical gratings and high-spatial-frequency tilted gratings (conjunction search). After a variable delay, two probes were flashed at random locations. Performance in reporting the probes was used to infer attentional deployment to those locations. By solving a second-degree equation, we determined the probability of probe report at the most (P1) and least (P2) attended locations on a given trial. Were P1 and P2 equal, we would conclude that attention had been uniformly distributed across all four locations. Otherwise, we would conclude that visual information sampling and attentional deployment had been nonuniformly distributed. Our results show that processing was nonuniformly distributed across the four locations in both searches, and was modulated periodically over time at ∼5 Hz for the conjunction search and ∼12 Hz for the feature search. We argue that the former corresponds to the periodicity of attentional deployment during the search, whereas the latter corresponds to ongoing sampling of visual information. Because different locations were not simultaneously processed, this study rules out a strict parallel model for both search types.

Search for Capacity-Limited and Super-Capacity Search

Abstract. The present study investigated capacity limitations of visual search. In a series of experiments, participants searched for a singleton target among homogenous distractors, a conjunction target defined by combination of two features, or a feature target among heterogeneous distractors. Using the simultaneous-sequential paradigm, I found that singleton search proceeded in a capacity-unlimited manner. By contrast, the performance of the conjunction search was found to depend on a capacity-limited process. For feature searches, the performance of searching for a specific color was not affected by how the stimuli were presented, while the orientation search performance was enhanced as the number of distractors simultaneously presented with the target increased. These results imply that distinct colors are individually coded, whereas multiple orientations are encoded as an ensemble in a structured way. Taken together, the present study clarifies which type of search process are capacity-limited and reveals how this limit can be overcome.

A Quantitative Relationship between Signal Detection in Attention and Approach/Avoidance Behavior

This study examines how the domains of reward and attention, which are often studied as independent processes, in fact interact at a systems level. We operationalize divided attention with a continuous performance task and variables from signal detection theory (SDT), and reward/aversion with a keypress task measuring approach/avoidance in the framework of relative preference theory (RPT). Independent experiments with the same subjects showed a significant association between one SDT and two RPT variables, visualized as a three-dimensional structure. Holding one of these three variables constant, further showed a significant relationship between a loss aversion-like metric from the approach/avoidance task, and the response bias observed during the divided attention task. These results indicate that a more liberal response bias under signal detection (i.e., a higher tolerance for noise, resulting in a greater proportion of false alarms) is associated with higher "loss aversion." Furthermore, our functional model suggests a mechanism for processing constraints with divided attention and reward/aversion. Together, our results argue for a systematic relationship between divided attention and reward/aversion processing in humans.

Competing Distractors Facilitate Visual Search in Heterogeneous Displays

In the present study, we examine how observers search among complex displays. Participants were asked to search for a big red horizontal line among 119 distractor lines of various sizes, orientations and colours, leading to 36 different feature combinations. To understand how people search in such a heterogeneous display, we evolved the search display by using a genetic algorithm (Experiment 1). The best displays (i.e., displays corresponding to the fastest reaction times) were selected and combined to create new, evolved displays. Search times declined over generations. Results show that items sharing the same colour and orientation as the target disappeared over generations, implying they interfered with search, but items sharing the same colour and were 12.5° different in orientation only interfered if they were also the same size. Furthermore, and inconsistent with most dominant visual search theories, we found that non-red horizontal distractors increased over generations, indicating that these distractors facilitated visual search while participants were searching for a big red horizontally oriented target. In Experiments 2 and 3, we replicated these results using conventional, factorial experiments. Interestingly, in Experiment 4, we found that this facilitation effect was only present when the displays were very heterogeneous. While current models of visual search are able to successfully describe search in homogeneous displays, our results challenge the ability of these models to describe visual search in heterogeneous environments.

Losing the trees for the forest in dynamic visual search

Representing temporally continuous objects across change (e.g., in position) requires integration of newly sampled visual information with existing object representations. We asked what consequences representational updating has for visual search. In this dynamic visual search task, bars rotated around their central axis. Observers searched for a single episodic target state (oblique bar among vertical and horizontal bars). Search was efficient when the target display was presented as an isolated static display. Performance declined to near chance, however, when the same display was a single state of a dynamically changing scene (Experiment 1), as though temporal selection of the target display from the stream of stimulation failed entirely (Experiment 3). The deficit is attributable neither to masking (Experiment 2), nor to a lack of temporal marker for the target display (Experiment 4). The deficit was partially reduced by visually marking the target display with unique feature information (Experiment 5). We suggest that representational updating causes a loss of access to instantaneous state information in search. Similar to spatially crowded displays that are perceived as textures (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001), we propose a temporal version of the trees (instantaneous orientation information) being lost for the forest (rotating bars).