How important is lateral masking in visual search?

Does crowding predict conjunction search? An individual differences approach

Searching for objects in the visual environment is an integral part of human behavior. Most of the information used during such visual search comes from the periphery of our vision, and understanding the basic mechanisms of search therefore requires taking into account the inherent limitations of peripheral vision. Our previous work using an individual differences approach has shown that one of the major factors limiting peripheral vision (crowding) is predictive of single feature search, as reflected in response time and eye movement measures. Here we extended this work, by testing the relationship between crowding and visual search in a conjunction-search paradigm. Given that conjunction search involves more fine-grained discrimination and more serial behavior, we predicted it would be strongly affected by crowding. We tested sixty participants with regard to their sensitivity to both orientation and color-based crowding (as measured by critical spacing) and their efficiency in searching for a color/orientation conjunction (as indicated by manual response times and eye movements). While the correlations between the different crowding tasks were high, the correlations between the different crowding measures and search performance were relatively modest, and no higher than those previously observed for single-feature search. Instead, observers showed very strong color selectivity during search. The results suggest that conjunction search behavior relies more on top-down guidance (here by color) and is therefore relatively less determined by individual differences in sensory limitations as caused by crowding.

Individual differences in crowding predict visual search performance

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

Encoding perceptual ensembles during visual search in peripheral vision

Observers can learn complex statistical properties of visual ensembles, such as their probability distributions. Even though ensemble encoding is considered critical for peripheral vision, whether observers learn such distributions in the periphery has not been studied. Here, we used a visual search task to investigate how the shape of distractor distributions influences search performance and ensemble encoding in peripheral and central vision. Observers looked for an oddly oriented bar among distractors taken from either uniform or Gaussian orientation distributions with the same mean and range. The search arrays were either presented in the foveal or peripheral visual fields. The repetition and role reversal effects on search times revealed observers' internal model of distractor distributions. Our results showed that the shape of the distractor distribution influenced search times only in foveal, but not in peripheral search. However, role reversal effects revealed that the shape of the distractor distribution could be encoded peripherally depending on the interitem spacing in the search array. Our results suggest that, although peripheral vision might rely heavily on summary statistical representations of feature distributions, it can also encode information about the distributions themselves.

Visual Search Without Selective Attention: A Cognitive Architecture Account

A key phenomenon in visual search experiments is the linear relation of reaction time (RT) to the number of objects to be searched (set size). The dominant theory of visual search claims that this is a result of covert selective attention operating sequentially to "bind" visual features into objects, and this mechanism operates differently depending on the nature of the search task and the visual features involved, causing the slope of the RT as a function of set size to range from zero to large values. However, a cognitive architectural model presented here shows these effects on RT in three different search task conditions can be easily obtained from basic visual mechanisms, eye movements, and simple task strategies. No selective attention mechanism is needed. In addition, there are little-explored effects of visual crowding, which is typically confounded with set size in visual search experiments. Including a simple mechanism for crowding in the model also allows it to account for significant effects on error rate (ER). The resulting model shows the interaction between visual mechanisms and task strategy, and thus it represents a more comprehensive and fruitful approach to visual search than the dominant theory.

Interference in lateral masking stimuli - The effects of relative phase, position, orientation, and spatial frequency

Lateral masking has been defined as the perception of a visual target stimulus being impaired when other stimuli are present in its adjacent surroundings. In such cases it has generally been assumed that the target stimulus presented along with a masking stimulus has the same stimulus power as when presented alone and that the reduced visibility reflects interactions in the visual system. It has, however, become clear that there may be interference between such stimuli [5]. Such interference, which takes place in the stimuli and is independent of the visual system, has the potential to reduce the stimulus power of target stimuli. The present report asks, employing 2-Dimensional Gabor functions as stimuli, how interference effects may depend on (1) relative spatial phase, (2) separation between target and masking stimuli, (3) difference in orientation, and (4) difference in spatial frequency between masking and target stimuli. Interference was estimated numerically based on the sums of the amplitudes in the Fourier spectra and the norms of these spectra. Clear evidence for interference was demonstrated with both measures. All the four parameters have the ability to influence the amount of interference. These findings, therefore, emphasize that one cannot count on a target stimulus presented along with masking stimuli to have the same stimulus power as when it is presented by itself.

Do sudden onsets need to be perceived as new objects to capture attention? The interplay between sensory transients and display configuration

Yantis and Jonides (1984) and Jonides and Yantis (1988) reported robust involuntary attentional capture by sudden-onsets, the origin of which has been debated. Prominent accounts have highlighted aspects that include the "new object" status of a sudden-onset (Yantis & Hillstrom, 1994) and the substantial luminance changes accompanying their appearance (Gellatly, Cole & Blurton, 1999; Franconeri, Hollingworth & Simons, 2005), including relative differences in the amount of sensory change between target and nontarget items (Pinto, Olivers & Theeuwes, 2008). In this research we dissociate the amount of sensory change accompanying sudden onsets from the extent to which they appear as newly created objects in search displays. We attempted to determine the relative contribution of local sensory changes and display configuration to attentional capture by sudden-onsets. We showed that the display configuration of old objects modulates the impact of capture caused by sudden-onsets.

Search performance is better predicted by tileability than presence of a unique basic feature

Traditional models of visual search such as feature integration theory (FIT; Treisman & Gelade, 1980), have suggested that a key factor determining task difficulty consists of whether or not the search target contains a "basic feature" not found in the other display items (distractors). Here we discriminate between such traditional models and our recent texture tiling model (TTM) of search (Rosenholtz, Huang, Raj, Balas, & Ilie, 2012b), by designing new experiments that directly pit these models against each other. Doing so is nontrivial, for two reasons. First, the visual representation in TTM is fully specified, and makes clear testable predictions, but its complexity makes getting intuitions difficult. Here we elucidate a rule of thumb for TTM, which enables us to easily design new and interesting search experiments. FIT, on the other hand, is somewhat ill-defined and hard to pin down. To get around this, rather than designing totally new search experiments, we start with five classic experiments that FIT already claims to explain: T among Ls, 2 among 5s, Q among Os, O among Qs, and an orientation/luminance-contrast conjunction search. We find that fairly subtle changes in these search tasks lead to significant changes in performance, in a direction predicted by TTM, providing definitive evidence in favor of the texture tiling model as opposed to traditional views of search.

Cube search, revisited

Observers can quickly search among shaded cubes for one lit from a unique direction. However, replace the cubes with similar 2-D patterns that do not appear to have a 3-D shape, and search difficulty increases. These results have challenged models of visual search and attention. We demonstrate that cube search displays differ from those with "equivalent" 2-D search items in terms of the informativeness of fairly low-level image statistics. This informativeness predicts peripheral discriminability of target-present from target-absent patches, which in turn predicts visual search performance, across a wide range of conditions. Comparing model performance on a number of classic search tasks, cube search does not appear unexpectedly easy. Easy cube search, per se, does not provide evidence for preattentive computation of 3-D scene properties. However, search asymmetries derived from rotating and/or flipping the cube search displays cannot be explained by the information in our current set of image statistics. This may merely suggest a need to modify the model's set of 2-D image statistics. Alternatively, it may be difficult cube search that provides evidence for preattentive computation of 3-D scene properties. By attributing 2-D luminance variations to a shaded 3-D shape, 3-D scene understanding may slow search for 2-D features of the target.

The prevalence effect in lateral masking and its relevance for visual search

In stimulus displays with or without a single target amid 1,644 identical distractors, target prevalence was varied between 20, 50 and 80 %. Maximum gaze deviation was measured to determine the strength of lateral masking in these arrays. The results show that lateral masking was strongest in the 20 % prevalence condition, which differed significantly from both the 50 and 80 % prevalence conditions. No difference was observed between the latter two. This pattern of results corresponds to that found in the literature on the prevalence effect in visual search (stronger lateral masking corresponding to longer search times). The data add to similar findings reported earlier (Wertheim et al. in Exp Brain Res, 170:387-402, 2006), according to which the effects of many well-known factors in visual search correspond to those on lateral masking. These were the effects of set size, disjunctions versus conjunctions, display area, distractor density, the asymmetry effect (Q vs. O's) and viewing distance. The present data, taken together with those earlier findings, may lend credit to a causal hypothesis that lateral masking could be a more important mechanism in visual search than usually assumed.

Attention in a bayesian framework

The behavioral phenomena of sensory attention are thought to reflect the allocation of a limited processing resource, but there is little consensus on the nature of the resource or why it should be limited. Here we argue that a fundamental bottleneck emerges naturally within Bayesian models of perception, and use this observation to frame a new computational account of the need for, and action of, attention - unifying diverse attentional phenomena in a way that goes beyond previous inferential, probabilistic and Bayesian models. Attentional effects are most evident in cluttered environments, and include both selective phenomena, where attention is invoked by cues that point to particular stimuli, and integrative phenomena, where attention is invoked dynamically by endogenous processing. However, most previous Bayesian accounts of attention have focused on describing relatively simple experimental settings, where cues shape expectations about a small number of upcoming stimuli and thus convey "prior" information about clearly defined objects. While operationally consistent with the experiments it seeks to describe, this view of attention as prior seems to miss many essential elements of both its selective and integrative roles, and thus cannot be easily extended to complex environments. We suggest that the resource bottleneck stems from the computational intractability of exact perceptual inference in complex settings, and that attention reflects an evolved mechanism for approximate inference which can be shaped to refine the local accuracy of perception. We show that this approach extends the simple picture of attention as prior, so as to provide a unified and computationally driven account of both selective and integrative attentional phenomena.

A summary statistic representation in peripheral vision explains visual search

Vision is an active process: We repeatedly move our eyes to seek out objects of interest and explore our environment. Visual search experiments capture aspects of this process, by having subjects look for a target within a background of distractors. Search speed often correlates with target-distractor discriminability; search is faster when the target and distractors look quite different. However, there are notable exceptions. A given discriminability can yield efficient searches (where the target seems to "pop-out") as well as inefficient ones (where additional distractors make search significantly slower and more difficult). Search is often more difficult when finding the target requires distinguishing a particular configuration or conjunction of features. Search asymmetries abound. These puzzling results have fueled three decades of theoretical and experimental studies. We argue that the key issue in search is the processing of image patches in the periphery, where visual representation is characterized by summary statistics computed over a sizable pooling region. By quantifying these statistics, we predict a set of classic search results, as well as peripheral discriminability of crowded patches such as those found in search displays.

Computational versus psychophysical bottom-up image saliency: a comparative evaluation study

The predictions of 13 computational bottom-up saliency models and a newly introduced Multiscale Contrast Conspicuity(MCC) metric are compared with human visual conspicuity measurements. The agreement between human visual conspicuity estimates and model saliency predictions is quantified through their rank order correlation. The maximum of the computational saliency value over the target support area correlates most strongly with visual conspicuity for 12 of the 13 models. A simple multiscale contrast model and the MCC metric both yield the largest correlation with human visual target conspicuity (>0:84). Local image saliency largely determines human visual inspection and interpretation of static and dynamic scenes. Computational saliency models therefore have a wide range of important applications, like adaptive content delivery, region-of-interest-based image compression, video summarization, progressive image transmission, image segmentation, image quality assessment, object recognition, and content-aware image scaling. However, current bottom-up saliency models do not incorporate important visual effects like crowding and lateral interaction. Additional knowledge about the exact nature of the interactions between the mechanisms mediating human visual saliency is required to develop these models further. The MCC metric and its associated psychophysical saliency measurement procedure are useful tools to systematically investigate the relative contribution of different feature dimensions to overall visual target saliency.

Saccadic selection and crowding in visual search: stronger lateral masking leads to shorter search times

We investigated the role of crowding in saccadic selection during visual search. To guide eye movements, often information from the visual periphery is used. Crowding is known to deteriorate the quality of peripheral information. In four search experiments, we studied the role of crowding, by accompanying individual search elements by flankers. Varying the difference between target and flankers allowed us to manipulate crowding strength throughout the stimulus. We found that eye movements are biased toward areas with little crowding for conditions where a target could be discriminated peripherally. Interestingly, for conditions in which the target could not be discriminated peripherally, this bias reversed to areas with strong crowding. This led to shorter search times for a target presented in areas with stronger crowding, compared to a target presented in areas with less crowding. These findings suggest a dual role for crowding in visual search. The presence of flankers similar to the target deteriorates the quality of the peripheral target signal but can also attract eye movements, as more potential targets are present over the area.

Visual conspicuity: a new simple standard, its reliability, validity and applicability

A general standard for quantifying conspicuity is described. It derives from a simple and easy method to quantitatively measure the visual conspicuity of an object. The method stems from the theoretical view that the conspicuity of an object is not a property of that object, but describes the degree to which the object is perceptually embedded in, i.e. laterally masked by, its visual environment. First, three variations of a simple method to measure the strength of such lateral masking are described and empirical evidence for its reliability and its validity is presented, as are several tests of predictions concerning the effects of viewing distance and ambient light. It is then shown how this method yields a conspicuity standard, expressed as a number, which can be made part of a rule of law, and which can be used to test whether or not, and to what extent, the conspicuity of a particular object, e.g. a traffic sign, meets a predetermined criterion. An additional feature is that, when used under different ambient light conditions, the method may also yield an index of the amount of visual clutter in the environment. Taken together the evidence illustrates the methods' applicability in both the laboratory and in real-life situations. STATEMENT OF RELEVANCE: This paper concerns a proposal for a new method to measure visual conspicuity, yielding a numerical index that can be used in a rule of law. It is of importance to ergonomists and human factor specialists who are asked to measure the conspicuity of an object, such as a traffic or rail-road sign, or any other object. The new method is simple and circumvents the need to perform elaborate (search) experiments and thus has great relevance as a simple tool for applied research.

A neurophysiologically plausible population code model for feature integration explains visual crowding

An object in the peripheral visual field is more difficult to recognize when surrounded by other objects. This phenomenon is called “crowding”. Crowding places a fundamental constraint on human vision that limits performance on numerous tasks. It has been suggested that crowding results from spatial feature integration necessary for object recognition. However, in the absence of convincing models, this theory has remained controversial. Here, we present a quantitative and physiologically plausible model for spatial integration of orientation signals, based on the principles of population coding. Using simulations, we demonstrate that this model coherently accounts for fundamental properties of crowding, including critical spacing, “compulsory averaging”, and a foveal-peripheral anisotropy. Moreover, we show that the model predicts increased responses to correlated visual stimuli. Altogether, these results suggest that crowding has little immediate bearing on object recognition but is a by-product of a general, elementary integration mechanism in early vision aimed at improving signal quality.

Visual crowding refers to the phenomenon that objects become more difficult to recognize when other objects surround them. Recently there has been an explosion of studies on crowding, driven, in part, by the belief that understanding crowding will help to understand a range of visual behaviours, including object recognition, visual search, reading, and texture recognition. Given the long-standing interest in the topic and its relevance for a wide range of research fields, it is quite surprising that after nearly a century of research the mechanisms underlying crowding are still as poorly understood as they are today. A nearly complete lack of quantitative models seems to be one of the main reasons for this. Here, we present a mathematical, biologically motivated model of feature integration at the level of neuron populations. Using simulations, we demonstrate that several fundamental properties of the crowding effect can be explained as the by-product of an integration mechanism that may have a function in contour integration. Altogether, these results help differentiate between earlier theories about both the neural and functional origin of crowding.

Overt and covert visual search in primates: reaction times and gaze shift strategies

In order to investigate the search performance and strategies of nonhuman primates, two macaque monkeys were trained to search for a target template among differently oriented distractors in both free-gaze and fixed-gaze viewing conditions (overt and covert search). In free-gaze search, reaction times (RT) and eye movements revealed the theoretically predicted characteristics of exhaustive and self-terminating serial search, with certain exceptions that are also observed in humans. RT was linearly related to the number of fixations but not necessarily to the number of items on display. Animals scanned the scenes in a nonrandom manner spending notably more time on targets and items inspected last (just before reaction). The characteristics of free-gaze search were then compared with search performance under fixed gaze (covert search) and with the performance of four human subjects tested in similar experiments. By and large the performance characteristics of both groups were similar; monkeys were slightly faster, and humans more accurate. Both species produced shorter RT in fixed-gaze than in free-gaze search. But while RT slopes of the human subjects still showed the theoretically predicted difference between hits and rejections, slopes of the two monkeys appeared to collapse. Despite considerable priming and short-term learning when similar tests were continuously repeated, no substantial long-term training effects were seen when test conditions and set sizes were frequently varied. Altogether, the data reveal many similarities between human and monkey search behavior but indicate that search is not necessarily restricted to exclusively serial processes.

Concealed by conspicuousness: distractive prey markings and backgrounds

High-contrast markings, called distractive or dazzle markings, have been suggested to draw and hold the attention of a viewer, thus hindering detection or recognition of revealing prey characteristics, such as the body outline. We tested this hypothesis in a predation experiment with blue tits (Cyanistes caeruleus) and artificial prey. We also tested whether this idea can be extrapolated to the background appearance and whether high-contrast markings in the background would improve prey concealment. We compared search times for a high-contrast range prey (HC-P) and a low-contrast range prey (LC-P) in a high-contrast range background (HC-B) and a low-contrast range background (LC-B). The HC-P was more difficult to detect in both backgrounds, although it did not match the LC-B. Also, both prey types were more difficult to find in the HC-B than in the LC-B, in spite of the mismatch of the LC-P. In addition, the HC-P was more difficult to detect, in both backgrounds, when compared with a generalist prey, not mismatching either background. Thus, we conclude that distractive prey pattern markings and selection of microhabitats with distractive features may provide an effective way to improve camouflage. Importantly, high-contrast markings, both as part of the prey coloration and in the background, can indeed increase prey concealment.

Search time critically depends on irrelevant subset size in visual search

In order for our visual system to deal with the massive amount of sensory input, some of this input is discarded, while other parts are processed [Wolfe, J. M. (1994). Guided search 2.0: a revised model of visual search. Psychonomic Bulletin and Review, 1, 202-238]. From the visual search literature it is unclear how well one set of items can be selected that differs in only one feature from target (a 1F set), while another set of items can be ignored that differs in two features from target (a 2F set). We systematically varied the percentage of 2F non-targets to determine the contribution of these non-targets to search behaviour. Increasing the percentage 2F non-targets, that have to be ignored, was expected to result in increasingly faster search, since it decreases the size of 1F set that has to be searched. Observers searched large displays for a target in the 1F set with a variable percentage of 2F non-targets. Interestingly, when the search displays contained 5% 2F non-targets, the search time was longer compared to the search time in other conditions. This effect of 2F non-targets on performance was independent of set size. An inspection of the saccades revealed that saccade target selection did not contribute to the longer search times in displays with 5% 2F non-targets. Occurrence of longer search times in displays containing 5% 2F non-targets might be attributed to covert processes related to visual analysis of the fixated part of the display. Apparently, visual search performance critically depends on the percentage of irrelevant 2F non-targets.

Predator perception and the interrelation between different forms of protective coloration

Animals possess a range of defensive markings to reduce the risk of predation, including warning colours, camouflage, eyespots and mimicry. These different strategies are frequently considered independently, and with little regard towards predator vision, even though they may be linked in various ways and can be fully understood only in terms of predator perception. For example, camouflage and warning coloration need not be mutually exclusive, and may frequently exploit similar features of visual perception. This paper outlines how different forms of protective markings can be understood from predator perception and illustrates how this is fundamental in determining the mechanisms underlying, and the interrelation between, different strategies. Suggestions are made for future work, and potential mechanisms discussed in relation to various forms of defensive coloration, including disruptive coloration, eyespots, dazzle markings, motion camouflage, aposematism and mimicry.