EPS Mid-Career Award 2014

Does crowding predict conjunction search? An individual differences approach

Searching for objects in the visual environment is an integral part of human behavior. Most of the information used during such visual search comes from the periphery of our vision, and understanding the basic mechanisms of search therefore requires taking into account the inherent limitations of peripheral vision. Our previous work using an individual differences approach has shown that one of the major factors limiting peripheral vision (crowding) is predictive of single feature search, as reflected in response time and eye movement measures. Here we extended this work, by testing the relationship between crowding and visual search in a conjunction-search paradigm. Given that conjunction search involves more fine-grained discrimination and more serial behavior, we predicted it would be strongly affected by crowding. We tested sixty participants with regard to their sensitivity to both orientation and color-based crowding (as measured by critical spacing) and their efficiency in searching for a color/orientation conjunction (as indicated by manual response times and eye movements). While the correlations between the different crowding tasks were high, the correlations between the different crowding measures and search performance were relatively modest, and no higher than those previously observed for single-feature search. Instead, observers showed very strong color selectivity during search. The results suggest that conjunction search behavior relies more on top-down guidance (here by color) and is therefore relatively less determined by individual differences in sensory limitations as caused by crowding.

The Preparatory Activation of Guidance Templates for Visual Search and of Target Templates in Non-Search Tasks

Representations of task-relevant object attributes (attentional templates) control the adaptive selectivity of visual processing. Previous studies have demonstrated that templates involved in the guidance of attention during visual search are activated in a preparatory fashion prior to the arrival of visual search displays. The current study investigated whether such proactive mechanisms are also triggered in non-search tasks, where attentional templates do not mediate the guidance of attention towards targets amongst distractors but are still necessary for subsequent target recognition processes. Participants either searched for colour-defined targets among multiple distractors or performed two other non-search tasks where imperative stimuli appeared without competing distractors (a colour-based Go/NoGo task, and a shape discrimination task where target colour was constant and could thus be ignored). Preparatory activation of colour-selective templates was tracked by measuring N2pc components (markers of attention allocation) to task-irrelevant colour singleton probes flashed every 200 ms during the interval between target displays. As expected, N2pcs were triggered by target-coloured probes in the search task, indicating that a corresponding guidance template was triggered proactively. Critically, clear probe N2pcs were also observed in the Go/NoGo task, and even in the shape discrimination task in an attenuated fashion. These findings demonstrate that the preparatory activation of feature-selective attentional task settings is not uniquely associated with the guidance of visual search but is also present in other types of visual selection tasks where guidance is not required.

Strategies, debates, and adversarial collaboration in working memory: The 51st Bartlett Lecture

Frederic Bartlett championed the importance of individual strategy differences when remembering details of events. I will describe how long-running theoretical debates in the area of working memory may be resolved by considering differences across participants in the strategies that they use when performing cognitive tasks, and through adversarial collaboration between rival laboratories. In common with the established view within experimental cognitive psychology, I assume that adults have a range of cognitive functions, evolved for everyday life. However, I will present evidence showing that these functions can be engaged selectively for laboratory tasks, and that how they are deployed may differ between and within individuals for the same task. Reliance on aggregate data, while treating inter- and intra-participant variability in data patterns as statistical noise, may lead to misleading conclusions about theoretical principles of cognition, and of working memory in particular. Moreover, different theoretical perspectives may be focused on different levels of explanation and different theoretical goals rather than being mutually incompatible. Yet researchers from contrasting theoretical frameworks pursue science as a competition, rarely do researchers from competing labs work in collaboration, and debates self-perpetuate. These approaches to research can stall debate resolution and generate ever-increasing scientific diversity rather than scientific progress. The article concludes by describing a recent extended adversarial collaboration (the WoMAAC project) focused on theoretical contrasts in working memory, and illustrates how this approach to conducting research may help resolve scientific debate and facilitate scientific advance.

Attention: a descriptive taxonomy

The term attention has been used to mean so many different things that some have despaired of it being useful at all. This paper is devoted to bringing a modicum of order to the chaos through the time-honored device of categorization. The chief purpose of this paper is to introduce a comprehensive descriptive taxonomy of the nuanced ways the term attention may be employed. It is presented in table form, followed by elucidations and illustrations of each of its items. But first, I offer reasons why a taxonomy of attention is needed and explore some of its possible benefits. After presenting the taxonomy, I sketch by way of example how it might be applied to two interesting questions: is the umbrella term attention still useful?; and, what is it that ties the subdefinitions of attention together yet distinguishes them from other kinds of cognition?

Individual differences in crowding predict visual search performance

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

Visual and central attention share a capacity limitation when the demands for serial item selection in visual search are high

Visual and central attention are limited in capacity. In conjunction search, visual attention is required to select the items and to bind their features (e.g., color, form, size), which results in a serial search process. In dual-tasks, central attention is required for response selection, but because central attention is limited in capacity, response selection can only be carried out for one task at a time. Here, we investigated whether visual and central attention rely on a common or on distinct capacity limitations. In two dual-task experiments, participants completed an auditory two-choice discrimination Task 1 and a conjunction search Task 2 that were presented with an experimentally modulated temporal interval between them (stimulus onset asynchrony [SOA]). In Experiment 1, Task 2 was a triple conjunction search task. Each item consisted of a conjunction of three features, so that target and distractors shared two features. In Experiment 2, Task 2 was a plus conjunction search task, in which target and distractors shared the same four features. The hypotheses for conjunction search time were derived from the locus-of-slack method. While plus conjunction search was performed after response selection in Task 1, a small part of triple conjunction search was still performed in parallel to response selection in Task 1. However, the between-experiment comparison was not significant, indicating that both search tasks may require central attention. Taken together, the present study provides evidence that visual and central attention share a common capacity limitation when conjunction search relies strongly on serial item selection.

Medium versus difficult visual search: How a quantitative change in the functional visual field leads to a qualitative difference in performance

The dominant theories of visual search assume that search is a process involving comparisons of individual items against a target description that is based on the properties of the target in isolation. Here, we present four experiments that demonstrate that this holds true only in difficult search. In medium search it seems that the relation between the target and neighbouring items is also part of the target description. We used two sets of oriented lines to construct the search items. The cardinal set contained horizontal and vertical lines, the diagonal set contained left diagonal and right diagonal lines. In all experiments, participants knew the identity of the target and the line set used to construct it. In difficult search this knowledge allowed performance to improve in displays where only half of the search items came from the same line set as the target (50% eligibility), relative to displays where all items did (100% eligibility). However, in medium search, performance was actually poorer for 50% eligibility, especially on target-absent trials. This opposite effect of ineligible items in medium search and difficult search is hard to reconcile with theories based on individual items. It is more in line with theories that conceive search as a sequence of fixations where the number of items processed during a fixation depends on the difficulty of the search task: When search is medium, multiple items are processed per fixation. But when search is difficult, only a single item is processed.

Attentional Access to Multiple Target Objects in Visual Search

Most investigations of visual search have focused on the discrimination between a search target and other task-irrelevant distractor objects (selection). The attentional limitations that arise when multiple target objects in the same display have to be processed simultaneously (access) remain poorly understood. Here, we employed behavioral and electrophysiological measures to investigate the factors that determine whether multiple target objects can be accessed in parallel. Performance and N2pc components were measured for search displays that contained either a single target or two target objects. When two target objects were present, they either had the same or different target-defining features. Participants reported whether search displays contained a single target, two targets with shared features, or two targets with different features. There were performance costs as well as reduced N2pc amplitudes for two-target/different relative to two-target/same displays, suggesting that access to multiple target objects defined by different features was impaired. These behavioral and electrophysiological costs were also observed in a task where all search display objects were physically different, but not during color or shape singleton search, confirming that they do not reflect a low-level perceptual grouping of physically identical targets. These results demonstrate strong feature-specific limitations of visual access, as proposed by the Boolean map theory of visual attention. They suggest that multiple target objects can be accessed in parallel only when they share task-relevant features and demonstrate that mechanisms of visual access can be studied with electrophysiological markers.

The time-course of component processes of selective attention

Attentional selection shapes human perception, enhancing relevant information, according to behavioral goals. While many studies have investigated individual neural signatures of attention, here we used multivariate decoding of electrophysiological brain responses (MEG/EEG) to track and compare multiple component processes of selective attention. Auditory cues instructed participants to select a particular visual target, embedded within a subsequent stream of displays. Combining single and multi-item displays with different types of distractors allowed multiple aspects of information content to be decoded, distinguishing distinct components of attention, as the selection process evolved. Although the task required comparison of items to an attentional "template" held in memory, signals consistent with such a template were largely undetectable throughout the preparatory period but re-emerged after presentation of a non-target choice display. Choice displays evoked strong neural representation of multiple target features, evolving over different timescales. We quantified five distinct processing operations with different time-courses. First, visual properties of the stimulus were strongly represented. Second, the candidate target was rapidly identified and localized in multi-item displays, providing the earliest evidence of modulation by behavioral relevance. Third, the identity of the target continued to be enhanced, relative to distractors. Fourth, only later was the behavioral significance of the target explicitly represented in single-item displays. Finally, if the target was not identified and search was to be resumed, then an attentional template was weakly reactivated. The observation that an item's behavioral relevance directs attention in multi-item displays prior to explicit representation of target/non-target status in single-item displays is consistent with two-stage models of attention.

Concurrent evaluation of independently cued features during perceptual decisions and saccadic targeting in visual search

Simultaneous search for one of two targets is slower and less accurate than search for a single target. Within the Signal Detection Theoretic (SDT) framework, this can be attributed to the division of resources during the comparison of visual input against independently cued targets. The current study used one or two cues to elicit single- and dual-target searches for orientation targets among similar and dissimilar distractors. In Experiment 1, the accuracy of target discrimination in brief displays was compared at setsizes of 1, 2 and 4. Results revealed a reduction in accuracy that scaled with the product of set size and the number of cued targets. In Experiment 2, the accuracy and latency of observers’ saccadic targeting were compared. Fixations on single-target searches were highly selective towards the target. On dual-target searches, the requirement to detect one of two targets produced a significant reduction in target fixations and equivalent rates of fixations to distractors with opposite orientations. For most observers, the dual-target cost was predicted by an SDT model that simulated increases in decision-noise and the distribution of capacity-limited resources during the comparison of selected input against independently cued targets. For others, search accuracy was consistent with a single-item limit on perceptual decisions and saccadic targeting during search. These findings support a flexible account of the dual-target cost based on different strategies to resolve competition between independently cued targets.

Lateralized Suppression of Alpha-Band EEG Activity As a Mechanism of Target Processing

Alpha-band (8-12 Hz) EEG activity has been linked to visual attention since the earliest EEG studies. More recent studies using spatial cuing paradigms have shown that alpha is suppressed over the hemisphere contralateral to a to-be-attended location, suggesting that alpha serves as a mechanism of preparatory attention. Here, we demonstrate that alpha also plays a role in active target processing. EEG activity was recorded from a group of healthy male and female human subjects in two visual search experiments. In addition to alpha activity, we also assessed the N2pc event-related potential component, a lateralized transient EEG response that has been tightly linked with the focusing of attention on visual targets. We found that the visual search targets triggered both an N2pc component and a suppression of alpha-band activity that was greatest over the hemisphere contralateral to the target (which we call "target-elicited lateralized alpha suppression" or TELAS). In Experiment 1, both N2pc and TELAS were observed for targets presented in the lower visual field but were absent for upper-field targets. However, these two lateralized effects had different time courses and they responded differently to manipulations of crowding in Experiment 2. These results indicate that lateralized alpha-band activity is involved in active target processing and is not solely a preparatory mechanism and also that TELAS and N2pc reflect a related but separable neural mechanism of visuospatial attention.SIGNIFICANCE STATEMENT The very first EEG studies demonstrated that alpha-band (8-12 Hz) EEG oscillations are suppressed when people attend to visual information and more recent research has shown that cuing an individual to expect a target at a specific location produces lateralized suppression in the contralateral hemisphere. Therefore, lateralized alpha may serve as a preparatory mechanism. In the present study, we found that a similar lateralized alpha effect is triggered by the appearance of a visual target even though the location could not be anticipated, demonstrating that alpha also serves as an active mechanism of target processing. Moreover, we found that alpha lateralization can be dissociated from other lateralized measures of target selection, indicating that it reflects a distinct mechanism of attention.

Effects of absolute luminance and luminance contrast on visual search in low mesopic environments

Diverse adaptive visual processing mechanisms allow us to complete visual search tasks in a wide visual photopic range (>0.6 cd/m²). Whether search strategies or mechanisms known from this range extend below, in the mesopic and scotopic luminance spectra (<0.6 cd/m²), has yet to be addressed. Based on a study that addressed simple target discrimination in luminance environments using contrast-dependent behavioral efficiency functions, we assessed visual search in more complex-feature and conjunction-search paradigms. The results verify the previously reported deficiency windows defined by an interaction of base luminance and luminance contrast for more complex visual-search tasks. Based on significant regression analyses, a more precise definition of the magnitude of contribution of different contrast parameters. Characterized feature search patterns had approximately a 2.5:1 ratio of contribution from the Michelson contrast property relative to Weber contrast, whereas the ratio was approximately 1:1 in a serial-search condition. The results implicate near-complete magnocellular isolation in a visual-search paradigm that has yet to be demonstrated. Our analyses provide a new method of characterizing visual search and the first insight in its underlying mechanisms in luminance environments in the low mesopic and scotopic spectra.

Feature-guided attentional capture cannot be prevented by spatial filtering

Feature-based control processes guide attention towards objects with target features in visual search. While these processes are assumed to operate globally across the entire visual field, it remains controversial whether target-matching objects at task-irrelevant locations can be excluded from attentional selection, especially when spatial attention is already narrowly focused elsewhere. We investigated whether probe stimuli at irrelevant lateral locations capture attention when they precede search displays where targets are defined either by a specific feature (colour or orientation) or by a colour/orientation conjunction by measuring N2pc components (an electrophysiological marker of attentional target selection) to these probes. Reliable N2pcs were triggered by probes not only in the feature search tasks but also when participants searched for feature conjunctions, in spite of the fact that conjunction search requires focal spatial attention. Analogous N2pc results were found in the absence of any spatial uncertainty about the location of conjunctively defined targets, which always appeared at fixation. These results show that rapid attentional capture by objects with target-matching features cannot be prevented by top-down spatial filtering mechanisms, and confirm that feature-based attentional guidance processes operate in a spatially global fashion.

Event-related Electroencephalographic Lateralizations Mark Individual Differences in Spatial and Nonspatial Visual Selection

Selective attention controls the distribution of our visual system's limited processing resources to stimuli in the visual field. Two independent parameters of visual selection can be quantified by modeling an individual's performance in a partial-report task based on the computational theory of visual attention (TVA): (i) top-down control α, the relative attentional weighting of relevant over irrelevant stimuli, and (ii) spatial bias w_λ, the relative attentional weighting of stimuli in the left versus right hemifield. In this study, we found that visual event-related electroencephalographic lateralizations marked interindividual differences in these two functions. First, individuals with better top-down control showed higher amplitudes of the posterior contralateral negativity than individuals with poorer top-down control. Second, differences in spatial bias were reflected in asymmetries in earlier visual event-related lateralizations depending on the hemifield position of targets; specifically, individuals showed a positivity contralateral to targets presented in their prioritized hemifield and a negativity contralateral to targets presented in their nonprioritized hemifield. Thus, our findings demonstrate that two functionally different aspects of attentional weighting quantified in the respective TVA parameters are reflected in two different neurophysiological measures: The observer-dependent spatial bias influences selection by a bottom-up processing advantage of stimuli appearing in the prioritized hemifield. By contrast, task-related target selection governed by top-down control involves active enhancement of target, and/or suppression of distractor, processing. These results confirm basic assumptions of the TVA framework, complement the functional interpretation of event-related lateralization components in selective attention studies, and are of relevance for the development of neurocognitive attentional assessment procedures.

More insight into the interplay of response selection and visual attention in dual-tasks: masked visual search and response selection are performed in parallel

Both response selection and visual attention are limited in capacity. According to the central bottleneck model, the response selection processes of two tasks in a dual-task situation are performed sequentially. In conjunction search, visual attention is required to select the items and to bind their features (e.g., color and form), which results in a serial search process. Search time increases as items are added to the search display (i.e., set size effect). When the search display is masked, visual attention deployment is restricted to a brief period of time and target detection decreases as a function of set size. Here, we investigated whether response selection and visual attention (i.e., feature binding) rely on a common or on distinct capacity limitations. In four dual-task experiments, participants completed an auditory Task 1 and a conjunction search Task 2 that were presented with an experimentally modulated temporal interval between them (Stimulus Onset Asynchrony, SOA). In Experiment 1, Task 1 was a two-choice discrimination task and the conjunction search display was not masked. In Experiment 2, the response selection difficulty in Task 1 was increased to a four-choice discrimination and the search task was the same as in Experiment 1. We applied the locus-of-slack method in both experiments to analyze conjunction search time, that is, we compared the set size effects across SOAs. Similar set size effects across SOAs (i.e., additive effects of SOA and set size) would indicate sequential processing of response selection and visual attention. However, a significantly smaller set size effect at short SOA compared to long SOA (i.e., underadditive interaction of SOA and set size) would indicate parallel processing of response selection and visual attention. In both experiments, we found underadditive interactions of SOA and set size. In Experiments 3 and 4, the conjunction search display in Task 2 was masked. Task 1 was the same as in Experiments 1 and 2, respectively. In both experiments, the d' analysis revealed that response selection did not affect target detection. Overall, Experiments 1-4 indicated that neither the response selection difficulty in the auditory Task 1 (i.e., two-choice vs. four-choice) nor the type of presentation of the search display in Task 2 (i.e., not masked vs. masked) impaired parallel processing of response selection and conjunction search. We concluded that in general, response selection and visual attention (i.e., feature binding) rely on distinct capacity limitations.

Why the item will remain the unit of attentional selection in visual search

Hulleman & Olivers (H&O) reject item-based serial models of visual search, and they suggest that items are processed equally and globally during each fixation period. However, neuroscientific studies have shown that attentional biases can emerge in parallel but in a spatially selective item-based fashion. Even within a parallel architecture for visual search, the item remains the critical unit of selection.

The Control of Single-color and Multiple-color Visual Search by Attentional Templates in Working Memory and in Long-term Memory

The question whether target selection in visual search can be effectively controlled by simultaneous attentional templates for multiple features is still under dispute. We investigated whether multiple-color attentional guidance is possible when target colors remain constant and can thus be represented in long-term memory but not when they change frequently and have to be held in working memory. Participants searched for one, two, or three possible target colors that were specified by cue displays at the start of each trial. In constant-color blocks, the same colors remained task-relevant throughout. In variable-color blocks, target colors changed between trials. The contralateral delay activity (CDA) to cue displays increased in amplitude as a function of color memory load in variable-color blocks, which indicates that cued target colors were held in working memory. In constant-color blocks, the CDA was much smaller, suggesting that color representations were primarily stored in long-term memory. N2pc components to targets were measured as a marker of attentional target selection. Target N2pcs were attenuated and delayed during multiple-color search, demonstrating less efficient attentional deployment to color-defined target objects relative to single-color search. Importantly, these costs were the same in constant-color and variable-color blocks. These results demonstrate that attentional guidance by multiple-feature as compared with single-feature templates is less efficient both when target features remain constant and can be represented in long-term memory and when they change across trials and therefore have to be maintained in working memory.

Rapid Parallel Attentional Selection Can Be Controlled by Shape and Alphanumerical Category

Previous research has shown that when two color-defined target objects appear in rapid succession at different locations, attention is deployed independently and in parallel to both targets. This study investigated whether this rapid simultaneous attentional target selection mechanism can also be employed in tasks where targets are defined by a different visual feature (shape) or when alphanumerical category is the target selection attribute. Two displays that both contained a target and a nontarget object on opposite sides were presented successively, and the SOA between the two displays was 100, 50, 20, or 10 msec in different blocks. N2pc components were recorded to both targets as a temporal marker of their attentional selection. When observers searched for shape-defined targets (Experiment 1), N2pc components to the two targets were equal in size and overlapped in time when the SOA between the two displays was short, reflecting two parallel shape-guided target selection processes with their own independent time course. Essentially the same temporal pattern of N2pc components was observed when alphanumerical category was the target-defining attribute (Experiment 2), demonstrating that the rapid parallel attentional selection of multiple target objects is not restricted to situations where the deployment of attention can be guided by elementary visual features but that these processes can even be employed in category-based attentional selection tasks. These findings have important implications for our understanding of the cognitive and neural basis of top–down attentional control.

The Role of Color in Search Templates for Real-world Target Objects

During visual search, target representations (attentional templates) control the allocation of attention to template-matching objects. The activation of new attentional templates can be prompted by verbal or pictorial target specifications. We measured the N2pc component of the ERP as a temporal marker of attentional target selection to determine the role of color signals in search templates for real-world search target objects that are set up in response to word or picture cues. On each trial run, a word cue (e.g., "apple") was followed by three search displays that contained the cued target object among three distractors. The selection of the first target was based on the word cue only, whereas selection of the two subsequent targets could be controlled by templates set up after the first visual presentation of the target (picture cue). In different trial runs, search displays either contained objects in their natural colors or monochromatic objects. These two display types were presented in different blocks (Experiment 1) or in random order within each block (Experiment 2). RTs were faster, and target N2pc components emerged earlier for the second and third display of each trial run relative to the first display, demonstrating that pictures are more effective than word cues in guiding search. N2pc components were triggered more rapidly for targets in the second and third display in trial runs with colored displays. This demonstrates that when visual target attributes are fully specified by picture cues, the additional presence of color signals in target templates facilitates the speed with which attention is allocated to template-matching objects. No such selection benefits for colored targets were found when search templates were set up in response to word cues. Experiment 2 showed that color templates activated by word cues can even impair the attentional selection of noncolored targets. Results provide new insights into the status of color during the guidance of visual search for real-world target objects. Color is a powerful guiding feature when the precise visual properties of these objects are known but seems to be less important when search targets are specified by word cues.

The impending demise of the item in visual search

The way the cognitive system scans the visual environment for relevant information - visual search in short - has been a long-standing central topic in vision science. From its inception as a research topic, and despite a number of promising alternative perspectives, the study of visual search has been governed by the assumption that a search proceeds on the basis of individual items (whether processed in parallel or not). This has led to the additional assumptions that shallow search slopes (at most a few tens of milliseconds per item for target-present trials) are most informative about the underlying process, and that eye movements are an epiphenomenon that can be safely ignored. We argue that the evidence now overwhelmingly favours an approach that takes fixations, not individual items, as its central unit. Within fixations, items are processed in parallel, and the functional field of view determines how many fixations are needed. In this type of theoretical framework, there is a direct connection between target discrimination difficulty, fixations, and reaction time (RT) measures. It therefore promises a more fundamental understanding of visual search by offering a unified account of both eye movement and manual response behaviour across the entire range of observed search efficiency, and provides new directions for research. A high-level conceptual simulation with just one free and four fixed parameters shows the viability of this approach.