Dynamic weighting of feature dimensions in visual search: behavioral and psychophysiological evidence

Bayesian multi-level modelling for predicting single and double feature visual search

Performance in visual search tasks is frequently summarised by "search slopes" - the additional cost in reaction time for each additional distractor. While search tasks with a shallow search slopes are termed efficient (pop-out, parallel, feature), there is no clear dichotomy between efficient and inefficient (serial, conjunction) search. Indeed, a range of search slopes are observed in empirical data. The Target Contrast Signal (TCS) Theory is a rare example of quantitative model that attempts to predict search slopes for efficient visual search. One study using the TCS framework has shown that the search slope in a double-feature search (where the target differs in both colour and shape from the distractors) can be estimated from the slopes of the associated single-feature searches. This estimation is done using a contrast combination model, and a collinear contrast integration model was shown to outperform other options. In our work, we extend TCS to a Bayesian multi-level framework. We investigate modelling using normal and shifted-lognormal distributions, and show that the latter allows for a better fit to previously published data. We run a new fully within-subjects experiment to attempt to replicate the key original findings, and show that overall, TCS does a good job of predicting the data. However, we do not replicate the finding that the collinear combination model outperforms the other contrast combination models, instead finding that it may be difficult to conclusively distinguish between them.

Adapting attentional control settings in a shape-changing environment

In rich visual environments, humans have to adjust their attentional control settings in various ways, depending on the task. Especially if the environment changes dynamically, it remains unclear how observers adapt to these changes. In two experiments (online and lab-based versions of the same task), we investigated how observers adapt their target choices while searching for color singletons among shape distractor contexts that changed over trials. The two equally colored targets had shapes that differed from each other and matched a varying number of distractors. Participants were free to select either target. The results show that participants adjusted target choices to the shape ratio of distractors: even though the task could be finished by focusing on color only, participants showed a tendency to choose targets matching with fewer distractors in shape. The time course of this adaptation showed that the regularities in the changing environment were taken into account. A Bayesian modeling approach was used to provide a fine-grained picture of how observers adapted their behavior to the changing shape ratio with three parameters: the strength of adaptation, its delay relative to the objective distractor shape ratio, and a general bias toward specific shapes. Overall, our findings highlight that systematic changes in shape, even when it is not a target-defining feature, influence how searchers adjust their attentional control settings. Furthermore, our comparison between lab-based and online assessments with this paradigm suggests that shape is a good choice as a feature dimension in adaptive choice online experiments.

The Distractor Positivity Component and the Inhibition of Distracting Stimuli

There has been a long-lasting debate about whether salient stimuli, such as uniquely colored objects, have the ability to automatically distract us. To resolve this debate, it has been suggested that salient stimuli do attract attention but that they can be suppressed to prevent distraction. Some research supporting this viewpoint has focused on a newly discovered ERP component called the distractor positivity (PD), which is thought to measure an inhibitory attentional process. This collaborative review summarizes previous research relying on this component with a specific emphasis on how the PD has been used to understand the ability to ignore distracting stimuli. In particular, we outline how the PD component has been used to gain theoretical insights about how search strategy and learning can influence distraction. We also review alternative accounts of the cognitive processes indexed by the PD component. Ultimately, we conclude that the PD component is a useful tool for understanding inhibitory processes related to distraction and may prove to be useful in other areas of study related to cognitive control.

Evidence for top-down suppression of negative features in the target feature dimension

While searching for a goal-relevant object, an internal representation of the features necessary to identify the to-be-searched-for object (i.e., target) guides attention towards visual stimuli with matching properties. Recent evidence suggests that features that negatively define a target (i.e., negative features) also bias attentional allocation through top-down suppression. Since humans usually know what to look for, it will rarely, if ever, be the case that a negative feature defines a goal-relevant object alone. Thus, to better understand the relevance of top-down suppression, our participants searched for a target conjunctively defined by a positive (e.g., a blue bar) and a negative feature (e.g., a nonred bar) with both features realized within the same dimension (color in Experiments 1, 3 and 4, orientation in Experiment 2). Experiments 1 and 2 showed that reaction times were slower if cues with a negative feature preceded the target at the same versus a different position (i.e., validly vs. invalidly cued targets), indicating suppression. In contrast, cues with a task-irrelevant different-dimension feature elicited no significant reaction time difference between validly cued and invalidly cued trials. In addition, Experiment 3 showed that while negative cues were top-down suppressed, cues with a positive feature captured attention. This finding indicated that both positive and negative features guide visual attention through capture and suppression, respectively, during the search for a target defined by the presence of one and the absence of another feature from the same dimension. However, suppression seems to apply to the negative and all nontarget features in the task-relevant dimension. This was shown in Experiment 4, in which participants suppressed cues with a task-irrelevant color similarly to cues with a negative color.

Biased Competition between Targets and Distractors Reduces Attentional Suppression: Evidence from the Positivity Posterior Contralateral and Distractor Positivity

The biased competition account claims that competition between two stimuli increases when they are close together compared with when they are far apart. The reason is that nearby stimuli are more likely to be represented in the same receptive fields, requiring top-down or bottom-up biases to resolve the ambiguity. Consistent with biased competition, previous research showed that an index of attentional enhancement, the N2pc component, was attenuated when two targets were close together. In contrast, it is unclear whether distractor processing would also be attenuated when the distractor is close to the target. To answer this question, we used the additional singleton paradigm where a target is sometimes accompanied by a more salient, but entirely irrelevant, distractor. In the conditions of interest, the distance between the target and the distractor was systematically manipulated whereas the eccentricity to central fixation was always the same. The results showed that two indices of attentional suppression, the positivity posterior contralateral and distractor positivity components, were attenuated when the distractor was close to the target. Consistent with biased competition, attentional suppression of distractors was inhibited when the distance between target and distractor was short. The reduced attentional suppression of distractors with nearby targets may contribute to the increased behavioral interference with close distractors.

Preparatory Control Against Distraction Is Not Feature-Based

Salient-but-irrelevant stimuli (distractors) co-occurring with search targets can capture attention against the observer's will. Recently, evidence has accumulated that preparatory control can prevent this misguidance of spatial attention in predictable situations. However, the underlying mechanisms have remained elusive. Most pertinent theories assume that attention is guided by specific features. This widespread theoretical claim provides several strong predictions with regard to distractor handling that are disconfirmed here: Employing electrophysiological markers of covert attentional dynamics, in three experiments, we show that distractors standing out by a feature that is categorically different from the target consistently captures attention. However, equally salient distractors standing out in a different feature dimension are effectively down-weighted, even if unpredictably swapping their defining feature with the target. This shows that preparing for a distractor's feature is neither necessary nor sufficient for successful avoidance of attentional capture. Rather, capture is prevented by preparing for the distractor's feature dimension.

Shifting Attention in Feature Space: Fast Facilitation of the To-Be-Attended Feature Is Followed by Slow Inhibition of the To-Be-Ignored Feature

In an explorative study, we investigated the time course of attentional selection shifts in feature-based attention in early visual cortex by means of steady-state visual evoked potentials (SSVEPs). To this end, we presented four flickering random dot kinematograms with red/blue, horizontal/vertical bars, respectively. Given the oscillatory nature of SSVEPs, we were able to investigate neural temporal dynamics of facilitation and inhibition/suppression when participants shifted attention either within (i.e., color to color) or between feature dimensions (i.e., color to orientation). Extending a previous study of our laboratory [Müller, M. M., Trautmann, M., & Keitel, C. Early visual cortex dynamics during top-down modulated shifts of feature-selective attention. Journal of Cognitive Neuroscience, 28, 643-655, 2016] to a full factorial design, we replicated a critical finding of our previous study: Facilitation of color was quickest, regardless of the origin of the shift (from color or orientation). Furthermore, facilitation of the newly to-be-attended and inhibition/suppression of the then to-be-ignored feature is not a time-invariant process that occurs instantaneously, but a biphasic one with longer time delays between the two processes. Interestingly, inhibition/suppression of the to-be-ignored feature after the shifting cue had a much longer latency with between- compared to within-dimensional shifts (by about 130-150 msec). The exploratory nature of our study is reasoned by two limiting factors: (a) Identical to our precursor study, we found no attentional SSVEP amplitude time course modulation for orientation, and (b) the signal-to-noise ratio for single trials was too poor to allow for reliable statistical testing of the latencies that were obtained with running t tests of averaged data.

Flexible weighting of target features based on distractor context

Models of attention posit that attentional priority is established by summing the saliency and relevancy signals from feature-selective maps. The dimension-weighting account further hypothesizes that information from each feature-selective map is weighted based on expectations of how informative each dimension will be. In the current studies, we investigated the question of whether attentional biases to the features of a conjunction target (color and orientation) differ when one dimension is expected to be more diagnostic of the target. In a series of color-orientation conjunction search tasks, observers saw an exact cue for the upcoming target, while the probability of distractors sharing a target feature in each dimension was manipulated. In one context, distractors were more likely to share the target color, and in another, distractors were more likely to share the target orientation. The results indicated that despite an overall bias toward color, attentional priority to each target feature was flexibly adjusted according to distractor context: RT and accuracy performance was better when the diagnostic feature was expected than unexpected. This occurred both when the distractor context was learned implicitly and explicitly. These results suggest that feature-based enhancement can occur selectively for the dimension expected to be most informative in distinguishing the target from distractors.

Which search are you on? Adapting to color while searching for shape

Human observers adjust their attentional control settings when searching for a target in the presence of predictable changes in the target-defining feature dimension. We investigated whether observers also adapt to changes in a nondefining target dimension. According to feature integration theory, stimuli that are unique in their environment in a single feature dimension can be detected with little effort. In two experiments, we studied how observers searching for such singletons adapt their attentional control settings to a dynamical change in a nondefining target dimension. Participants searched for a shape singleton and freely chose between two targets in each trial. The two targets differed in color, and the ratio of distractors colored like each target varied dynamically across trials. A model-based analysis with a Bayesian estimation approach showed that participants adapted their target choices to the color ratio: They tended to select the target from the smaller color subset, and switched their preference both when the color ratio changed between gray and heterogeneous colors (Exp. 1) and when it changed between red and blue (Exp. 2). Participants thus tuned their attentional control settings toward color, although the target was defined by shape. We concluded that observers spontaneously adapted their behavior to changing regularities in the environment. Because adaptation was more pronounced when color homogeneity allowed for element grouping, we suggest that observers adapt to regularities that can be registered without attentional resources. They do so even if the changes are not relevant for accomplishing the task-a process presumably based on statistical learning.

Reward-predicting distractor orientations support contextual cueing: Persistent effects in homogeneous distractor contexts

Recent work on contextual cueing has shown that reward can facilitate context learning, e.g., when salient but task-irrelevant context features signal reward magnitude. Whether task-relevant context features yield a similar facilitating effect is unclear. Also, emergence and persistence of context learning for contexts associated with different reward magnitudes remains unclear. The present study investigated whether reward increases the speed with which context learning emerges, resulting in an earlier but asymptotically similar contextual cueing effect, or whether reward persistently increases context learning, visible as a larger contextual cueing effect on an asymptotical level. Reward was associated to the predominant orientation of the L-distractors, and the number of context repetitions was increased considerably. Results showed contextual cueing, i.e., faster responses and fewer fixations in repeated compared to novel contexts for all reward magnitudes. Moreover, a high reward led to a more pronounced contextual cueing effect. We developed a model-based approach to explicitly assess the non-linear decline and asymptotic level of the response time curves and to quantify how they were altered by reward. A hierarchical Bayesian parameter estimation revealed that reward decreased the asymptotic level of the repeated contexts' response times. Our results therefore show that reward leads to a persistent advantage in contextual cueing rather than to earlier but asymptotically similar context learning.

Biasing Allocations of Attention via Selective Weighting of Saliency Signals: Behavioral and Neuroimaging Evidence for the Dimension-Weighting Account

Objects that stand out from the environment tend to be of behavioral relevance, and the visual system is tuned to preferably process these salient objects by allocating focused attention. However, attention is not just passively (bottom-up) driven by stimulus features, but previous experiences and task goals exert strong biases toward attending or actively ignoring salient objects. The core and eponymous assumption of the dimension-weighting account (DWA) is that these top-down biases are not as flexible as one would like them to be; rather, they are subject to dimensional constraints. In particular, DWA assumes that people can often not search for objects that have a particular feature but only for objects that stand out from the environment (i.e., that are salient) in a particular feature dimension. We review behavioral and neuroimaging evidence for such dimensional constraints in three areas: search history, voluntary target enhancement, and distractor handling. The first two have been the focus of research on DWA since its inception and the latter the subject of our more recent research. Additionally, we discuss various challenges to the DWA and its relation to other prominent theories on top-down influences in visual search.

Change perception and change interference within and across feature dimensions

The ability to perceive a change in a visual object is reduced when that change is presented in competition with other changes which are task-irrelevant. We performed two experiments which investigate the basis of this change interference effect. We tested whether change interference occurs as a consequence of some form of attentional capture, or whether the interference occurs at a stage prior to attentional selection of the task-relevant change. A modified probe-detection task was used to explore this issue. Observers were required to report the presence/absence of a specified change-type (colour, shape) in the probe, in a context in which - on certain trials - irrelevant changes occur in non-probe items. There were two key variables in these experiments: the attentional state of the observer, and the dimensional congruence of changes in the probe and non-probe items. Change interference was strongest when the irrelevant changes were the same as those on the report dimension. However the interference pattern persisted even when observers did not know the report dimension at the time the changes occurred. These results seem to rule out attention as a factor. Our results fit best with an interpretation in which change interference produces feature-specific sensory noise which degrades the signal quality of the target change.

Dimension-based attention in visual short-term memory

We investigated how dimension-based attention influences visual short-term memory (VSTM). This was done through examining the effects of cueing a feature dimension in two perceptual comparison tasks (change detection and sameness detection). In both tasks, a memory array and a test array consisting of a number of colored shapes were presented successively, interleaved by a blank interstimulus interval (ISI). In Experiment 1 (change detection), the critical event was a feature change in one item across the memory and test arrays. In Experiment 2 (sameness detection), the critical event was the absence of a feature change in one item across the two arrays. Auditory cues indicated the feature dimension (color or shape) of the critical event with 80 % validity; the cues were presented either prior to the memory array, during the ISI, or simultaneously with the test array. In Experiment 1, the cue validity influenced sensitivity only when the cue was given at the earliest position; in Experiment 2, the cue validity influenced sensitivity at all three cue positions. We attributed the greater effectiveness of top-down guidance by cues in the sameness detection task to the more active nature of the comparison process required to detect sameness events (Hyun, Woodman, Vogel, Hollingworth, & Luck, Journal of Experimental Psychology: Human Perception and Performance, 35; 1140-1160, 2009).

Retro-dimension-cue benefit in visual working memory

In visual working memory (VWM) tasks, participants’ performance can be improved by a retro-object-cue. However, previous studies have not investigated whether participants’ performance can also be improved by a retro-dimension-cue. Three experiments investigated this issue. We used a recall task with a retro-dimension-cue in all experiments. In Experiment 1, we found benefits from retro-dimension-cues compared to neutral cues. This retro-dimension-cue benefit is reflected in an increased probability of reporting the target, but not in the probability of reporting the non-target, as well as increased precision with which this item is remembered. Experiment 2 replicated the retro-dimension-cue benefit and showed that the length of the blank interval after the cue disappeared did not influence recall performance. Experiment 3 replicated the results of Experiment 2 with a lower memory load. Our studies provide evidence that there is a robust retro-dimension-cue benefit in VWM. Participants can use internal attention to flexibly allocate cognitive resources to a particular dimension of memory representations. The results also support the feature-based storing hypothesis.

From features to dimensions: cognitive and motor development in pop-out search in children and young adults

In an experiment involving a total of 124 participants, divided into eight age groups (6-, 8-, 10-, 12-, 14-, 16-, 18-, and 20-year-olds) the development of the processing components underlying visual search for pop-out targets was tracked. Participants indicated the presence or absence of color or orientation feature singleton targets. Observers also solved a detection task, in which they responded to the onset of search arrays. There were two main results. First, analyses of inter-trial effects revealed differences in the search strategies of the 6-year-old participants compared to older age groups. Participants older than 8 years based target detection on feature-less dimensional salience signals (indicated by cross-trial RT costs in target dimension change relative to repetition trials), the 6-year-olds accessed the target feature to make a target present or absent decision (cross-trial RT costs in target feature change relative to feature repetition trials). The result agrees with predictions derived from the Dimension Weighting account and previous investigations of inter-trial effects in adult observers (Müller et al., 1995; Found and Müller, 1996). The results are also in line with theories of cognitive development suggesting that the ability to abstract specific visual features into feature categories is developed after the age of 7 years. Second, overall search RTs decreased with increasing age in a decelerated fashion. RT differences between consecutive age groups can be explained by sensory-motor maturation up to the age of 10 years (as indicated by RTs in the onset detection task). Expedited RTs in older age groups (10-, vs. 12-year-olds; 14- vs. 16-year-olds), but also in the 6- vs. 8-year-olds, are due to the development of search-related (cognitive) processes. Overall, the results suggest that the level of adult performance in visual search for pop-out targets is achieved by the age of 16.

Task probability and report of feature information: what you know about what you 'see' depends on what you expect to need

We investigated the influence of dimensional set on report of object feature information using an immediate memory probe task. Participants viewed displays containing up to 36 coloured geometric shapes which were presented for several hundred milliseconds before one item was abruptly occluded by a probe. A cue presented simultaneously with the probe instructed participants to report either about the colour or shape of the probe item. A dimensional set towards the colour or shape of the presented items was induced by manipulating task probability - the relative probability with which the two feature dimensions required report. This was done across two participant groups: One group was given trials where there was a higher report probability of colour, the other a higher report probability of shape. Two experiments showed that features were reported most accurately when they were of high task probability, though in both cases the effect was largely driven by the colour dimension. Importantly the task probability effect did not interact with display set size. This is interpreted as tentative evidence that this manipulation influences feature processing in a global manner and at a stage prior to visual short term memory.