Selecting and ignoring salient objects within and across dimensions in visual search

Dissociable event-related potential modulations of intrinsic and extrinsic factors in temporal integration

We investigated visual temporal integration, by which multiple stimuli appearing in rapid succession are perceived as a single event. Temporal integration not only depends intrinsically on the passage of time but also, extrinsically, on the number and distribution of successive stimuli that are presented across that time interval. Here, we used a missing element task to investigate intrinsic and extrinsic factors in temporal integration, by manipulating stimulus duration and number, respectively. We found that both contributed interactively to integration performance and that varying the information rate over time did not further modulate this pattern. Intrinsic and extrinsic factors had dissociable effects on the N1, N2, N2pc, and P3 components of the event-related potential, implicating unique contributions to perceptual discrimination, spatio-temporal grouping, attention, and response decision-making. Stimulus number-induced effects on the event-related potential also generally arose later than those of stimulus duration. The latter already modulated the amplitude of the N1 and the early phase of the N2pc, while the former did not. The collective results suggest that while both intrinsic and extrinsic factors drive temporal integration, they do so in different ways. This difference during integration may eventually be reflected in the way in which we perceive longer, episodic events.

Selection history is relative

Visual attention can be tuned to specific features to aid in visual search. The way in which these search strategies are established and maintained is flexible, reflecting goal-directed attentional control, but can exert a persistent effect on selection that remains even when these strategies are no longer advantageous, reflecting an attentional bias driven by selection history. Apart from feature-specific search, recent studies have shown that attention can be tuned to target-nontarget relationships. Here we tested whether a relational search strategy continues to bias attention in a subsequent task, where the relationally better color and former target color both serve as distractors (Experiment 1) or as potential targets (Experiment 2). We demonstrate that a relational bias can persist in a subsequent task in which color serves as a task-irrelevant feature, both impairing and facilitating visual search performance. Our findings extend our understanding of the relational account of attentional control and the nature of selection history effects on attention.

Biasing Allocations of Attention via Selective Weighting of Saliency Signals: Behavioral and Neuroimaging Evidence for the Dimension-Weighting Account

Objects that stand out from the environment tend to be of behavioral relevance, and the visual system is tuned to preferably process these salient objects by allocating focused attention. However, attention is not just passively (bottom-up) driven by stimulus features, but previous experiences and task goals exert strong biases toward attending or actively ignoring salient objects. The core and eponymous assumption of the dimension-weighting account (DWA) is that these top-down biases are not as flexible as one would like them to be; rather, they are subject to dimensional constraints. In particular, DWA assumes that people can often not search for objects that have a particular feature but only for objects that stand out from the environment (i.e., that are salient) in a particular feature dimension. We review behavioral and neuroimaging evidence for such dimensional constraints in three areas: search history, voluntary target enhancement, and distractor handling. The first two have been the focus of research on DWA since its inception and the latter the subject of our more recent research. Additionally, we discuss various challenges to the DWA and its relation to other prominent theories on top-down influences in visual search.

Forced to remember: when memory is biased by salient information

The last decades have seen a rapid growing in the attempt to understand the key factors involved in the internal memory representation of the external world. Visual salience have been found to provide a major contribution in predicting the probability for an item/object embedded in a complex setting (i.e., a natural scene) to be encoded and then remembered later on. Here I review the existing literature highlighting the impact of perceptual- (based on low-level sensory features) and semantics-related salience (based on high-level knowledge) on short-term memory representation, along with the neural mechanisms underpinning the interplay between these factors. The available evidence reveal that both perceptual- and semantics-related factors affect attention selection mechanisms during the encoding of natural scenes. Biasing internal memory representation, both perceptual and semantics factors increase the probability to remember high- to the detriment of low-saliency items. The available evidence also highlight an interplay between these factors, with a reduced impact of perceptual-related salience in biasing memory representation as a function of the increasing availability of semantics-related salient information. The neural mechanisms underpinning this interplay involve the activation of different portions of the frontoparietal attention control network. Ventral regions support the assignment of selection/encoding priorities based on high-level semantics, while the involvement of dorsal regions reflects priorities assignment based on low-level sensory features.

Attentional spreading to task-irrelevant object features: experimental support and a 3-step model of attention for object-based selection and feature-based processing modulation

Directing attention to a specific feature of an object has been linked to different forms of attentional modulation. Object-based attention theory founds on the finding that even task-irrelevant features at the selected object are subject to attentional modulation, while feature-based attention theory proposes a global processing benefit for the selected feature even at other objects. Most studies investigated either the one or the other form of attention, leaving open the possibility that both object- and feature-specific attentional effects do occur at the same time and may just represent two sides of a single attention system. We here investigate this issue by testing attentional spreading within and across objects, using reaction time (RT) measurements to changes of attended and unattended features on both attended and unattended objects. We asked subjects to report color and speed changes occurring on one of two overlapping random dot patterns (RDPs), presented at the center of gaze. The key property of the stimulation was that only one of the features (e.g., motion direction) was unique for each object, whereas the other feature (e.g., color) was shared by both. The results of two experiments show that co-selection of unattended features even occurs when those features have no means for selecting the object. At the same time, they demonstrate that this processing benefit is not restricted to the selected object but spreads to the task-irrelevant one. We conceptualize these findings by a 3-step model of attention that assumes a task-dependent top-down gain, object-specific feature selection based on task- and binding characteristics, and a global feature-specific processing enhancement. The model allows for the unification of a vast amount of experimental results into a single model, and makes various experimentally testable predictions for the interaction of object- and feature-specific processes.

Late electrophysiological modulations of feature-based attention to object shapes

Feature-based attention has been shown to aid object perception. Our previous ERP effects revealed temporally late feature-based modulation in response to objects relative to motion. The aim of the current study was to confirm the timing of feature-based influences on object perception while cueing within the feature dimension of shape. Participants were told to expect either "pillow" or "flower" objects embedded among random white and black lines. Participants more accurately reported the object's main color for valid compared to invalid shapes. ERPs revealed modulation from 252-502 ms, from occipital to frontal electrodes. Our results are consistent with previous findings examining the time course for processing similar stimuli (illusory contours). Our results provide novel insights into how attending to features of higher complexity aids object perception presumably via feed-forward and feedback mechanisms along the visual hierarchy.

Neural basis of feature-based contextual effects on visual search behavior

Searching for a visual object is known to be adaptable to context, and it is thought to result from the selection of neural representations distributed on a visual salience map, wherein stimulus-driven and goal-directed signals are combined. Here we investigated the neural basis of this adaptability by recording superior colliculus (SC) neurons while three female rhesus monkeys (Macaca mulatta) searched with saccadic eye movements for a target presented in an array of visual stimuli whose feature composition varied from trial to trial. We found that sensory-motor activity associated with distracters was enhanced or suppressed depending on the search array composition and that it corresponded to the monkey's search strategy, as assessed by the distribution of the occasional errant saccades. This feature-related modulation occurred independently from the saccade goal and facilitated the process of saccade target selection. We also observed feature-related enhancement in the activity associated with distracters that had been the search target during the previous session. Consistent with recurrent processing, both feature-related neuronal modulations occurred more than 60 ms after the onset of the visually evoked responses, and their near coincidence with the time of saccade target selection suggests that they are integral to this process. These results suggest that SC neuronal activity is shaped by the visual context as dictated by both stimulus-driven and goal-directed signals. Given the close proximity of the SC to the motor circuit, our findings suggest a direct link between perception and action and no need for distinct salience and motor maps.

The timing of feature-based attentional effects during object perception

Allocating attention to basic features such as colour enhances perception of the respective features throughout the visual field. We have previously shown that feature-based attention also plays a role for more complex features required for object perception. To investigate at which level object perception is modulated by feature-based attention we recorded high-density event-related potentials (ERPs). Participants detected contour-defined objects or motion, and were informed to expect each feature dimension. Participants perceived contour-defined objects and motion better when they expected the congruent feature. This is consistent with modulation of the P1 when attending to lower-level features. For contours, modulation occurred at 290 ms, first at frontal electrodes and then at posterior sites, associated with sources in ventral visual areas accompanied by greater signal strength. This pattern of results is consistent with what has been observed in response to illusory contours. Our data provide novel insights into the contribution of feature-based attention to object perception that are associated with higher tier brain areas.

Capture versus suppression of attention by salient singletons: electrophysiological evidence for an automatic attend-to-me signal

There is considerable controversy about whether salient singletons capture attention in a bottom-up fashion, irrespective of top-down control settings. One possibility is that salient singletons always generate an attention capture signal, but this signal can be actively suppressed to avoid capture. In the present study, we investigated this issue by using event-related potential recordings, focusing on N2pc (N2-posterior-contralateral; a measure of attentional deployment) and Pd (distractor positivity; a measure of attentional suppression). Participants searched for a specific letter within one of two regions, and irrelevant color singletons were sometimes present. We found that the irrelevant singletons did not elicit N2pc but instead elicited Pd; this occurred equally within the attended and unattended regions. These findings suggest that salient singletons may automatically produce an attend-to-me signal, irrespective of top-down control settings, but this signal can be overridden by an active suppression process to prevent the actual capture of attention.

Electrophysiological correlates of detecting a visual target and detecting its absence: the role of feature dimensions

Electrophysiological measurements were taken from observers performing a visual search within a single feature dimension, and between multiple dimensions. The N2pc to selected singleton target stimuli (rightward tilted lines) was increased when targets varied between feature dimensions, compared to the N2pc to the same rightward tilted line targets, in a condition in which targets varied only between feature values within the same dimension. The anterior and posterior N2 were not reliably modulated, but P3(b) amplitude was higher for singleton present trials that varied between dimensions than for those that varied within. The ERP elicited by singleton absent trials showed reduced P3 amplitude in the between-dimension condition. The electrophysiological modulations were accompanied by increased reaction times in the between-dimension condition, on both singleton present and absent trials. The results suggested that visual target detection is affected by early dimension-specific weighting of the current attentional task set. Furthermore, exhaustively searching multiple feature dimensions to determine the absence of a target incurs dimensional switching costs, possibly at a later stage.