Determinants of Global Color-Based Selection in Human Visual Cortex

Parallel gain modulation mechanisms set the resolution of color selectivity in human visual cortex

Color discrimination is fundamental to human behavior. We find bananas by coarsely searching for yellow but then differentiate nuances of yellow to pick the best exemplars. How does the brain adjust the resolution of color selectivity to our changing needs? Here, we analyze the brain magnetic response in the human visual cortex to show that color selectivity is adaptively set by coarse- and fine-resolving processes running in parallel at different hierarchical levels. Those include a gain enhancement in the higher-lever cortex of color units tuned away from the target to resolve very similar colors and a coarsely resolving gain enhancement in the mid-level cortex of units tuned to the target. Our findings suggest that attention operates on a form of multiresolution representation of color at different levels in the visual hierarchy, which keeps selectivity adaptive to a changing resolution context.

The time course of feature-selective attention inside and outside the focus of spatial attention

Research on attentional selection of stimulus features has yielded seemingly contradictory results. On the one hand, many experiments in humans and animals have observed a "global" facilitation of attended features across the entire visual field, even when spatial attention is focused on a single location. On the other hand, several event-related potential studies in humans reported that attended features are enhanced at the attended location only. The present experiment demonstrates that these conflicting results can be explained by differences in the timing of attentional allocation inside and outside the spatial focus of attention. Participants attended to fields of either red or blue randomly moving dots on either the left or right side of fixation with the task of detecting brief coherent motion targets. Recordings of steady-state visual evoked potentials elicited by the flickering stimuli allowed concurrent measurement of the time course of feature-selective attention in visual cortex on both the attended and the unattended sides. The onset of feature-selective attentional modulation on the attended side occurred around 150 ms earlier than on the unattended side. This finding that feature-selective attention is not spatially global from the outset but extends to unattended locations after a temporal delay resolves previous contradictions between studies finding global versus hierarchical selection of features and provides insight into the fundamental relationship between feature-based and location-based (spatial) attention mechanisms.

Serial attentional resource allocation during parallel feature value tracking

The visual system has evolved the ability to track features like color and orientation in parallel. This property aligns with the specialization of processing these feature dimensions in the visual cortex. But what if we ask to track changing feature-values within the same feature dimension? Parallel tracking would then have to share the same cortical representation, which would set strong limitations on tracking performance. We address this question by measuring the precision of color representations when human observers track the color of two superimposed dot clouds that simultaneously change color along independent trajectories in color-space. We find that tracking precision is highly imbalanced between streams and that tracking precision changes over time by alternating between streams at a rate of ~1 Hz. These observations suggest that, while parallel color tracking is possible, it is highly limited, essentially allowing for only one color-stream to be tracked with precision at a given time.

Early sensory gain control is dominated by obligatory and global feature-based attention in top-down shifts of combined spatial and feature-based attention

What are the dynamics of global feature-based and spatial attention, when deployed together? In an attentional shifting experiment, flanked by three control experiments, we investigated neural temporal dynamics of combined attentional shifts. For this purpose, orange- and blue-frequency-tagged spatially overlapping Random Dot Kinematograms were presented in the left and right visual hemifield to elicit continuous steady-state-visual-evoked-potentials. After being initially engaged in a fixation cross task, participants were at some point in time cued to shift attention to one of the Random Dot Kinematograms, to detect and respond to brief coherent motion events, while ignoring all such events in other Random Dot Kinematograms. The analysis of steady-state visual-evoked potentials allowed us to map time courses and dynamics of early sensory-gain modulations by attention. This revealed a time-invariant amplification of the to-be attended color both at the attended and the unattended side, followed by suppression for the to-be-ignored color at attended and unattended sides. Across all experiments, global and obligatory feature-based selection dominated early sensory gain modulations, whereas spatial attention played a minor modulatory role. However, analyses of behavior and neural markers such as alpha-band activity and event-related potentials to target- and distractor-event processing, revealed clear modulations by spatial attention.

A cortical zoom-in operation underlies covert shifts of visual spatial attention

Shifting the focus of attention without moving the eyes poses challenges for signal coding in visual cortex in terms of spatial resolution, signal routing, and cross-talk. Little is known how these problems are solved during focus shifts. Here, we analyze the spatiotemporal dynamic of neuromagnetic activity in human visual cortex as a function of the size and number of focus shifts in visual search. We find that large shifts elicit activity modulations progressing from highest (IT) through mid-level (V4) to lowest hierarchical levels (V1). Smaller shifts cause those modulations to start at lower levels in the hierarchy. Successive shifts involve repeated backward progressions through the hierarchy. We conclude that covert focus shifts arise from a cortical coarse-to-fine process progressing from retinotopic areas with larger toward areas with smaller receptive fields. This process localizes the target and increases the spatial resolution of selection, which resolves the above issues of cortical coding.

Global attentional selection of visual features is not associated with selective modulation of posterior alpha-band activity

Attending to a single feature, such as color or motion, leads to global modulation of neural processing associated with the representation of the attended features. Alpha-band modulations are hypothesized to be a marker (and even a mechanism) of the modulation of neural processing. By adopting a previously used attentional shifting paradigm, we examined whether alpha-band dynamics are linked to sustained Feature-Based-Attentional (FBA) selection. For this purpose, we presented task-irrelevant flickering random dot kinematograms (RDKs) in the periphery that either did or did not share the to-be-attended color of centrally presented task-relevant RDKs and should thus be subject to global FBA selection. Steady-state visual evoked potentials (SSVEPs) and alpha-band activity associated with these task-irrelevant RDKs were analyzed to quantify FBA modulation. Overall, the SSVEP results replicated previous findings: relative to a pre-cue baseline, SSVEP amplitudes for peripheral RDKs were significantly enhanced when these RDKs shared the to-be-attended color of the central RDKs and were not modulated when they shared the centrally to-be-ignored color. Nevertheless, there were no differences in alpha-band amplitude modulations between signals recorded contralateral to the RDKs sharing the centrally attended color and RDKs sharing the centrally ignored color. Hence, alpha-band modulations seem not to index the sustained global selection of attended over unattended feature values within the same feature dimension.

Updating the dual-mechanism model for cross-sensory attentional spreading: The influence of space-based visual selective attention

Selective attention to visual stimuli can spread cross‐modally to task‐irrelevant auditory stimuli through either the stimulus‐driven binding mechanism or the representation‐driven priming mechanism. The stimulus‐driven attentional spreading occurs whenever a task‐irrelevant sound is delivered simultaneously with a spatially attended visual stimulus, whereas the representation‐driven attentional spreading occurs only when the object representation of the sound is congruent with that of the to‐be‐attended visual object. The current study recorded event‐related potentials in a space‐selective visual object‐recognition task to examine the exact roles of space‐based visual selective attention in both the stimulus‐driven and representation‐driven cross‐modal attentional spreading, which remain controversial in the literature. Our results yielded that the representation‐driven auditory Nd component (200–400 ms after sound onset) did not differ according to whether the peripheral visual representations of audiovisual target objects were spatially attended or not, but was decreased when the auditory representations of target objects were presented alone. In contrast, the stimulus‐driven auditory Nd component (200–300 ms) was decreased but still prominent when the peripheral visual constituents of audiovisual nontarget objects were spatially unattended. These findings demonstrate not only that the representation‐driven attentional spreading is independent of space‐based visual selective attention and benefits in an all‐or‐nothing manner from object‐based visual selection for actually presented visual representations of target objects, but also that although the stimulus‐driven attentional spreading is modulated by space‐based visual selective attention, attending to visual modality per se is more likely to be the endogenous determinant of the stimulus‐driven attentional spreading.

Attention expedites target selection by prioritizing the neural processing of distractor features

Whether doing the shopping, or driving the car – to navigate daily life, our brain has to rapidly identify relevant color signals among distracting ones. Despite a wealth of research, how color attention is dynamically adjusted is little understood. Previous studies suggest that the speed of feature attention depends on the time it takes to enhance the neural gain of cortical units tuned to the attended feature. To test this idea, we had human participants switch their attention on the fly between unpredicted target color alternatives, while recording the electromagnetic brain response to probes matching the target, a non-target, or a distracting alternative target color. Paradoxically, we observed a temporally prioritized processing of distractor colors. A larger neural modulation for the distractor followed by its stronger attenuation expedited target identification. Our results suggest that dynamic adjustments of feature attention involve the temporally prioritized processing and elimination of distracting feature representations.

In order to investigate underlying mechanisms of color attention, Bartsch et al measured electromagnetic brain responses in participants who were challenged to switch their attention in accordance with unpredicted target colors changes in the absence or presence of ‘distractor colors’. They demonstrated that dynamic adjustments of feature attention involve the temporally prioritized processing and elimination of distracting feature representations.

Attentional capture by a color singleton is stronger at spatially relevant than irrelevant locations: Evidence from an ERP study

Top-down spatial attention can modulate contingent attentional capture, but the underlying mechanism is still not clear. Using variants of spatial cueing paradigms, our previous event-related potential study showed that peripheral color singleton cues with task-relevant features captured attention (indexed by cue-elicited N2pc) even when the targets appeared at central locations, but the magnitude of attentional capture was smaller than when the targets appeared at same peripheral locations. One reasonable explanation is that the modulation effect is due to spatial relevance of cues. However, several other confounding factors might also explain the modulation effect, such as task difficulty, spread of attentional window, and inside/outside relation between cue and attentional window. In the present study, we rearranged the relative locations between cues and targets to control these factors and to further examine whether inequivalence of attentional capture across attended and unattended locations was a common phenomenon. In two experiments, color singleton cues elicited apparent N2pc components when participants were searching for targets possessing the same color, which replicated typical findings of contingent attentional capture. More importantly, the modulation effect of top-down spatial attention on the cue-elicited N2pc still existed even when the factors of task difficulty (Experiment 1), spread of attentional window (Experiment 1), or inside/outside relation between cue and attentional window (Experiment 2) were controlled. These results consistently demonstrate that attentional capture by a color singleton is stronger at spatially relevant locations than at spatially irrelevant locations, suggesting an important role of spatial relevance on contingent attentional capture.

Dissociating Reward- and Attention-driven Biasing of Global Feature-based Selection in Human Visual Cortex

Objects that promise rewards are prioritized for visual selection. The way this prioritization shapes sensory processing in visual cortex, however, is debated. It has been suggested that rewards motivate stronger attentional focusing, resulting in a modulation of sensory selection in early visual cortex. An open question is whether those reward-driven modulations would be independent of similar modulations indexing the selection of attended features that are not associated with reward. Here, we use magnetoencephalography in human observers to investigate whether the modulations indexing global color-based selection in visual cortex are separable for target- and (monetary) reward-defining colors. To assess the underlying global color-based activity modulation, we compare the event-related magnetic field response elicited by a color probe in the unattended hemifield drawn either in the target color, the reward color, both colors, or a neutral task-irrelevant color. To test whether target and reward relevance trigger separable modulations, we manipulate attention demands on target selection while keeping reward-defining experimental parameters constant. Replicating previous observations, we find that reward and target relevance produce almost indistinguishable gain modulations in ventral extratriate cortex contralateral to the unattended color probe. Importantly, increasing attention demands on target discrimination increases the response to the target-defining color, whereas the response to the rewarded color remains largely unchanged. These observations indicate that, although task relevance and reward influence the very same feature-selective area in extrastriate visual cortex, the associated modulations are largely independent.

Enhanced spatial focusing increases feature-based selection in unattended locations

Attention is a multifaceted phenomenon, which operates on features (e.g., colour or motion) and over space. A fundamental question is whether the attentional selection of features is confined to the spatially-attended location or operates independently across the entire visual field (global feature-based attention, GFBA). Studies providing evidence for GFBA often employ feature probes presented at spatially unattended locations, which elicit enhanced brain responses when they match a currently-attended target feature. However, the validity of this interpretation relies on consistent spatial focusing onto the target. If the probe were to temporarily attract spatial attention, the reported effects could reflect transient spatial selection processes, rather than GFBA. Here, using magnetoencephalographic recordings (MEG) in humans, we manipulate the strength and consistency of spatial focusing to the target by increasing the target discrimination difficulty (Experiment 1), and by demarcating the upcoming target’s location with a placeholder (Experiment 2), to see if GFBA effects are preserved. We observe that motivating stronger spatial focusing to the target did not diminish the effects of GFBA. Instead, aiding spatial pre-focusing with a placeholder enhanced the feature response at unattended locations. Our findings confirm that feature selection effects measured with spatially-unattended probes reflect a true location-independent neural bias.

Attending to What and Where: Background Connectivity Integrates Categorical and Spatial Attention

Top-down attention prioritizes the processing of goal-relevant information throughout visual cortex based on where that information is found in space and what it looks like. Whereas attentional goals often have both spatial and featural components, most research on the neural basis of attention has examined these components separately. Here we investigated how these attentional components are integrated by examining the attentional modulation of functional connectivity between visual areas with different selectivity. Specifically, we used fMRI to measure temporal correlations between spatially selective regions of early visual cortex and category-selective regions in ventral temporal cortex while participants performed a task that benefitted from both spatial and categorical attention. We found that categorical attention modulated the connectivity of category-selective areas, but only with retinotopic areas that coded for the spatially attended location. Similarly, spatial attention modulated the connectivity of retinotopic areas only with the areas coding for the attended category. This pattern of results suggests that attentional modulation of connectivity is driven both by spatial selection and featural biases. Combined with exploratory analyses of frontoparietal areas that track these changes in connectivity among visual areas, this study begins to shed light on how different components of attention are integrated in support of more complex behavioral goals.

Attention to Color Sharpens Neural Population Tuning via Feedback Processing in the Human Visual Cortex Hierarchy

Attention can facilitate the selection of elementary object features such as color, orientation, or motion. This is referred to as feature-based attention and it is commonly attributed to a modulation of the gain and tuning of feature-selective units in visual cortex. Although gain mechanisms are well characterized, little is known about the cortical processes underlying the sharpening of feature selectivity. Here, we show with high-resolution magnetoencephalography in human observers (men and women) that sharpened selectivity for a particular color arises from feedback processing in the human visual cortex hierarchy. To assess color selectivity, we analyze the response to a color probe that varies in color distance from an attended color target. We find that attention causes an initial gain enhancement in anterior ventral extrastriate cortex that is coarsely selective for the target color and transitions within ∼100 ms into a sharper tuned profile in more posterior ventral occipital cortex. We conclude that attention sharpens selectivity over time by attenuating the response at lower levels of the cortical hierarchy to color values neighboring the target in color space. These observations support computational models proposing that attention tunes feature selectivity in visual cortex through backward-propagating attenuation of units less tuned to the target.SIGNIFICANCE STATEMENT Whether searching for your car, a particular item of clothing, or just obeying traffic lights, in everyday life, we must select items based on color. But how does attention allow us to select a specific color? Here, we use high spatiotemporal resolution neuromagnetic recordings to examine how color selectivity emerges in the human brain. We find that color selectivity evolves as a coarse to fine process from higher to lower levels within the visual cortex hierarchy. Our observations support computational models proposing that feature selectivity increases over time by attenuating the responses of less-selective cells in lower-level brain areas. These data emphasize that color perception involves multiple areas across a hierarchy of regions, interacting with each other in a complex, recursive manner.

The Rapid Capture of Attention by Rewarded Objects

When a stimulus is associated with a reward, it becomes prioritized, and the allocation of attention to that stimulus increases. For low-level features, such as color, this reward-based allocation of attention can manifest early in time and as a faster and stronger shift of attention to targets with that color, as reflected by the N2pc (a parieto-occipital electrophysiological component peaking at ∼250 msec). It is unknown, however, if reward associations can similarly modulate attentional shifts to complex objects or object categories, or if reward-related modulation of attentional allocation to such stimuli would occur later in time or through a different mechanism. Here, we used magnetoencephalographic recordings in 24 participants to investigate how object categories with a reward association would modulate the shift of attention. On each trial, two colored squares were presented, one in a target color and the other in a distractor color, each with an embedded object. Participants searched for the target-colored square and performed a corner discrimination task. The embedded objects were from either a rewarded or non-rewarded category, and if a rewarded-category object were present within the target-colored square, participants could earn extra money for correct performance. We observed that when the target color contained an object from a rewarded versus a non-rewarded category, the neural shift of attention to the target was faster and of greater magnitude, although the rewarded objects were not relevant for correct task performance. These results suggest that reward associations of complex objects can rapidly modulate attentional allocation to a target.

Colour expectations during object perception are associated with early and late modulations of electrophysiological activity

It is well known that visual expectation and attention modulate object perception. Yet, the mechanisms underlying these top-down influences are not completely understood. Event-related potentials (ERPs) indicate late contributions of expectations to object processing around the P2 or N2. This is true independent of whether people expect objects (vs. no objects) or specific shapes, hence when expectations pertain to complex visual features. However, object perception can also benefit from expecting colour information, which can facilitate figure/ground segregation. Studies on attention to colour show attention-sensitive modulations of the P1, but are limited to simple transient detection paradigms. The aim of the current study was to examine whether expecting simple features (colour information) during challenging object perception tasks produce early or late ERP modulations. We told participants to expect an object defined by predominantly black or white lines that were embedded in random arrays of distractor lines and then asked them to report the object's shape. Performance was better when colour expectations were met. ERPs revealed early and late phases of modulation. An early modulation at the P1/N1 transition arguably reflected earlier stages of object processing. Later modulations, at the P3, could be consistent with decisional processes. These results provide novel insights into feature-specific contributions of visual expectations to object perception.