Having More Choices Changes How Human Observers Weight Stable Sensory Evidence

Conflicting Sensory Information Sharpens the Neural Representations of Early Selective Visuospatial Attention

Adaptive behaviors require the ability to resolve conflicting information caused by the processing of incompatible sensory inputs. Prominent theories of attention have posited that early selective attention helps mitigate cognitive interference caused by conflicting sensory information by facilitating the processing of task-relevant sensory inputs and filtering out behaviorally irrelevant information. Surprisingly, many recent studies that investigated the role of early selective attention on conflict mitigation have failed to provide positive evidence. Here, we examined changes in the selectivity of early visuospatial attention in male and female human subjects performing an attention-cueing Eriksen flanker task, where they discriminated the shape of a visual target surrounded by congruent or incongruent distractors. We used the inverted encoding model to reconstruct spatial representations of visual selective attention from the topographical patterns of amplitude modulations in alpha band oscillations in scalp EEG (∼8-12 Hz). We found that the fidelity of the alpha-based spatial reconstruction was significantly higher in the incongruent compared with the congruent condition. Importantly, these conflict-related modulations in the reconstruction fidelity occurred at a much earlier time window than those of the lateralized posterior event-related potentials associated with target selection and distractor suppression processes, as well as conflict-related modulations in the frontocentral negative-going wave and midline-frontal theta oscillations (∼3-7 Hz), thought to track executive control functions. Taken together, our data suggest that conflict resolution is supported by the cascade of neural processes underlying early selective visuospatial attention and frontal executive functions that unfold over time.

Sustained attention operates via dissociable neural mechanisms across different eccentric locations

In primates, foveal and peripheral vision have distinct neural architectures and functions. However, it has been debated if selective attention operates via the same or different neural mechanisms across eccentricities. We tested these alternative accounts by examining the effects of selective attention on the steady-state visually evoked potential (SSVEP) and the fronto-parietal signal measured via EEG from human subjects performing a sustained visuospatial attention task. With a negligible level of eye movements, both SSVEP and SND exhibited the heterogeneous patterns of attentional modulations across eccentricities. Specifically, the attentional modulations of these signals peaked at the parafoveal locations and such modulations wore off as visual stimuli appeared closer to the fovea or further away towards the periphery. However, with a relatively higher level of eye movements, the heterogeneous patterns of attentional modulations of these neural signals were less robust. These data demonstrate that the top-down influence of covert visuospatial attention on early sensory processing in human cortex depends on eccentricity and the level of saccadic responses. Taken together, the results suggest that sustained visuospatial attention operates differently across different eccentric locations, providing new understanding of how attention augments sensory representations regardless of where the attended stimulus appears.

Sustained attention operates via dissociable neural mechanisms across different eccentric locations

In primates, foveal and peripheral vision have distinct neural architectures and functions. However, it has been debated if selective attention operates via the same or different neural mechanisms across eccentricities. We tested these alternative accounts by examining the effects of selective attention on the steady-state visually evoked potential (SSVEP) and the fronto-parietal signal measured via EEG from human subjects performing a sustained visuospatial attention task. With a negligible level of eye movements, both SSVEP and SND exhibited the heterogeneous patterns of attentional modulations across eccentricities. Specifically, the attentional modulations of these signals peaked at the parafoveal locations and such modulations wore off as visual stimuli appeared closer to the fovea or further away towards the periphery. However, with a relatively higher level of eye movements, the heterogeneous patterns of attentional modulations of these neural signals were less robust. These data demonstrate that the top-down influence of covert visuospatial attention on early sensory processing in human cortex depends on eccentricity and the level of saccadic responses. Taken together, the results suggest that sustained visuospatial attention operates differently across different eccentric locations, providing new understanding of how attention augments sensory representations regardless of where the attended stimulus appears.

Dissociable Neural Mechanisms Underlie the Effects of Attention on Visual Appearance and Response Bias

A prominent theoretical framework spanning philosophy, psychology, and neuroscience holds that selective attention penetrates early stages of perceptual processing to alter the subjective visual experience of behaviorally relevant stimuli. For example, searching for a red apple at the grocery store might make the relevant color appear brighter and more saturated compared with seeing the exact same red apple while searching for a yellow banana. In contrast, recent proposals argue that data supporting attention-related changes in appearance reflect decision- and motor-level response biases without concurrent changes in perceptual experience. Here, we tested these accounts by evaluating attentional modulations of EEG responses recorded from male and female human subjects while they compared the perceived contrast of attended and unattended visual stimuli rendered at different levels of physical contrast. We found that attention enhanced the amplitude of the P1 component, an early evoked potential measured over visual cortex. A linking model based on signal detection theory suggests that response gain modulations of the P1 component track attention-induced changes in perceived contrast as measured with behavior. In contrast, attentional cues induced changes in the baseline amplitude of posterior alpha band oscillations (∼9-12 Hz), an effect that best accounts for cue-induced response biases, particularly when no stimuli are presented or when competing stimuli are similar and decisional uncertainty is high. The observation of dissociable neural markers that are linked to changes in subjective appearance and response bias supports a more unified theoretical account and demonstrates an approach to isolate subjective aspects of selective information processing.SIGNIFICANCE STATEMENT Does attention alter visual appearance, or does it simply induce response bias? In the present study, we examined these competing accounts using EEG and linking models based on signal detection theory. We found that response gain modulations of the visually evoked P1 component best accounted for attention-induced changes in visual appearance. In contrast, cue-induced baseline shifts in alpha band activity better explained response biases. Together, these results suggest that attention concurrently impacts visual appearance and response bias, and that these processes can be experimentally isolated.

Perceptual Difficulty Regulates Attentional Gain Modulations in Human Visual Cortex

Perceptual difficulty is sometimes used to manipulate selective attention. However, these two factors are logically distinct. Selective attention is defined by priority given to specific stimuli based on their behavioral relevance, whereas perceptual difficulty is often determined by perceptual demands required to discriminate relevant stimuli. That said, both perceptual difficulty and selective attention are thought to modulate the gain of neural responses in early sensory areas. Previous studies found that selectively attending to a stimulus or increasing perceptual difficulty enhanced the gain of neurons in visual cortex. However, some other studies suggest that perceptual difficulty can have either a null or even reversed effect on gain modulations in visual cortex. According to Yerkes-Dodson's Law, it is possible that this discrepancy arises because of an interaction between perceptual difficulty and attentional gain modulations yielding a nonlinear inverted-U function. Here, we used EEG to measure modulations in the visual cortex of male and female human participants performing an attention-cueing task where we systematically manipulated perceptual difficulty across blocks of trials. The behavioral and neural data implicate a nonlinear inverted-U relationship between selective attention and perceptual difficulty: a focused-attention cue led to larger response gain in both neural and behavioral data at intermediate difficulty levels compared with when the task was more or less difficult. Moreover, difficulty-related changes in attentional gain positively correlated with those predicted by quantitative modeling of the behavioral data. These findings suggest that perceptual difficulty mediates attention-related changes in perceptual performance via selective neural modulations in human visual cortex.SIGNIFICANCE STATEMENT Both perceptual difficulty and selective attention are thought to influence perceptual performance by modulating response gain in early sensory areas. That said, less is known about how selective attention interacts with perceptual difficulty. Here, we measured neural gain modulations in the visual cortex of human participants performing an attention-cueing task where perceptual difficulty was systematically manipulated. Consistent with Yerkes-Dodson's Law, our behavioral and neural data implicate a nonlinear inverted-U relationship between selective attention and perceptual difficulty. These results suggest that perceptual difficulty mediates attention-related changes in perceptual performance via selective neural modulations in visual cortex, extending our understanding of the attentional operation under different levels of perceptual demands.

Different gain modulations of selective attention at short and long cue-target delays

Previous studies have tested attentional gain patterns, for example, within the normalization model of attention (NMoA), by altering the relative sizes of the attention field and stimuli. Existing studies have not investigated whether the gain patterns of altering the relative exogenous cue size as compared with the target stimuli matches the prediction of NMoA and whether these gain patterns exist in the late stage of attentional processing. To resolve these questions, the present study maintained the target grating size and changed the exogenous cue size in both short and long cue-target onset asynchronies (CTOAs) conditions. The results revealed response gain for small cue/large target size conditions and contrast gain for large cue/small target size conditions, which was consistent with the NMoA. However, we observed the decrease in the contrast gain factor only with long CTOAs, regardless of whether the cue size was relatively small or large. This indicated that NMoA-related effects based on the relative attentional field dominated in the early stage and that the contrast gain dominated in the late stage.

Differential Involvement of EEG Oscillatory Components in Sameness versus Spatial-Relation Visual Reasoning Tasks

The development of deep convolutional neural networks (CNNs) has recently led to great successes in computer vision, and CNNs have become de facto computational models of vision. However, a growing body of work suggests that they exhibit critical limitations on tasks beyond image categorization. Here, we study one such fundamental limitation, concerning the judgment of whether two simultaneously presented items are the same or different (SD) compared with a baseline assessment of their spatial relationship (SR). In both human subjects and artificial neural networks, we test the prediction that SD tasks recruit additional cortical mechanisms which underlie critical aspects of visual cognition that are not explained by current computational models. We thus recorded electroencephalography (EEG) signals from human participants engaged in the same tasks as the computational models. Importantly, in humans the two tasks were matched in terms of difficulty by an adaptive psychometric procedure; yet, on top of a modulation of evoked potentials (EPs), our results revealed higher activity in the low β (16–24 Hz) band in the SD compared with the SR conditions. We surmise that these oscillations reflect the crucial involvement of additional mechanisms, such as working memory and attention, which are missing in current feed-forward CNNs.

Evidence accumulation during perceptual decision-making is sensitive to the dynamics of attentional selection

The ability to select and combine multiple sensory inputs in support of accurate decisions is a hallmark of adaptive behaviour. Attentional selection is often needed to prioritize task-relevant stimuli relative to irrelevant, potentially distracting stimuli. As most studies of perceptual decision-making to date have made use of task-relevant stimuli only, relatively little is known about how attention modulates decision making. To address this issue, we developed a novel 'integrated' decision-making task, in which participants judged the average direction of successive target motion signals while ignoring concurrent and spatially overlapping distractor motion signals. In two experiments that varied the role of attentional selection, we used regression to quantify the influence of target and distractor stimuli on behaviour. Using electroencephalography, we characterised the neural correlates of decision making, attentional selection and feature-specific responses to target and distractor signals. While targets strongly influenced perceptual decisions and associated neural activity, we also found that concurrent and spatially coincident distractors exerted a measurable bias on both behaviour and brain activity. Our findings suggest that attention operates as a real-time but imperfect filter during perceptual decision-making by dynamically modulating the contributions of task-relevant and irrelevant sensory inputs.

Stimulus visibility and uncertainty mediate the influence of attention on response bias and visual contrast appearance

Although attention is known to improve the efficacy of sensory processing, the impact of attention on subjective visual appearance is still a matter of debate. Although recent studies suggest that attention can alter the appearance of stimulus contrast, others argue that these changes reflect response bias induced by attention cues. Here, we provide evidence that attention has effects on both appearance and response bias. In a comparative judgment task in which subjects reported whether the attended or unattended visual stimulus had a higher perceived contrast, attention induced substantial baseline-offset response bias as well as small but significant changes in subjective contrast appearance when subjects viewed near-threshold stimuli. However, when subjects viewed suprathreshold stimuli, baseline-offset response bias decreased and attention primarily changed contrast appearance. To address the possibility that these changes in appearance might be influenced by uncertainty due to the attended and unattended stimuli having similar physical contrasts, subjects performed an equality judgment task in which they reported if the contrast of the two stimuli was the same or different. We found that, although there were still attention-induced changes in contrast appearance at lower contrasts, the robust changes in contrast appearance at higher contrasts observed in the comparative judgment task were diminished in the equality judgment task. Together, these results suggest that attention can impact both response bias and appearance, and these two types of attention effects are differentially mediated by stimulus visibility and uncertainty. Collectively, these findings help constrain arguments about the cognitive penetrability of perception.

When Conflict Cannot be Avoided: Relative Contributions of Early Selection and Frontal Executive Control in Mitigating Stroop Conflict

When viewing familiar stimuli (e.g., common words), processing is highly automatized such that it can interfere with the processing of incompatible sensory information. At least two mechanisms may help mitigate this interference. Early selection accounts posit that attentional processes filter out distracting sensory information to avoid conflict. Alternatively, late selection accounts hold that all sensory inputs receive full semantic analysis and that frontal executive mechanisms are recruited to resolve conflict. To test how these mechanisms operate to overcome conflict induced by highly automatized processing, we developed a novel version of the color-word Stroop task, where targets and distractors were simultaneously flickered at different frequencies. We measured the quality of early sensory processing by assessing the amplitude of steady-state visually evoked potentials (SSVEPs) elicited by targets and distractors. We also indexed frontal executive processes by assessing changes in frontal theta oscillations induced by color-word incongruency. We found that target- and distractor-related SSVEPs were not modulated by changes in the level of conflict whereas frontal theta activity increased on high compared to low conflict trials. These results suggest that frontal executive processes play a more dominant role in mitigating cognitive interference driven by the automatic tendency to process highly familiar stimuli.

Functional MRI and EEG Index Complementary Attentional Modulations

Functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) are two noninvasive methods commonly used to study neural mechanisms supporting visual attention in humans. Studies using these tools, which have complementary spatial and temporal resolutions, implicitly assume they index similar underlying neural modulations related to external stimulus and internal attentional manipulations. Accordingly, they are often used interchangeably for constraining understanding about the impact of bottom-up and top-down factors on neural modulations. To test this core assumption, we simultaneously manipulated bottom-up sensory inputs by varying stimulus contrast and top-down cognitive modulations by changing the focus of spatial attention. Each of the male and female subjects participated in both fMRI and EEG sessions performing the same experimental paradigm. We found categorically different patterns of attentional modulation on fMRI activity in early visual cortex and early stimulus-evoked potentials measured via EEG (e.g., the P1 component and steady-state visually-evoked potentials): fMRI activation scaled additively with attention, whereas evoked EEG components scaled multiplicatively with attention. However, across longer time scales, a contralateral negative-going potential and oscillatory EEG signals in the alpha band revealed additive attentional modulation patterns like those observed with fMRI. These results challenge prior assumptions that fMRI and early stimulus-evoked potentials measured with EEG can be interchangeably used to index the same neural mechanisms of attentional modulations at different spatiotemporal scales. Instead, fMRI measures of attentional modulations are more closely linked with later EEG components and alpha-band oscillations. Considered together, hemodynamic and electrophysiological signals can jointly constrain understanding of the neural mechanisms supporting cognition.SIGNIFICANCE STATEMENT fMRI and EEG have been used as tools to measure the location and timing of attentional modulations in visual cortex and are often used interchangeably for constraining computational models under the assumption that they index similar underlying neural processes. However, by varying attentional and stimulus parameters, we found differential patterns of attentional modulations of fMRI activity in early visual cortex and commonly used stimulus-evoked potentials measured via EEG. Instead, across longer time scales, a contralateral negative-going potential and EEG oscillations in the alpha band exhibited attentional modulations similar to those observed with fMRI. Together, these results suggest that different physiological processes assayed by these complementary techniques must be jointly considered when making inferences about the neural underpinnings of cognitive operations.