Within-hemifield competition in early visual areas limits the ability to track multiple objects with attention

Sustained attention operates via dissociable neural mechanisms across different eccentric locations

In primates, foveal and peripheral vision have distinct neural architectures and functions. However, it has been debated if selective attention operates via the same or different neural mechanisms across eccentricities. We tested these alternative accounts by examining the effects of selective attention on the steady-state visually evoked potential (SSVEP) and the fronto-parietal signal measured via EEG from human subjects performing a sustained visuospatial attention task. With a negligible level of eye movements, both SSVEP and SND exhibited the heterogeneous patterns of attentional modulations across eccentricities. Specifically, the attentional modulations of these signals peaked at the parafoveal locations and such modulations wore off as visual stimuli appeared closer to the fovea or further away towards the periphery. However, with a relatively higher level of eye movements, the heterogeneous patterns of attentional modulations of these neural signals were less robust. These data demonstrate that the top-down influence of covert visuospatial attention on early sensory processing in human cortex depends on eccentricity and the level of saccadic responses. Taken together, the results suggest that sustained visuospatial attention operates differently across different eccentric locations, providing new understanding of how attention augments sensory representations regardless of where the attended stimulus appears.

Location- and Object-Based Representational Mechanisms Account for Bilateral Field Advantage in Multiple-Object Tracking

Keeping track of multiple visually identical and independently moving objects is a remarkable feature of the human visual system. Theoretical accounts for this ability focus on resource-based models that describe parametric decreases of performance with increasing demands during the task (i.e., more relevant items, closer distances, higher speed). Additionally, the presence of two central tracking resources, one within each hemisphere, has been proposed, allowing for an independent maintenance of moving targets within each visual hemifield. Behavioral evidence in favor of such a model shows that human subjects are able to track almost twice as many targets across both hemifields compared with within one hemifield. A number of recent publications argue for two separate and parallel tracking mechanisms during standard object tracking tasks that allow for the maintenance of the relevant information in a location-based and object-based manner. Unique electrophysiological correlates for each of those processes have been identified. The current study shows that these electrophysiological components are differentially present during tracking within either the left or right hemifield. The present results suggest that targets are mostly maintained as an object-based representation during left hemifield tracking, while location-based resources are preferentially engaged during right hemifield tracking. Interestingly, the manner of representation does not seem to have an impact on behavioral performance within the subjects, while the electrophysiological component indicating object-based tracking does correlate with performance between subjects. We propose that hemifield independence during multiple-object tracking may be an indication of the underlying hemispheric bias for parallel location-based and object-based tracking mechanisms.

Sustained attention operates via dissociable neural mechanisms across different eccentric locations

In primates, foveal and peripheral vision have distinct neural architectures and functions. However, it has been debated if selective attention operates via the same or different neural mechanisms across eccentricities. We tested these alternative accounts by examining the effects of selective attention on the steady-state visually evoked potential (SSVEP) and the fronto-parietal signal measured via EEG from human subjects performing a sustained visuospatial attention task. With a negligible level of eye movements, both SSVEP and SND exhibited the heterogeneous patterns of attentional modulations across eccentricities. Specifically, the attentional modulations of these signals peaked at the parafoveal locations and such modulations wore off as visual stimuli appeared closer to the fovea or further away towards the periphery. However, with a relatively higher level of eye movements, the heterogeneous patterns of attentional modulations of these neural signals were less robust. These data demonstrate that the top-down influence of covert visuospatial attention on early sensory processing in human cortex depends on eccentricity and the level of saccadic responses. Taken together, the results suggest that sustained visuospatial attention operates differently across different eccentric locations, providing new understanding of how attention augments sensory representations regardless of where the attended stimulus appears.

Testing hemifield independence for divided attention in visual object tasks

In this study, we asked to what degree hemifields contribute to divided attention effects observed in tasks with object-based judgments. If object recognition processes in the two hemifields were fully independent, then placing stimuli in separate hemifields would eliminate divided attention effects; in the alternative extreme, if object recognition processes in the two hemifields were fully integrated, then placing stimuli in separate hemifields would not modulate divided attention effects. Using a dual-task paradigm, we compared performance in a semantic categorization task for relevant stimuli arranged in the same hemifield to performance for relevant stimuli arranged in separate left and right hemifields. In two experiments, there was a reliable decrease in divided attention effects when stimuli were shown in separate hemifields compared to the same hemifield. However, the effect of divided attention was not eliminated. These results reject both the independent and integrated hypotheses, and instead support a third alternative - that object recognition processes in the two hemifields are partially dependent. More specifically, the magnitude of modulation by hemifields was closer to the prediction of the integrated hypothesis, suggesting that for dual tasks with objects, dependent processing is mostly shared across the visual field.

Tracking momentary fluctuations in human attention with a cognitive brain-machine interface

Selective attention produces systematic effects on neural states. It is unclear whether, conversely, momentary fluctuations in neural states have behavioral significance for attention. We investigated this question in the human brain with a cognitive brain-machine interface (cBMI) for tracking electrophysiological steady-state visually evoked potentials (SSVEPs) in real-time. Discrimination accuracy (d') was significantly higher when target stimuli were triggered at high, versus low, SSVEP power states. Target and distractor SSVEP power was uncorrelated across the hemifields, and target d' was unaffected by distractor SSVEP power states. Next, we trained participants on an auditory neurofeedback paradigm to generate biased, cross-hemispheric competitive interactions between target and distractor SSVEPs. The strongest behavioral effects emerged when competitive SSVEP dynamics unfolded at a timescale corresponding to the deployment of endogenous attention. In sum, SSVEP power dynamics provide a reliable readout of attentional state, a result with critical implications for tracking and training human attention.

Attentional Enhancement of Tracked Stimuli in Early Visual Cortex Has Limited Capacity

Keeping track of the location of multiple moving objects is one of the well documented functions of visual attention. However, the mechanism of attentional selection that supports such continuous tracking is unclear. In particular, it has been proposed that target selection in early visual cortex occurs in parallel, with tracking errors arising because of attentional limitations at later processing stages. Here, we examine whether, instead, total attentional capacity for enhancement of early visual processing of tracked targets is shared between all attended stimuli. If the magnitude of attentional facilitation of multiple tracked targets was a key limiting factor of tracking ability, then one should expect it to drop systematically with increasing set-size of tracked targets. Human observers (male and female) were instructed to track two, four, or six moving objects among a pool of identical distractors. Steady-state visual evoked potentials (SSVEPs) recorded during the tracking period revealed that the processing of tracked targets was consistently amplified compared with the processing of the distractors. The magnitude of this amplification decreased with increasing set size, and at lateral occipital electrodes it closely followed inverse proportionality to the number of tracked items, suggesting that limited attentional resources must be shared among the tracked stimuli. Accordingly, the magnitude of attentional facilitation predicted the behavioral outcome at the end of the trial. Together, these findings demonstrate that the limitations of multiple object tracking (MOT) across set-sizes stem from the limitations of top-down selective attention already at the early stages of visual processing.SIGNIFICANCE STATEMENT The ability to selectively attend to relevant features or objects is the key to flexibility of perception and action in the continuously changing environment. This ability is demonstrated in the multiple object tracking (MOT) task where observers monitor multiple independently moving objects at different locations in the visual field. The role of early attentional enhancement in tracking was previously acknowledged in the literature, however, the limitations on tracking were thought to arise during later stages of processing. Here, we demonstrate that the strength of attentional facilitation depends on the number of tracked objects and predicts successful tracking performance. Thus, it is the limitations of attentional enhancement at the early stages of visual processing that determine behavioral performance limits.

Attention to visual motion suppresses neuronal and behavioral sensitivity in nearby feature space

Background

Feature-based attention prioritizes the processing of the attended feature while strongly suppressing the processing of nearby ones. This creates a non-linearity or “attentional suppressive surround” predicted by the Selective Tuning model of visual attention. However, previously reported effects of feature-based attention on neuronal responses are linear, e.g., feature-similarity gain. Here, we investigated this apparent contradiction by neurophysiological and psychophysical approaches.

Results

Responses of motion direction-selective neurons in area MT/MST of monkeys were recorded during a motion task. When attention was allocated to a stimulus moving in the neurons’ preferred direction, response tuning curves showed its minimum for directions 60–90° away from the preferred direction, an attentional suppressive surround. This effect was modeled via the interaction of two Gaussian fields representing excitatory narrowly tuned and inhibitory widely tuned inputs into a neuron, with feature-based attention predominantly increasing the gain of inhibitory inputs. We further showed using a motion repulsion paradigm in humans that feature-based attention produces a similar non-linearity on motion discrimination performance.

Conclusions

Our results link the gain modulation of neuronal inputs and tuning curves examined through the feature-similarity gain lens to the attentional impact on neural population responses predicted by the Selective Tuning model, providing a unified framework for the documented effects of feature-based attention on neuronal responses and behavior.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01428-7.

An adaptive perspective on visual working memory distortions

When holding multiple items in visual working memory, representations of individual items are often attracted to, or repelled from, each other. While this is empirically well-established, existing frameworks do not account for both types of distortions, which appear to be in opposition. Here, we demonstrate that both types of memory distortion may confer functional benefits under different circumstances. When there are many items to remember and subjects are near their capacity to accurately remember each item individually, memories for each item become more similar (attraction). However, when remembering smaller sets of highly similar but discernible items, memory for each item becomes more distinct (repulsion), possibly to support better discrimination. Importantly, this repulsion grows stronger with longer delays, suggesting that it dynamically evolves in memory and is not just a differentiation process that occurs during encoding. Furthermore, both attraction and repulsion occur even in tasks designed to mitigate response bias concerns, suggesting they are genuine changes in memory representations. Together, these results are in line with the theory that attraction biases act to stabilize memory signals by capitalizing on information about an entire group of items, whereas repulsion biases reflect a tradeoff between maintaining accurate but distinct representations. Both biases suggest that human memory systems may sacrifice veridical representations in favor of representations that better support specific behavioral goals. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Preattentive facilitation of target trajectories in a dragonfly visual neuron

The ability to pursue targets in visually cluttered and distraction-rich environments is critical for predators such as dragonflies. Previously, we identified Centrifugal Small-Target Motion Detector 1 (CSTMD1), a dragonfly visual neuron likely involved in such target-tracking behaviour. CSTMD1 exhibits facilitated responses to targets moving along a continuous trajectory. Moreover, CSTMD1 competitively selects a single target out of a pair. Here, we conducted in vivo, intracellular recordings from CSTMD1 to examine the interplay between facilitation and selection, in response to the presentation of paired targets. We find that neuronal responses to both individual trajectories of simultaneous, paired targets are facilitated, rather than being constrained to the single, selected target. Additionally, switches in selection elicit suppression which is likely an important attribute underlying target pursuit. However, binocular experiments reveal these results are constrained to paired targets within the same visual hemifield, while selection of a target in one visual hemifield establishes ocular dominance that prevents facilitation or response to contralaterally presented targets. These results reveal that the dragonfly brain preattentively represents more than one target trajectory, to balance between attentional flexibility and resistance against distraction.

A dragonfly visual neuron independently facilitates responses to rival targets within the same visual field, mediating selective attention.

The protocol for assessing olfactory working memory capacity in mice

BACKGROUND

Working memory capacity (WMC) is the ability to maintain information over a few seconds. Although it has been extensively studied in healthy subjects and neuropsychiatric patients, few tasks have been developed to measure such changes in rodents. Many procedures have been used to measure WM in rodents, including the radial arm maze, the WM version of the Morris swimming task, and various delayed matching and nonmatching-to-sample tasks. It should be noted, however, that the memory components assessed in these procedures do not include memory capacity.

METHODS

We developed an olfactory working memory capacity (OWMC) paradigm to assess the WMC of 3-month-old 5×FAD mice, a mouse model of Alzheimer's disease. The task is divided into five phases: context adaptation, digging training, rule learning for nonmatching to a single sample odor (NMSS), rule learning for nonmatching to multiple sample odors (NMMS), and capacity testing.

RESULTS

In the NMSS rule-learning phase, there was no difference between wild-type (WT) mice and 5×FAD mice in the performance correct rate, correct option rate, and correct rejection rate. The WT mice and 5×FAD mice showed similar memory capacity in the NMMS rule-learning phase. After capacity test, we found that the WMC was significantly diminished in 5×FAD mice. As the memory load increased, 5×FAD mice also made significantly more errors than WT mice.

CONCLUSION

The OWMC task, based on a nonmatch-to-sample rule, is a sensitive and robust behavioral assay that we validated as a reliable method for measuring WMC and exploring different components of memory in mice.

Mid-level Feature Differences Support Early Animacy and Object Size Distinctions: Evidence from EEG Decoding

Responses to visually presented objects along the cortical surface of the human brain have a large-scale organization reflecting the broad categorical divisions of animacy and object size. Emerging evidence indicates that this topographical organization is supported by differences between objects in mid-level perceptual features. With regard to the timing of neural responses, images of objects quickly evoke neural responses with decodable information about animacy and object size, but are mid-level features sufficient to evoke these rapid neural responses? Or is slower iterative neural processing required to untangle information about animacy and object size from mid-level features, requiring hundreds of milliseconds more processing time? To answer this question, we used EEG to measure human neural responses to images of objects and their texform counterparts-unrecognizable images that preserve some mid-level feature information about texture and coarse form. We found that texform images evoked neural responses with early decodable information about both animacy and real-world size, as early as responses evoked by original images. Furthermore, successful cross-decoding indicates that both texform and original images evoke information about animacy and size through a common underlying neural basis. Broadly, these results indicate that the visual system contains a mid-level feature bank carrying linearly decodable information on animacy and size, which can be rapidly activated without requiring explicit recognition or protracted temporal processing.

Visual attention around a hand location localized by proprioceptive information

Facilitation of visual processing has been reported in the space near the hand. To understand the underlying mechanism of hand proximity attention, we conducted experiments that isolated hand-related effects from top–down attention, proprioceptive information from visual information, the position effect from the influence of action, and the distance effect from the peripersonal effect. The flash-lag effect was used as an index of attentional modulation. Because the results showed that the flash-lag effect was smaller at locations near the hand, we concluded that there was a facilitation effect of the visual stimuli around the hand location identified through proprioceptive information. This was confirmed by conventional reaction time measures. We also measured steady-state visual evoked potential (SSVEP) in order to investigate the spatial properties of hand proximity attention and top–down attention. The results showed that SSVEP reflects the effect of top–down attention but not that of hand proximity attention. This suggests that the site of hand proximity attention is at a later stage of visual processing, assuming that SSVEP responds to neural activities at the early stages. The results of left-handers differed from those of right-handers, and this is discussed in relation to handedness variation.

The what and when of olfactory working memory in humans

Encoding and retaining novel sequences of sensory stimuli in working memory is crucial for adaptive behavior. A fundamental challenge for the central nervous system is to maintain each sequence item in an active and discriminable state, while also preserving their temporal context. Nested neural oscillations have been postulated to disambiguate the "what" and "when" of sequences, but the mechanisms by which these multiple streams of information are coordinated in the human brain remain unclear. Drawing from foundational animal studies, we recorded local field potentials from the human piriform cortex and hippocampus during a working memory task in which subjects experienced sequences of three distinct odors. Our data revealed a unique organization of odor memories across multiple timescales of the theta rhythm. During encoding, odors elicited greater gamma at distinct theta phases in both regions, time stamping their positions in the sequence, whereby the robustness of this effect was predictive of temporal order memory. During maintenance, stimulus-driven patterns of theta-coupled gamma were spontaneously reinstated in piriform cortex, recapitulating the order of the initial sequence. Replay events were time compressed across contiguous theta cycles, coinciding with periods of enhanced piriform-hippocampal theta-phase synchrony, and their prevalence forecasted subsequent recall accuracy on a trial-by-trial basis. Our data provide a novel link between endogenous replay orchestrated by the theta rhythm and short-term retention of sequential memories in the human brain.

Behavioral gain following isolation of attention

Stable sensory perception is achieved through balanced excitatory-inhibitory interactions of lateralized sensory processing. In real world experience, sensory processing is rarely equal across lateralized processing regions, resulting in continuous rebalancing. Using lateralized attention as a case study, we predicted rebalancing lateralized processing following prolonged spatial attention imbalance could cause a gain in attention in the opposite direction. In neurotypical human adults, we isolated covert attention to one visual field with a 30-min attention-demanding task and found an increase in attention in the opposite visual field after manipulation. We suggest a gain in lateralized attention in the previously unattended visual field is due to an overshoot through attention rebalancing. The offline post-manipulation effect is suggestive of long-term potentiation affecting behavior. Our finding of visual field specific attention increase could be critical for the development of clinical rehabilitation for patients with a unilateral lesion and lateralized attention deficits. This proof-of-concept study initiates the examination of overshoot following the release of imbalance in other lateralized control and sensory domains, important in our basic understanding of lateralized processing.

Cross-modal orienting of exogenous attention results in visual-cortical facilitation, not suppression

Attention may be oriented exogenously (i.e., involuntarily) to the location of salient stimuli, resulting in improved perception. However, it is unknown whether exogenous attention improves perception by facilitating processing of attended information, suppressing processing of unattended information, or both. To test this question, we measured behavioral performance and cue-elicited neural changes in the electroencephalogram as participants (N = 19) performed a task in which a spatially non-predictive auditory cue preceded a visual target. Critically, this cue was either presented at a peripheral target location or from the center of the screen, allowing us to isolate spatially specific attentional activity. We find that both behavior and attention-mediated changes in visual-cortical activity are enhanced at the location of a cue prior to the onset of a target, but that behavior and neural activity at an unattended target location is equivalent to that following a central cue that does not direct attention (i.e., baseline). These results suggest that exogenous attention operates via facilitation of information at an attended location.

Using fast visual rhythmic stimulation to control inter-hemispheric phase offsets in visual areas

Spike timing dependent plasticity (STDP) is believed to be important for neural communication and plasticity in human episodic memory, but causal evidence is lacking due to technical challenges. Rhythmic sensory stimulation that has been used to investigate causal relations between oscillations and cognition may be able to address this question. The challenge, however, is that the frequency corresponding to the critical time window for STDP is gamma (~40 Hz), yet the application of rhythmic sensory stimulation has been limited primarily to lower frequencies (<30 Hz). It remains unknown whether this method can be applied to precisely control the activation time delay between distant groups of neurons at a millisecond scale. To answer this question and examine the role of STDP in human episodic memory, we simulated the STDP function by controlling the activation time delay between the left and right visual cortices during memory encoding. This was achieved by presenting flickering (37.5 Hz) movie pairs in the left and right visual fields with a phase lag of either 0, 90, 180 or 270°. Participants were asked to memorize the two movies within each pair and the association was later tested. Behavioral results revealed no significant difference in memory performance across conditions with different degrees of gamma phase synchrony. Yet importantly, our study showed for the first time, that oscillatory activity can be driven with a precision of 6.67 ms delay between neuronal groups. Our method hereby provides an approach to investigate relations between precise neuronal timing and cognitive functions.

Attentional tracking takes place over perceived rather than veridical positions

Illusions can induce striking differences between perception and retinal input. For instance, a static Gabor with a moving internal texture appears to be shifted in the direction of its internal motion, a shift that increases dramatically when the Gabor itself is also in motion. Here, we ask whether attention operates on the perceptual or physical location of this stimulus. To do so, we generated an attentional tracking task where participants (N = 15) had to keep track of a single target among three Gabors that rotated around a common center in the periphery. During tracking, the illusion was used to make three Gabors appear either shifted away from or toward one another while maintaining the same physical separation. Because tracking performance depends in part on target to distractor spacing, if attention selects targets from perceived positions, performance should be better when the Gabors appear further apart and worse when they appear closer together. We find that tracking performance is superior with greater perceived separation, implying that attentional tracking operates over perceived rather than physical positions.

Changes in visual cortical processing attenuate singleton distraction during visual search

The ability to suppress distractions is essential to successful completion of goal-directed behaviors. Several behavioral studies have recently provided strong evidence that learned suppression may be particularly efficient in reducing distractor interference. Expectations about a distractor's repeated location, color, or even presence are rapidly learned and used to attenuate interference. In this study, we use a visual search paradigm in which a color singleton, which is known to capture attention, occurs within blocks with high or low frequency. The behavioral results show reduced singleton interference during the high compared to the low frequency block (Won et al., 2019). The fMRI results provide evidence that the attenuation of distractor interference is supported by changes in singleton, target, and non-salient distractor representations within retinotopic visual cortex. These changes in visual cortex are accompanied by findings that singleton-present trials compared to non-singleton trials produce greater activation in bilateral parietal cortex, indicative of attentional capture, in low frequency, but not high frequency blocks. Together, these results suggest that the readout of saliency signals associated with an expected color singleton from visual cortex is suppressed, resulting in less competition for attentional priority in frontoparietal attentional control regions.

Lateralized alpha activity and slow potential shifts over visual cortex track the time course of both endogenous and exogenous orienting of attention

Spatial attention can be oriented endogenously, based on current task goals, or exogenously, triggered by salient events in the environment. Based upon literature demonstrating differences in the time course and neural substrates of each type of orienting, these two attention systems are often treated as fundamentally distinct. However, recent studies suggest that rhythmic neural activity in the alpha band (8-13 Hz) and slow waves in the event-related potential (ERP) may emerge over parietal-occipital cortex following both endogenous and exogenous attention cues. To assess whether these neural changes index common processes of spatial attention, we conducted two within-subject experiments varying the two main dimensions over which endogenous and exogenous attention tasks typically differ: cue informativity (spatially predictive vs. non-predictive) and cue format (centrally vs. peripherally presented). This task design allowed us to tease apart neural changes related to top-down goals and those driven by the reflexive orienting of spatial attention, and examine their interactions in a novel hybrid cross-modal attention task. Our data demonstrate that both central and peripheral cues elicit lateralized ERPs over parietal-occipital cortex, though at different points in time, consistent with these ERPs reflecting the orienting of spatial attention. Lateralized alpha activity was also present across all tasks, emerging rapidly for peripheral cues and sustaining longer for spatially informative cues. Overall, these data indicate that distinct slow-wave ERPs index the spatial orienting of endogenous and exogenous attention, while lateralized alpha activity represents a common signature of visual-cortical biasing in anticipation of potential targets across both types of attention.

Hemifield-specific control of spatial attention and working memory: Evidence from hemifield crossover costs

Attentional tracking and working memory tasks are often performed better when targets are divided evenly between the left and right visual hemifields, rather than contained within a single hemifield (Alvarez & Cavanagh, 2005; Delvenne, 2005). However, this bilateral field advantage does not provide conclusive evidence of hemifield-specific control of attention and working memory, because it can be explained solely from hemifield-limited spatial interference at early stages of visual processing. If control of attention and working memory is specific to each hemifield, maintaining target information should become more difficult as targets move between the two hemifields. Observers in the present study maintained targets that moved either within or between the left and right hemifields, using either attention (Experiment 1) or working memory (Experiment 2). Maintaining spatial information was more difficult when target items moved between the hemifields compared with when target items moved within their original hemifields, consistent with hemifield-specific control of spatial attention and working memory. However, this pattern was not found for maintaining identity information (e.g., color) in working memory (Experiment 3). Together, these results provide evidence that control of spatial attention and working memory is specific to each hemifield, and that hemifield-specific control is a unique signature of spatial processing.

Labor division in joint tasks: Humans maximize use of their individual attentional capacities

In daily life, humans frequently perform visuospatial tasks together (e.g., visual search) and distribute the labor in such tasks. Previous research has shown that humans prefer a left and right labor division in a joint multiple object tracking (MOT) task. Yet, findings from studies investigating individuals' tracking ability suggest attentional capacities may be more maximally used with a top and bottom labor division. We investigated whether co-actors' labor division preference is influenced by how they are seated (neighboring vs. opposite of each other) or how the MOT task is displayed (portrait vs. landscape). We find that pairs attain a higher performance using a top and bottom labor division and preferred this labor division compared to a left and right division. This preference was unaffected by the seating arrangement. For the landscape display, however, we find that participants no longer attain a higher performance for the top and bottom labor division and accordingly participants' preference for this labor division was greatly reduced as well. Overall, we propose that co-actors are sensitive to changes within their environment, which allows them to choose a labor division that maximizes use of their individual attentional capacities.

Affective Bias without Hemispheric Competition: Evidence for Independent Processing Resources in Each Cortical Hemisphere

We assessed the extent of neural competition for attentional processing resources in early visual cortex between foveally presented task stimuli and peripheral emotional distracter images. Task-relevant and distracting stimuli were shown in rapid serial visual presentation (RSVP) streams to elicit the steady-state visual evoked potential, which serves as an electrophysiological marker of attentional resource allocation in early visual cortex. A task-related RSVP stream of symbolic letters was presented centrally at 15 Hz while distracting RSVP streams were displayed at 4 or 6 Hz in the left and right visual hemifields. These image streams always had neutral content in one visual field and would unpredictably switch from neutral to unpleasant content in the opposite visual field. We found that the steady-state visual evoked potential amplitude was consistently modulated as a function of change in emotional valence in peripheral RSVPs, indicating sensory gain in response to distracting affective content. Importantly, the facilitated processing for emotional content shown in one visual hemifield was not paralleled by any perceptual costs in response to the task-related processing in the center or the neutral image stream in the other visual hemifield. Together, our data provide further evidence for sustained sensory facilitation in favor of emotional distracters. Furthermore, these results are in line with previous reports of a “different hemifield advantage” with low-level visual stimuli and are suggestive of independent processing resources in each cortical hemisphere that operate beyond low-level visual cues, that is, with complex images that impact early stages of visual processing via reentrant feedback loops from higher order processing areas.

Gamma coherence mediates interhemispheric integration during multiple object tracking

Our ability to track the paths of multiple visual objects moving between the hemifields requires effective integration of information between the two cerebral hemispheres. Coherent neural oscillations in the gamma band (35-70 Hz) are hypothesized to drive this information transfer. Here we manipulated the need for interhemispheric integration using a novel multiple object tracking (MOT) task in which stimuli either moved between the two visual hemifields, requiring interhemispheric integration, or moved within separate visual hemifields. We used electroencephalography (EEG) to measure interhemispheric coherence during the task. Human observers (21 women; 20 men) were poorer at tracking objects between versus within hemifields, reflecting a cost of interhemispheric integration. Critically, gamma coherence was greater in trials requiring interhemispheric integration, particularly between sensors over parieto-occipital areas. In approximately half of the participants, the observed cost of integration was associated with a failure of the cerebral hemispheres to become coherent in the gamma band. Moreover, individual differences in this integration cost correlated with endogenous gamma coherence at these same sensors, although with generally opposing relationships for the real and imaginary part of coherence. The real part (capturing synchronization with a near-zero phase lag) benefited between-hemifield tracking; imaginary coherence was detrimental. Finally, instantaneous phase coherence over the tracking period uniquely predicted between-hemifield tracking performance, suggesting that effective integration benefits from sustained interhemispheric synchronization. Our results show that gamma coherence mediates interhemispheric integration during MOT and add to a growing body of work demonstrating that coherence drives communication across cortically distributed neural networks.NEW & NOTEWORTHY Using a multiple object tracking paradigm, we were able to manipulate the need for interhemispheric integration on a per-trial basis, while also having an objective measure of integration efficacy (i.e., tracking performance). We show that tracking performance reflects a cost of integration, which correlates with individual differences in interhemispheric EEG coherence. Gamma coherence appears to uniquely benefit between-hemifield tracking, predicting performance both across participants and across trials.

Interaction between steady-state visually evoked potentials at nearby flicker frequencies

Steady-state visually evoked potential (SSVEP) studies routinely employ simultaneous presentation of two temporally modulated stimuli, with SSVEP amplitude modulations serving to index top-down cognitive processes. However, the nature of SSVEP amplitude modulations as a function of competing temporal frequency (TF) has not been systematically studied, especially in relation to the normalization framework which has been extensively used to explain visual responses to multiple stimuli. We recorded spikes and local field potential (LFP) from the primary visual cortex (V1) as well as EEG from two awake macaque monkeys while they passively fixated plaid stimuli with components counterphasing at different TFs. We observed asymmetric SSVEP response suppression by competing TFs (greater suppression for lower TFs), which further depended on the relative orientations of plaid components. A tuned normalization model, adapted to SSVEP responses, provided a good account of the suppression. Our results provide new insights into processing of temporally modulated visual stimuli.

Spatial Attentional Selection Modulates Early Visual Stimulus Processing Independently of Visual Alpha Modulations

The capacity-limited human brain is constantly confronted with a huge amount of sensory information. Selective attention is needed for biasing neural processing towards relevant information and consequently allows meaningful interaction with the environment. Activity in the alpha-band has been proposed to be related to top-down modulation of neural inhibition and could thus represent a viable candidate to control the priority of stimulus processing. It is, however, unknown whether modulations in the alpha-band directly relate to changes in the sensory gain control of the early visual cortex. Here, we used a spatial cueing paradigm while simultaneously measuring ongoing alpha-band oscillations and steady-state visual evoked potentials (SSVEPs) as a marker of continuous early sensory processing in the human visual cortex. Thereby, the effects of spatial attention for both of these signals and their potential interactions were assessed. As expected, spatial attention modulated both alpha-band and SSVEP responses. However, their modulations were independent of each other and the corresponding activity profiles differed across task demands. Thus, our results challenge the view that modulations of alpha-band activity represent a mechanism that directly alters or controls sensory gain. The potential role of alpha-band oscillations beyond sensory processing will be discussed in light of the present results.

Stimulus-Driven Brain Rhythms within the Alpha Band: The Attentional-Modulation Conundrum

Two largely independent research lines use rhythmic sensory stimulation to study visual processing. Despite the use of strikingly similar experimental paradigms, they differ crucially in their notion of the stimulus-driven periodic brain responses: one regards them mostly as synchronized (entrained) intrinsic brain rhythms; the other assumes they are predominantly evoked responses [classically termed steady-state responses (SSRs)] that add to the ongoing brain activity. This conceptual difference can produce contradictory predictions about, and interpretations of, experimental outcomes. The effect of spatial attention on brain rhythms in the alpha band (8-13 Hz) is one such instance: alpha-range SSRs have typically been found to increase in power when participants focus their spatial attention on laterally presented stimuli, in line with a gain control of the visual evoked response. In nearly identical experiments, retinotopic decreases in entrained alpha-band power have been reported, in line with the inhibitory function of intrinsic alpha. Here we reconcile these contradictory findings by showing that they result from a small but far-reaching difference between two common approaches to EEG spectral decomposition. In a new analysis of previously published human EEG data, recorded during bilateral rhythmic visual stimulation, we find the typical SSR gain effect when emphasizing stimulus-locked neural activity and the typical retinotopic alpha suppression when focusing on ongoing rhythms. These opposite but parallel effects suggest that spatial attention may bias the neural processing of dynamic visual stimulation via two complementary neural mechanisms.SIGNIFICANCE STATEMENT Attending to a visual stimulus strengthens its representation in visual cortex and leads to a retinotopic suppression of spontaneous alpha rhythms. To further investigate this process, researchers often attempt to phase lock, or entrain, alpha through rhythmic visual stimulation under the assumption that this entrained alpha retains the characteristics of spontaneous alpha. Instead, we show that the part of the brain response that is phase locked to the visual stimulation increased with attention (as do steady-state evoked potentials), while the typical suppression was only present in non-stimulus-locked alpha activity. The opposite signs of these effects suggest that attentional modulation of dynamic visual stimulation relies on two parallel cortical mechanisms-retinotopic alpha suppression and increased temporal tracking.

A Normalization Circuit Underlying Coding of Spatial Attention in Primate Lateral Prefrontal Cortex

Lateral prefrontal cortex (LPFC) neurons signal the allocation of voluntary attention; however, the neural computations underlying this function remain unknown. To investigate this, we recorded from neuronal ensembles in the LPFC of two Macaca fascicularis performing a visuospatial attention task. LPFC neural responses to a single stimulus were normalized when additional stimuli/distracters appeared across the visual field and were well-characterized by an averaging computation. Deploying attention toward an individual stimulus surrounded by distracters shifted neural activity from an averaging regime toward a regime similar to that when the attended stimulus was presented in isolation (winner-take-all; WTA). However, attentional modulation is both qualitatively and quantitatively dependent on a neuron's visuospatial tuning. Our results show that during attentive vision, LPFC neuronal ensemble activity can be robustly read out by downstream areas to generate motor commands, and/or fed back into sensory areas to filter out distracter signals in favor of target signals.

Separating memoranda in depth increases visual working memory performance

Visual working memory is the mechanism supporting the continued maintenance of information after sensory inputs are removed. Although the capacity of visual working memory is limited, memoranda that are spaced farther apart on a 2-D display are easier to remember, potentially because neural representations are more distinct within retinotopically organized areas of visual cortex during memory encoding, maintenance, or retrieval. The impact on memory of spatial separability in depth is less clear, even though depth information is essential to guiding interactions with objects in the environment. On one account, separating memoranda in depth may facilitate performance if interference between items is reduced. However, depth information must be inferred indirectly from the 2-D retinal image, and less is known about how visual cortex represents depth. Thus, an alternative possibility is that separation in depth does not attenuate between-items interference; it may even impair performance, as attention must be distributed across a larger volume of 3-D space. We tested these alternatives using a stereo display while participants remembered the colors of stimuli presented either near or far in the 2-D plane or in depth. Increasing separation in-plane and in depth both enhanced performance. Furthermore, participants who were better able to utilize stereo depth cues showed larger benefits when memoranda were separated in depth, particularly for large memory arrays. The observation that spatial separation in the inferred 3-D structure of the environment improves memory performance, as is the case in 2-D environments, suggests that separating memoranda in depth might reduce neural competition by utilizing cortically separable resources.

Lateralization of Executive Function: Working Memory Advantage for Same Hemifield Stimuli in the Monkey

Working memory capacity, the amount of information that may be maintained in mind over a period of seconds, is extremely limited, to a handful of items. Some evidence exists that the number of visual items that may be maintained in working memory is independent for the two hemifields. To test this idea, we trained monkeys to perform visual working memory tasks that required maintenance in memory of the locations and/or shapes of 3–5 visual stimuli. We then tested whether systematic performance differences were present for stimuli concentrated in the same hemifield, vs. distributed across hemifields. We found little evidence to support the expectation that working memory capacity is independent in the two hemifields. Instead, when an advantage of stimulus arrangement was present, it involved multiple stimuli presented in the same hemifield. This conclusion was consistent across variations of the task, performance levels, and apparent strategies adopted by individual subjects. This result suggests that factors such as grouping that favor processing of stimuli in relative proximity may counteract the benefits of independent processing in the two hemispheres. Our results reveal an important property of working memory and place constraints on models of working memory capacity.

Audio-visual synchrony and spatial attention enhance processing of dynamic visual stimulation independently and in parallel: A frequency-tagging study

The neural processing of a visual stimulus can be facilitated by attending to its position or by a co-occurring auditory tone. Using frequency-tagging, we investigated whether facilitation by spatial attention and audio-visual synchrony rely on similar neural processes. Participants attended to one of two flickering Gabor patches (14.17 and 17 Hz) located in opposite lower visual fields. Gabor patches further "pulsed" (i.e. showed smooth spatial frequency variations) at distinct rates (3.14 and 3.63 Hz). Frequency-modulating an auditory stimulus at the pulse-rate of one of the visual stimuli established audio-visual synchrony. Flicker and pulsed stimulation elicited stimulus-locked rhythmic electrophysiological brain responses that allowed tracking the neural processing of simultaneously presented Gabor patches. These steady-state responses (SSRs) were quantified in the spectral domain to examine visual stimulus processing under conditions of synchronous vs. asynchronous tone presentation and when respective stimulus positions were attended vs. unattended. Strikingly, unique patterns of effects on pulse- and flicker driven SSRs indicated that spatial attention and audiovisual synchrony facilitated early visual processing in parallel and via different cortical processes. We found attention effects to resemble the classical top-down gain effect facilitating both, flicker and pulse-driven SSRs. Audio-visual synchrony, in turn, only amplified synchrony-producing stimulus aspects (i.e. pulse-driven SSRs) possibly highlighting the role of temporally co-occurring sights and sounds in bottom-up multisensory integration.

Graded Neuronal Modulations Related to Visual Spatial Attention

Studies of visual attention in monkeys typically measure neuronal activity when the stimulus event to be detected occurs at a cued location versus when it occurs at an uncued location. But this approach does not address how neuronal activity changes relative to conditions where attention is unconstrained by cueing. Human psychophysical studies have used neutral cueing conditions and found that neutrally cued behavioral performance is generally intermediate to that of cued and uncued conditions (Posner et al., 1978; Mangun and Hillyard, 1990; Montagna et al., 2009). To determine whether the neuronal correlates of visual attention during neutral cueing are similarly intermediate, we trained macaque monkeys to detect changes in stimulus orientation that were more likely to occur at one location (cued) than another (uncued), or were equally likely to occur at either stimulus location (neutral). Consistent with human studies, performance was best when the location was cued, intermediate when both locations were neutrally cued, and worst when the location was uncued. Neuronal modulations in visual area V4 were also graded as a function of cue validity and behavioral performance. By recording from both hemispheres simultaneously, we investigated the possibility of switching attention between stimulus locations during neutral cueing. The results failed to support a unitary "spotlight" of attention. Overall, our findings indicate that attention-related changes in V4 are graded to accommodate task demands.

SIGNIFICANCE STATEMENT

Studies of the neuronal correlates of attention in monkeys typically use visual cues to manipulate where attention is focused ("cued" vs "uncued"). Human psychophysical studies often also include neutrally cued trials to study how attention naturally varies between points of interest. But the neuronal correlates of this neutral condition are unclear. We measured behavioral performance and neuronal activity in cued, uncued, and neutrally cued blocks of trials. Behavioral performance and neuronal responses during neutral cueing were intermediate to those of the cued and uncued conditions. We found no signatures of a single mechanism of attention that switches between stimulus locations. Thus, attention-related changes in neuronal activity are largely hemisphere-specific and graded according to task demands.

Attention to Multiple Objects Facilitates Their Integration in Prefrontal and Parietal Cortex

Selective attention is known to interact with perceptual organization. In visual scenes, individual objects that are distinct and discriminable may occur on their own, or in groups such as a stack of books. The main objective of this study is to probe the neural interaction that occurs between individual objects when attention is directed toward one or more objects. Here we record steady-state visual evoked potentials via electrocorticography to directly assess the responses to individual stimuli and to their interaction. When human participants attend to two adjacent stimuli, prefrontal and parietal cortex shows a selective enhancement of only the neural interaction between stimuli, but not the responses to individual stimuli. When only one stimulus is attended, the neural response to that stimulus is selectively enhanced in prefrontal and parietal cortex. In contrast, early visual areas generally manifest responses to individual stimuli and to their interaction regardless of attentional task, although a subset of the responses is modulated similarly to prefrontal and parietal cortex. Thus, the neural representation of the visual scene as one progresses up the cortical hierarchy becomes more highly task-specific and represents either individual stimuli or their interaction, depending on the behavioral goal. Attention to multiple objects facilitates an integration of objects akin to perceptual grouping.SIGNIFICANCE STATEMENT Individual objects in a visual scene are seen as distinct entities or as parts of a whole. Here we examine how attention to multiple objects affects their neural representation. Previous studies measured single-cell or fMRI responses and obtained only aggregate measures that combined the activity to individual stimuli as well as their potential interaction. Here, we directly measure electrocorticographic steady-state responses corresponding to individual objects and to their interaction using a frequency-tagging technique. Attention to two stimuli increases the interaction component that is a hallmark for perceptual integration of stimuli. Furthermore, this stimulus-specific interaction is represented in prefrontal and parietal cortex in a task-dependent manner.

Characterizing object- and position-dependent response profiles to uni- and bilateral stimulus configurations in human higher visual cortex: a 7T fMRI study

Visual scenes are initially processed via segregated neural pathways dedicated to either of the two visual hemifields. Although higher-order visual areas are generally believed to utilize invariant object representations (abstracted away from features such as stimulus position), recent findings suggest they retain more spatial information than previously thought. Here, we assessed the nature of such higher-order object representations in human cortex using high-resolution fMRI at 7T, supported by corroborative 3T data. We show that multi-voxel activation patterns in both the contra- and ipsilateral hemisphere can be exploited to successfully classify the object category of unilaterally presented stimuli. Moreover, robustly identified rank order-based response profiles demonstrated a strong contralateral bias which frequently outweighed object category preferences. Finally, we contrasted different combinatorial operations to predict the responses during bilateral stimulation conditions based on responses to their constituent unilateral elements. Results favored a max operation predominantly reflecting the contralateral stimuli. The current findings extend previous work by showing that configuration-dependent modulations in higher-order visual cortex responses as observed in single unit activity have a counterpart in human neural population coding. They furthermore corroborate the emerging view that position coding is a fundamental functional characteristic of ventral visual stream processing.

Visual cortex responses reflect temporal structure of continuous quasi-rhythmic sensory stimulation

Neural processing of dynamic continuous visual input, and cognitive influences thereon, are frequently studied in paradigms employing strictly rhythmic stimulation. However, the temporal structure of natural stimuli is hardly ever fully rhythmic but possesses certain spectral bandwidths (e.g. lip movements in speech, gestures). Examining periodic brain responses elicited by strictly rhythmic stimulation might thus represent ideal, yet isolated cases. Here, we tested how the visual system reflects quasi-rhythmic stimulation with frequencies continuously varying within ranges of classical theta (4–7 Hz), alpha (8–13 Hz) and beta bands (14–20 Hz) using EEG. Our findings substantiate a systematic and sustained neural phase-locking to stimulation in all three frequency ranges. Further, we found that allocation of spatial attention enhances EEG-stimulus locking to theta- and alpha-band stimulation. Our results bridge recent findings regarding phase locking (“entrainment”) to quasi-rhythmic visual input and “frequency-tagging” experiments employing strictly rhythmic stimulation. We propose that sustained EEG-stimulus locking can be considered as a continuous neural signature of processing dynamic sensory input in early visual cortices. Accordingly, EEG-stimulus locking serves to trace the temporal evolution of rhythmic as well as quasi-rhythmic visual input and is subject to attentional bias.

•

Dynamic visual stimuli constitute large parts of our perceptual experience.

•

Strictly rhythmic dynamics condense in EEG-recorded mass-neural activity.

•

We tested how stimuli with fluctuating rhythms reflect in the EEG.

•

We found that the EEG allows tracing two quasi-rhythmic stimuli in parallel.

•

Dynamics of attended stimuli may be tracked with greater temporal precision.

Visual attention spreads broadly but selects information locally

Visual attention spreads over a range around the focus as the spotlight metaphor describes. Spatial spread of attentional enhancement and local selection/inhibition are crucial factors determining the profile of the spatial attention. Enhancement and ignorance/suppression are opposite effects of attention, and appeared to be mutually exclusive. Yet, no unified view of the factors has been provided despite their necessity for understanding the functions of spatial attention. This report provides electroencephalographic and behavioral evidence for the attentional spread at an early stage and selection/inhibition at a later stage of visual processing. Steady state visual evoked potential showed broad spatial tuning whereas the P3 component of the event related potential showed local selection or inhibition of the adjacent areas. Based on these results, we propose a two-stage model of spatial attention with broad spread at an early stage and local selection at a later stage.

Effects of reward on oculomotor control

The present study examines the extent to which distractors that signal the availability of monetary reward on a given trial affect eye movements. We used a novel eye movement task in which observers had to follow a target around the screen while ignoring distractors presented at varying locations. We examined the effects of reward magnitude and distractor location on a host of oculomotor properties, including saccade latency, amplitude, landing position, curvature, and erroneous saccades toward the distractor. We found consistent effects of reward magnitude on classic oculomotor phenomena such as the remote distractor effect, the global effect, and oculomotor capture by the distractor. We also show that a distractor in the visual hemifield opposite to the target had a larger effect on oculomotor control than an equidistant distractor in the same hemifield as the target. Bayesian hierarchical drift diffusion modeling revealed large differences in drift rate depending on the reward value, location, and visual hemifield of the distractor stimulus. Our findings suggest that high reward distractors not only capture the eyes but also affect a multitude of oculomotor properties associated with oculomotor inhibition and control.

Flickering task-irrelevant distractors induce dilation of target duration depending upon cortical distance

Flickering stimuli are perceived to be longer than stable stimuli. This so-called “flicker-induced time dilation” has been investigated in a number of studies, but the factors critical for this effect remain unclear. We explored the spatial distribution of the flicker effect and examined how the flickering task-irrelevant distractors spatially distant from the target induce time dilation. In two experiments, we demonstrated that flickering distractors dilated the perceived duration of the target stimulus even though the target stimulus itself was stable. In addition, when the distractor duration was much longer than the target duration, a flickering distractor located ipsilateral to the target caused greater time dilation than did a contralateral distractor. Thus the amount of dilation depended on the distance between the cortical areas responsible for the stimulus locations. These findings are consistent with the recent study reporting that modulation of neural oscillators encoding the interval duration could explain flicker-induced time dilation.

The neuroscience of working memory capacity and training

Working memory - the ability to maintain and manipulate information over a period of seconds - is a core component of higher cognitive functions. The storage capacity of working memory is limited but can be expanded by training, and evidence of the neural mechanisms underlying this effect is accumulating. Human imaging studies and neurophysiological recordings in non-human primates, together with computational modelling studies, reveal that training increases the activity of prefrontal neurons and the strength of connectivity in the prefrontal cortex and between the prefrontal and parietal cortex. Dopaminergic transmission could have a facilitatory role. These changes more generally inform us of the plasticity of higher cognitive functions.

Covert enaction at work: Recording the continuous movements of visuospatial attention to visible or imagined targets by means of Steady-State Visual Evoked Potentials (SSVEPs)

Whereas overt visuospatial attention is customarily measured with eye tracking, covert attention is assessed by various methods. Here we exploited Steady-State Visual Evoked Potentials (SSVEPs) - the oscillatory responses of the visual cortex to incoming flickering stimuli - to record the movements of covert visuospatial attention in a way operatively similar to eye tracking (attention tracking), which allowed us to compare motion observation and motion extrapolation with and without eye movements. Observers fixated a central dot and covertly tracked a target oscillating horizontally and sinusoidally. In the background, the left and the right halves of the screen flickered at two different frequencies, generating two SSVEPs in occipital regions whose size varied reciprocally as observers attended to the moving target. The two signals were combined into a single quantity that was modulated at the target frequency in a quasi-sinusoidal way, often clearly visible in single trials. The modulation continued almost unchanged when the target was switched off and observers mentally extrapolated its motion in imagery, and also when observers pointed their finger at the moving target during covert tracking, or imagined doing so. The amplitude of modulation during covert tracking was ∼25-30% of that measured when observers followed the target with their eyes. We used 4 electrodes in parieto-occipital areas, but similar results were achieved with a single electrode in Oz. In a second experiment we tested ramp and step motion. During overt tracking, SSVEPs were remarkably accurate, showing both saccadic-like and smooth pursuit-like modulations of cortical responsiveness, although during covert tracking the modulation deteriorated. Covert tracking was better with sinusoidal motion than ramp motion, and better with moving targets than stationary ones. The clear modulation of cortical responsiveness recorded during both overt and covert tracking, identical for motion observation and motion extrapolation, suggests to include covert attention movements in enactive theories of mental imagery.

Sustained Splits of Attention within versus across Visual Hemifields Produce Distinct Spatial Gain Profiles

Visual attention can be focused concurrently on two stimuli at noncontiguous locations while intermediate stimuli remain ignored. Nevertheless, behavioral performance in multifocal attention tasks falters when attended stimuli fall within one visual hemifield as opposed to when they are distributed across left and right hemifields. This "different-hemifield advantage" has been ascribed to largely independent processing capacities of each cerebral hemisphere in early visual cortices. Here, we investigated how this advantage influences the sustained division of spatial attention. We presented six isoeccentric light-emitting diodes (LEDs) in the lower visual field, each flickering at a different frequency. Participants attended to two LEDs that were spatially separated by an intermediate LED and responded to synchronous events at to-be-attended LEDs. Task-relevant pairs of LEDs were either located in the same hemifield ("within-hemifield" conditions) or separated by the vertical meridian ("across-hemifield" conditions). Flicker-driven brain oscillations, steady-state visual evoked potentials (SSVEPs), indexed the allocation of attention to individual LEDs. Both behavioral performance and SSVEPs indicated enhanced processing of attended LED pairs during "across-hemifield" relative to "within-hemifield" conditions. Moreover, SSVEPs demonstrated effective filtering of intermediate stimuli in "across-hemifield" condition only. Thus, despite identical physical distances between LEDs of attended pairs, the spatial profiles of gain effects differed profoundly between "across-hemifield" and "within-hemifield" conditions. These findings corroborate that early cortical visual processing stages rely on hemisphere-specific processing capacities and highlight their limiting role in the concurrent allocation of visual attention to multiple locations.

The steady-state visual evoked potential in vision research: A review

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.