Using frequency tagging to quantify attentional deployment in a visual divided attention task

Measuring attentional selection of object categories using hierarchical frequency tagging

In the present study, we used Hierarchical Frequency Tagging (Gordon et al., 2017) to investigate in electroencephalography how different levels of the neural processing hierarchy interact with category-selective attention during visual object recognition. We constructed stimulus sequences of cyclic wavelet scrambled face and house stimuli at two different frequencies (f1 = 0.8 Hz and f2 = 1 Hz). For each trial, two stimulus sequences of different frequencies were superimposed and additionally augmented by a sinusoidal contrast modulation with f3 = 12.5 Hz. This allowed us to simultaneously assess higher level processing using semantic wavelet-induced frequency-tagging (SWIFT) and processing in earlier visual levels using steady-state visually evoked potentials (SSVEPs), along with their intermodulation (IM) components. To investigate the category specificity of the SWIFT signal, we manipulated the category congruence between target and distractor by superimposing two sequences containing stimuli from the same or different object categories. Participants attended to one stimulus (target) and ignored the other (distractor). Our results showed successful tagging of different levels of the cortical hierarchy. Using linear mixed-effects modeling, we detected different attentional modulation effects on lower versus higher processing levels. SWIFT and IM components were substantially increased for target versus distractor stimuli, reflecting attentional selection of the target stimuli. In addition, distractor stimuli from the same category as targets elicited stronger SWIFT signals than distractor stimuli from a different category indicating category-selective attention. In contrast, for IM components, this category-selective attention effect was largely absent, indicating that IM components probably reflect more stimulus-specific processing.

Division and spreading of attention across color

Biological systems must allocate limited perceptual resources to relevant elements in their environment. This often requires simultaneous selection of multiple elements from the same feature dimension (e.g. color). To establish the determinants of divided attentional selection of color, we conducted an experiment that used multicolored displays with four overlapping random dot kinematograms that differed only in hue. We manipulated (i) requirement to focus attention to a single color or divide it between two colors; (ii) distances of distractor hues from target hues in a perceptual color space. We conducted a behavioral and an electroencephalographic experiment, in which each color was tagged by a specific flicker frequency and driving its own steady-state visual evoked potential. Behavioral and neural indices of attention showed several major consistencies. Concurrent selection halved the neural signature of target enhancement observed for single targets, consistent with an approximately equal division of limited resources between two hue-selective foci. Distractors interfered with behavioral performance in a context-dependent fashion but their effects were asymmetric, indicating that perceptual distance did not adequately capture attentional distance. These asymmetries point towards an important role of higher-level mechanisms such as categorization and grouping-by-color in determining the efficiency of attentional allocation in complex, multicolored scenes.

Steady-state Visual Evoked Potentials Reveal Dynamic (Re)allocation of Spatial Attention during Maintenance and Utilization of Visual Working Memory

Visual working memory (VWM) allows storing goal-relevant information to guide future behavior. Prior work suggests that VWM is spatially organized and relies on spatial attention directed toward locations at which memory items were encoded, even if location is task-irrelevant. Importantly, attention often needs to be dynamically redistributed between locations, for example, in preparation for an upcoming probe. Very little is known about how attentional resources are distributed between multiple locations during a VWM task and even less about the dynamic changes governing such attentional shifts over time. This is largely due to the inability to use behavioral outcomes to reveal fast dynamic changes within trials. We here demonstrated that EEG steady-state visual evoked potentials (SSVEPs) successfully track the dynamic allocation of spatial attention during a VWM task. Participants were presented with to-be-memorized gratings and distractors at two distinct locations, tagged with flickering discs. This allowed us to dynamically track attention allocated to memory and distractor items via their coupling with space by quantifying the amplitude and coherence of SSVEP responses in the EEG signal to flickering stimuli at the former memory and distractor locations. SSVEP responses did not differ between memory and distractor locations during early maintenance. However, shortly before probe comparison, we observed a decrease in SSVEP coherence over distractor locations indicative of a reallocation of spatial attentional resources. RTs were shorter when preceded by stronger decreases in SSVEP coherence at distractor locations, likely reflecting attentional shifts from the distractor to the probe or memory location. We demonstrate that SSVEPs can inform about dynamic processes in VWM, even if location does not have to be reported by participants. This finding not only supports the notion of a spatially organized VWM but also reveals that SSVEPs betray a dynamic prioritization process of working memory items and locations over time that is directly predictive of memory performance.

Attention Drives Visual Processing and Audiovisual Integration During Multimodal Communication

During communication in real-life settings, our brain often needs to integrate auditory and visual information and at the same time actively focus on the relevant sources of information, while ignoring interference from irrelevant events. The interaction between integration and attention processes remains poorly understood. Here, we use rapid invisible frequency tagging and magnetoencephalography to investigate how attention affects auditory and visual information processing and integration, during multimodal communication. We presented human participants (male and female) with videos of an actress uttering action verbs (auditory; tagged at 58 Hz) accompanied by two movie clips of hand gestures on both sides of fixation (attended stimulus tagged at 65 Hz; unattended stimulus tagged at 63 Hz). Integration difficulty was manipulated by a lower-order auditory factor (clear/degraded speech) and a higher-order visual semantic factor (matching/mismatching gesture). We observed an enhanced neural response to the attended visual information during degraded speech compared to clear speech. For the unattended information, the neural response to mismatching gestures was enhanced compared to matching gestures. Furthermore, signal power at the intermodulation frequencies of the frequency tags, indexing nonlinear signal interactions, was enhanced in the left frontotemporal and frontal regions. Focusing on the left inferior frontal gyrus, this enhancement was specific for the attended information, for those trials that benefitted from integration with a matching gesture. Together, our results suggest that attention modulates audiovisual processing and interaction, depending on the congruence and quality of the sensory input.

Attentional Modulation in Early Visual Cortex: A Focused Reanalysis of Steady-state Visual Evoked Potential Studies

Steady-state visual evoked potentials (SSVEPs) are a powerful tool for investigating selective attention. Here, we conducted a combined reanalysis of multiple studies employing this technique in a variety of attentional experiments to, first, establish benchmark effect sizes of attention on amplitude and phase of SSVEPs and, second, harness the power of a large data set to test more specific hypotheses. Data of eight published SSVEP studies were combined, in which human participants (n = 135 in total) attended to flickering random dot stimuli based on their defining features (e.g., location, color, luminance, or orientation) or feature conjunctions. The reanalysis established that, in all the studies, attention reliably enhanced amplitudes, with color-based attention providing the strongest effect. In addition, the latency of SSVEPs elicited by attended stimuli was reduced by ∼4 msec. Next, we investigated the modulation of SSVEP amplitudes in a subset of studies where two different features were attended concurrently. Although most models assume that attentional effects of multiple features are combined additively, our results suggest that neuronal enhancement provided by concurrent attention is better described by multiplicative integration. Finally, we used the combined data set to demonstrate that the increase in trial-averaged SSVEP amplitudes with attention cannot be explained by increased synchronization of single-trial phases. Contrary to the prediction of the phase-locking account, the variance across trials of complex Fourier coefficients increases with attention, which is more consistent with boosting of a largely phase-locked signal embedded in non-phase-locked noise.

Attentional modulation as a mechanism for enhanced facial emotion discrimination: The case of action video game players

Action video game players (AVGPs) outperform nonvideo game players (NVGPs) on a wide variety of attentional tasks, mediating benefits to perceptual and cognitive decision processes. A key issue in the literature is the extent to which such benefits transfer beyond cognition. Using steady-state visual evoked potentials (SSVEP) as a neural measure of attentional resource allocation, we investigated whether the attentional benefit of AVGPs generalizes to the processing of rapidly presented facial emotions. AVGPs (n = 36) and NVGPs (n = 32) performed a novel, attention-demanding emotion discrimination task, requiring the identification of a target emotion in one of two laterally presented streams of emotional faces. The emotional faces flickered at either 2.0 Hz or 2.5 Hz. AVGPs outperformed NVGPs at detecting the target emotions regardless of the type of emotion. Correspondingly, attentional modulation of the SSVEP at parieto-occipital recording sites was larger in AVGPs compared with NVGPs. This difference appeared to be driven by a larger response to attended information, as opposed to a reduced response to irrelevant distractor information. Exploratory analyses confirmed that this novel paradigm elicited the expected pattern of event-related potentials associated with target detection and error processing. These components did not, however, differ between groups. Overall, the results indicate enhanced discrimination of facial emotions in AVGPs arising from enhanced attentional processing of emotional information. This presents evidence for the attentional advantage of AVGPs to extend beyond perceptual and cognitive processes.

Improving AR-SSVEP Recognition Accuracy Under High Ambient Brightness Through Iterative Learning

Augmented reality-based brain-computer interface (AR-BCI) system is one of the important ways to promote BCI technology outside of the laboratory due to its portability and mobility, but its performance in real-world scenarios has not been fully studied. In the current study, we first investigated the effect of ambient brightness on AR-BCI performance. 5 different light intensities were set as experimental conditions to simulate typical brightness in real scenes, while the same steady-state visual evoked potentials (SSVEP) stimulus was displayed in the AR glass. The data analysis results showed that SSVEP can be evoked under all 5 light intensities, but the response intensity became weaker when the brightness increased. The recognition accuracies of AR-SSVEP were negatively correlated to light intensity, the highest accuracies were 89.35% with FBCCA and 83.33% with CCA under 0 lux light intensity, while they decreased to 62.53% and 49.24% under 1200 lux. To solve the accuracy loss problem in high ambient brightness, we further designed a SSVEP recognition algorithm with iterative learning capability, named ensemble online adaptive CCA (eOACCA). The main strategy is to provide initial filters for high-intensity data by iteratively learning low-light-intensity AR-SSVEP data. The experimental results showed that the eOACCA algorithm had significant advantages under higher light intensities ( 600 lux). Compared with FBCCA, the accuracy of eOACCA under 1200 lux was increased by 13.91%. In conclusion, the current study contributed to the in-depth understanding of the performance variations of AR-BCI under different lighting conditions, and was helpful in promoting the AR-BCI application in complex lighting environments.

Application of rapid invisible frequency tagging for brain computer interfaces

BACKGROUND

Brain-computer interfaces (BCI) based on steady-state visual evoked potentials (SSVEPs/SSVEFs) are among the most commonly used BCI systems. They require participants to covertly attend to visual objects flickering at specified frequencies. The attended location is decoded online by analysing the power of neuronal responses at the flicker frequency.

NEW METHOD

We implemented a novel rapid invisible frequency-tagging technique, utilizing a state-of-the-art projector with refresh rates of up to 1440 Hz. We flickered the luminance of visual objects at 56 and 60 Hz, which was invisible to participants but produced strong neuronal responses measurable with magnetoencephalography (MEG). The direction of covert attention, decoded from frequency-tagging responses, was used to control an online BCI PONG game.

RESULTS

Our results show that seven out of eight participants were able to play the pong game controlled by the frequency-tagging signal, with average accuracies exceeding 60 %. Importantly, participants were able to modulate the power of the frequency-tagging response within a 1-second interval, while only seven occipital sensors were required to reliably decode the neuronal response.

COMPARISON WITH EXISTING METHODS

In contrast to existing SSVEP-based BCI systems, rapid frequency-tagging does not produce a visible flicker. This extends the time-period participants can use it without fatigue, by avoiding distracting visual input. Furthermore, higher frequencies increase the temporal resolution of decoding, resulting in higher communication rates.

CONCLUSION

Using rapid invisible frequency-tagging opens new avenues for fundamental research and practical applications. In combination with novel optically pumped magnetometers (OPMs), it could facilitate the development of high-speed and mobile next-generation BCI systems.

Attentional Enhancement of Tracked Stimuli in Early Visual Cortex Has Limited Capacity

Keeping track of the location of multiple moving objects is one of the well documented functions of visual attention. However, the mechanism of attentional selection that supports such continuous tracking is unclear. In particular, it has been proposed that target selection in early visual cortex occurs in parallel, with tracking errors arising because of attentional limitations at later processing stages. Here, we examine whether, instead, total attentional capacity for enhancement of early visual processing of tracked targets is shared between all attended stimuli. If the magnitude of attentional facilitation of multiple tracked targets was a key limiting factor of tracking ability, then one should expect it to drop systematically with increasing set-size of tracked targets. Human observers (male and female) were instructed to track two, four, or six moving objects among a pool of identical distractors. Steady-state visual evoked potentials (SSVEPs) recorded during the tracking period revealed that the processing of tracked targets was consistently amplified compared with the processing of the distractors. The magnitude of this amplification decreased with increasing set size, and at lateral occipital electrodes it closely followed inverse proportionality to the number of tracked items, suggesting that limited attentional resources must be shared among the tracked stimuli. Accordingly, the magnitude of attentional facilitation predicted the behavioral outcome at the end of the trial. Together, these findings demonstrate that the limitations of multiple object tracking (MOT) across set-sizes stem from the limitations of top-down selective attention already at the early stages of visual processing.SIGNIFICANCE STATEMENT The ability to selectively attend to relevant features or objects is the key to flexibility of perception and action in the continuously changing environment. This ability is demonstrated in the multiple object tracking (MOT) task where observers monitor multiple independently moving objects at different locations in the visual field. The role of early attentional enhancement in tracking was previously acknowledged in the literature, however, the limitations on tracking were thought to arise during later stages of processing. Here, we demonstrate that the strength of attentional facilitation depends on the number of tracked objects and predicts successful tracking performance. Thus, it is the limitations of attentional enhancement at the early stages of visual processing that determine behavioral performance limits.

Frequency tagging with infants: The visual oddball paradigm

Combining frequency tagging with electroencephalography (EEG) provides excellent opportunities for developmental research and is increasingly employed as a powerful tool in cognitive neuroscience within the last decade. In particular, the visual oddball paradigm has been employed to elucidate face and object categorization and intermodal influences on visual perception. Still, EEG research with infants poses special challenges that require consideration and adaptations of analyses. These challenges include limits to attentional capacity, variation in looking times, and presence of artefacts in the EEG signal. Moreover, potential differences between age-groups must be carefully evaluated. This manuscript evaluates challenges theoretically and empirically by (1) a systematic review of frequency tagging studies employing the oddball paradigm and (2) combining and re-analyzing data from seven-month-old infants (N = 124, 59 females) collected in a categorization task with artifical, unfamiliar stimuli. Specifically, different criteria for sequence retention and selection of harmonics, the influence of bins considered for baseline correction and the relation between fast periodic visual stimulation (FPVS) responses and looking time are analyzed. Overall, evidence indicates that analysis decisions should be tailored based on age-group to optimally capture the observed signal. Recommendations for infant frequency tagging studies are developed to aid researchers in selecting appropriate stimulation and analysis strategies in future work.

Cognitive tasks propagate the neural entrainment in response to a visual 40 Hz stimulation in humans

Introduction

Alzheimer's disease is one of the great challenges in the coming decades, and despite great efforts, a widely effective disease-modifying therapy in humans remains elusive. One particular promising non-pharmacological therapy that has received increased attention in recent years is based on the Gamma ENtrainment Using Sensory stimulation (GENUS), a high-frequency neural response elicited by a visual and/or auditory stimulus at 40 Hz. While this has shown to be effective in animal models, studies on human participants have reported varying success. The current work hypothesizes that the varying success in humans is due to differences in cognitive workload during the GENUS sessions.

Methods

We recruited a cohort of 15 participants who underwent a scalp-EEG recording as well as one epilepsy patient who was implanted with 50 subdural surface electrodes over temporo-occipital and temporo-basal cortex and 14 depth contacts that targeted the hippocampus and insula. All participants completed several GENUS sessions, in each of which a different cognitive task was performed.

Results

We found that the inclusion of a cognitive task during the GENUS session not only has a positive effect on the strength and extent of the gamma entrainment, but also promotes the propagation of gamma entrainment to additional neural areas including deep ones such as hippocampus which were not recruited when no cognitive task was required from the participants. The latter is of particular interest given that the hippocampal complex is considered to be one of the primary targets for AD therapies.

Discussion

This work introduces a possible improvement strategy for GENUS therapy that might contribute to increasing the efficacy of the therapy or shortening the time needed for the positive outcome.

The steady state visual evoked potential (SSVEP) tracks "sticky" thinking, but not more general mind-wandering

For a large proportion of our daily lives, spontaneously occurring thoughts tend to disengage our minds from goal-directed thinking. Previous studies showed that EEG features such as the P3 and alpha oscillations can predict mind-wandering to some extent, but only with accuracies of around 60%. A potential candidate for improving prediction accuracy is the Steady-State Visual Evoked Potential (SSVEP), which is used frequently in single-trial contexts such as brain-computer interfaces as a marker of the direction of attention. In this study, we modified the sustained attention to response task (SART) that is usually employed to measure spontaneous thought to incorporate the SSVEP elicited by a 12.5-Hz flicker. We then examined whether the SSVEP could track and allow for the prediction of the stickiness and task-relatedness dimensions of spontaneous thought. Our results show that the SSVEP evoked by flickering words was able to distinguish between more and less sticky thinking but not between whether a participant was on- or off-task. This suggests that the SSVEP is able to track spontaneous thinking when it is strongly disengaged from the task (as in the sticky form of off-task thinking) but not off-task thought in general. Future research should determine the exact dimensions of spontaneous thought to which the SSVEP is most sensitive.

Local Interactions between Steady-State Visually Evoked Potentials at Nearby Flickering Frequencies

Steady-state visually evoked potentials (SSVEPs) are widely used to index top-down cognitive processing in human electroencephalogram (EEG) studies. Typically, two stimuli flickering at different temporal frequencies (TFs) are presented, each producing a distinct response in the EEG at its flicker frequency. However, how SSVEP responses in EEGs are modulated in the presence of a competing flickering stimulus just because of sensory interactions is not well understood. We have previously shown in local field potentials (LFPs) recorded from awake monkeys that when two overlapping full-screen gratings are counterphased at different TFs, there is an asymmetric SSVEP response suppression, with greater suppression from lower TFs, which further depends on the relative orientations of the gratings (stronger suppression and asymmetry for parallel compared with orthogonal gratings). Here, we first confirmed these effects in both male and female human EEG recordings. Then, we mapped the response suppression of one stimulus (target) by a competing stimulus (mask) over a much wider range than the previous study. Surprisingly, we found that the suppression was not stronger at low frequencies in general, but systematically varied depending on the target TF, indicating local interactions between the two competing stimuli. These results were confirmed in both human EEG and monkey LFP and electrocorticogram (ECoG) data. Our results show that sensory interactions between multiple SSVEPs are more complex than shown previously and are influenced by both local and global factors, underscoring the need to cautiously interpret the results of studies involving SSVEP paradigms.SIGNIFICANCE STATEMENT Steady-state visually evoked potentials (SSVEPs) are extensively used in human cognitive studies and brain-computer interfacing applications where multiple stimuli flickering at distinct frequencies are concurrently presented in the visual field. We recently characterized interactions between competing flickering stimuli in animal recordings and found that stimuli flickering slowly produce larger suppression. Here, we confirmed these in human EEGs, and further characterized the interactions by using a much wider range of target and competing (mask) frequencies in both human EEGs and invasive animal recordings. These revealed a new "local" component, whereby the suppression increased when competing stimuli flickered at nearby frequencies. Our results highlight the complexity of sensory interactions among multiple SSVEPs and underscore the need to cautiously interpret studies involving SSVEP paradigms.

Brain Oscillation Entrainment by Perceptible and Non-perceptible Rhythmic Light Stimulation

Objective and Background: Decades of research in the field of steady-state visual evoked potentials (SSVEPs) have revealed great potential of rhythmic light stimulation for brain-computer interfaces. Additionally, rhythmic light stimulation provides a non-invasive method for entrainment of oscillatory activity in the brain. Especially effective protocols enabling non-perceptible rhythmic stimulation and, thereby, reducing eye fatigue and user discomfort are favorable. Here, we investigate effects of (1) perceptible and (2) non-perceptible rhythmic light stimulation as well as attention-based effects of the stimulation by asking participants to focus (a) on the stimulation source directly in an overt attention condition or (b) on a cross-hair below the stimulation source in a covert attention condition. Method: SSVEPs at 10 Hz were evoked with a light-emitting diode (LED) driven by frequency-modulated signals and amplitudes of the current intensity either below or above a previously estimated individual threshold. Furthermore, we explored the effect of attention by asking participants to fixate on the LED directly in the overt attention condition and indirectly attend it in the covert attention condition. By measuring electroencephalography, we analyzed differences between conditions regarding the detection of reliable SSVEPs via the signal-to-noise ratio (SNR) and functional connectivity in occipito-frontal(-central) regions. Results: We could observe SSVEPs at 10 Hz for the perceptible and non-perceptible rhythmic light stimulation not only in the overt but also in the covert attention condition. The SNR and SSVEP amplitudes did not differ between the conditions and SNR values were in all except one participant above significance thresholds suggested by previous literature indicating reliable SSVEP responses. No difference between the conditions could be observed in the functional connectivity in occipito-frontal(-central) regions. Conclusion: The finding of robust SSVEPs even for non-intrusive rhythmic stimulation protocols below an individual perceptibility threshold and without direct fixation on the stimulation source reveals strong potential as a safe stimulation method for oscillatory entrainment in naturalistic applications.

Steady-State Visually Evoked Potentials and Feature-based Attention: Preregistered Null Results and a Focused Review of Methodological Considerations

Feature-based attention is the ability to selectively attend to a particular feature (e.g., attend to red but not green items while looking for the ketchup bottle in your refrigerator), and steady-state visually evoked potentials (SSVEPs) measured from the human EEG signal have been used to track the neural deployment of feature-based attention. Although many published studies suggest that we can use trial-by-trial cues to enhance relevant feature information (i.e., greater SSVEP response to the cued color), there is ongoing debate about whether participants may likewise use trial-by-trial cues to voluntarily ignore a particular feature. Here, we report the results of a preregistered study in which participants either were cued to attend or to ignore a color. Counter to prior work, we found no attention-related modulation of the SSVEP response in either cue condition. However, positive control analyses revealed that participants paid some degree of attention to the cued color (i.e., we observed a greater P300 component to targets in the attended vs. the unattended color). In light of these unexpected null results, we conducted a focused review of methodological considerations for studies of feature-based attention using SSVEPs. In the review, we quantify potentially important stimulus parameters that have been used in the past (e.g., stimulation frequency, trial counts) and we discuss the potential importance of these and other task factors (e.g., feature-based priming) for SSVEP studies.

Cross-Subject Assistance: Inter- and Intra-Subject Maximal Correlation for Enhancing the Performance of SSVEP-Based BCIs

OBJECTIVE

The current state-of-the-art methods significantly improve the detection performance of the steady-state visual evoked potentials (SSVEPs) by using the individual calibration data. However, the time-consuming calibration sessions limit the number of training trials and may give rise to visual fatigue, which weakens the effectiveness of the individual training data. For addressing this issue, this study proposes a novel inter- and intra-subject maximal correlation (IISMC) method to enhance the robustness of SSVEP recognition via employing the inter- and intra-subject similarity and variability. Through efficient transfer learning, similar experience under the same task is shared across subjects.

METHODS

IISMC extracts subject-specific information and similar task-related information from oneself and other subjects performing the same task by maximizing the inter- and intra-subject correlation. Multiple weak classifiers are built from several existing subjects and then integrated to construct the strong classifiers by the average weighting. Finally, a powerful fusion predictor is obtained for target recognition.

RESULTS

The proposed framework is validated on a benchmark data set of 35 subjects, and the experimental results demonstrate that IISMC obtains better performance than the state of the art task-related component analysis (TRCA).

SIGNIFICANCE

The proposed method has great potential for developing high-speed BCIs.

Reinforcement history shapes primary visual cortical responses: An SSVEP study

Efficient learning requires allocating limited attentional resources to meaningful stimuli and away from irrelevant stimuli. This prioritization may occur via covert attention, evident in the activity of the visual cortex. We used steady-state visual evoked potentials (SSVEPs) to assess whether associability-driven changes in stimulus processing were evident in visuocortical responses. Participants were trained on a learned-predictiveness protocol, whereby one stimulus on each trial accurately predicted the correct response for that trial, and the other was irrelevant. In a second phase the task was arranged so that all cues were objectively predictive. Participants' overt attention (eye gaze) was affected by each cue's reinforcement history, as was their covert attention (SSVEP responses). These biases persisted into Phase 2 when all stimuli were objectively predictive, thereby demonstrating that learned attentional processes are evident in basic sensory processing, and exert an effect on covert attention above and beyond the effects of overt gaze bias.

Exploring the temporal dynamics of inhibition of return using steady-state visual evoked potentials

Inhibition of return is characterized by delayed responses to previously attended locations when the interval between stimuli is long enough. The present study employed steady-state visual evoked potentials (SSVEPs) as a measure of attentional modulation to explore the nature and time course of input- and output-based inhibitory cueing mechanisms that each slow response times at previously stimulated locations under different experimental conditions. The neural effects of behavioral inhibition were examined by comparing post-cue SSVEPs between cued and uncued locations measured across two tasks that differed only in the response modality (saccadic or manual response to targets). Grand averages of SSVEP amplitudes for each condition showed a reduction in amplitude at cued locations in the window of 100-500 ms post-cue, revealing an early, short-term decrease in the responses of neurons that can be attributed to sensory adaptation, regardless of response modality. Because primary visual cortex has been found to be one of the major sources of SSVEP signals, the results suggest that the SSVEP modulations observed were caused by input-based inhibition that occurred in V1, or visual areas earlier than V1, as a consequence of reduced visual input activity at previously cued locations. No SSVEP modulations were observed in either response condition late in the cue-target interval, suggesting that neither late input- nor output-based IOR modulates SSVEPs. These findings provide further electrophysiological support for the theory of multiple mechanisms contributing to behavioral cueing effects.

Combined frequency-tagging EEG and eye-tracking measures provide no support for the "excess mouth/diminished eye attention" hypothesis in autism

BACKGROUND

Scanning faces is important for social interactions. Difficulty with the social use of eye contact constitutes one of the clinical symptoms of autism spectrum disorder (ASD). It has been suggested that individuals with ASD look less at the eyes and more at the mouth than typically developing (TD) individuals, possibly due to gaze aversion or gaze indifference. However, eye-tracking evidence for this hypothesis is mixed. While gaze patterns convey information about overt orienting processes, it is unclear how this is manifested at the neural level and how relative covert attention to the eyes and mouth of faces might be affected in ASD.

METHODS

We used frequency-tagging EEG in combination with eye tracking, while participants watched fast flickering faces for 1-min stimulation sequences. The upper and lower halves of the faces were presented at 6 Hz and 7.5 Hz or vice versa in different stimulation sequences, allowing to objectively disentangle the neural saliency of the eyes versus mouth region of a perceived face. We tested 21 boys with ASD (8-12 years old) and 21 TD control boys, matched for age and IQ.

RESULTS

Both groups looked longer at the eyes than the mouth, without any group difference in relative fixation duration to these features. TD boys looked significantly more to the nose, while the ASD boys looked more outside the face. EEG neural saliency data partly followed this pattern: neural responses to the upper or lower face half were not different between groups, but in the TD group, neural responses to the lower face halves were larger than responses to the upper part. Face exploration dynamics showed that TD individuals mostly maintained fixations within the same facial region, whereas individuals with ASD switched more often between the face parts.

LIMITATIONS

Replication in large and independent samples may be needed to validate exploratory results.

CONCLUSIONS

Combined eye-tracking and frequency-tagged neural responses show no support for the excess mouth/diminished eye gaze hypothesis in ASD. The more exploratory face scanning style observed in ASD might be related to their increased feature-based face processing style.

The SSVEP tracks attention, not consciousness, during perceptual filling-in

Research on the neural basis of conscious perception has almost exclusively shown that becoming aware of a stimulus leads to increased neural responses. By designing a novel form of perceptual filling-in (PFI) overlaid with a dynamic texture display, we frequency-tagged multiple disappearing targets as well as their surroundings. We show that in a PFI paradigm, the disappearance of a stimulus and subjective invisibility is associated with increases in neural activity, as measured with steady-state visually evoked potentials (SSVEPs), in electroencephalography (EEG). We also find that this increase correlates with alpha-band activity, a well-established neural measure of attention. These findings cast doubt on the direct relationship previously reported between the strength of neural activity and conscious perception, at least when measured with current tools, such as the SSVEP. Instead, we conclude that SSVEP strength more closely measures changes in attention.

In pursuit of visual attention: SSVEP frequency-tagging moving targets

Previous research has shown that visual attention does not always exactly follow gaze direction, leading to the concepts of overt and covert attention. However, it is not yet clear how such covert shifts of visual attention to peripheral regions impact the processing of the targets we directly foveate as they move in our visual field. The current study utilised the co-registration of eye-position and EEG recordings while participants tracked moving targets that were embedded with a 30 Hz frequency tag in a Steady State Visually Evoked Potentials (SSVEP) paradigm. When the task required attention to be divided between the moving target (overt attention) and a peripheral region where a second target might appear (covert attention), the SSVEPs elicited by the tracked target at the 30 Hz frequency band were significantly, but transiently, lower than when participants did not have to covertly monitor for a second target. Our findings suggest that neural responses of overt attention are only briefly reduced when attention is divided between covert and overt areas. This neural evidence is in line with theoretical accounts describing attention as a pool of finite resources, such as the perceptual load theory. Altogether, these results have practical implications for many real-world situations where covert shifts of attention may discretely reduce visual processing of objects even when they are directly being tracked with the eyes.

Interaction between steady-state visually evoked potentials at nearby flicker frequencies

Steady-state visually evoked potential (SSVEP) studies routinely employ simultaneous presentation of two temporally modulated stimuli, with SSVEP amplitude modulations serving to index top-down cognitive processes. However, the nature of SSVEP amplitude modulations as a function of competing temporal frequency (TF) has not been systematically studied, especially in relation to the normalization framework which has been extensively used to explain visual responses to multiple stimuli. We recorded spikes and local field potential (LFP) from the primary visual cortex (V1) as well as EEG from two awake macaque monkeys while they passively fixated plaid stimuli with components counterphasing at different TFs. We observed asymmetric SSVEP response suppression by competing TFs (greater suppression for lower TFs), which further depended on the relative orientations of plaid components. A tuned normalization model, adapted to SSVEP responses, provided a good account of the suppression. Our results provide new insights into processing of temporally modulated visual stimuli.

Hemispheric Asymmetries in Deaf and Hearing During Sustained Peripheral Selective Attention

Previous studies have shown that compared to hearing individuals, early deaf individuals allocate relatively more attention to the periphery than central visual field. However, it is not clear whether these two groups also differ in their ability to selectively attend to specific peripheral locations. We examined deaf and hearing participants' selective attention using electroencephalography (EEG) and a frequency tagging paradigm, in which participants attended to one of two peripheral displays of moving dots that changed directions at different rates. Both participant groups showed similar amplifications and reductions in the EEG signal at the attended and unattended frequencies, indicating similar control over their peripheral attention for motion stimuli. However, for deaf participants these effects were larger in a right hemispheric region of interest (ROI), while for hearing participants these effects were larger in a left ROI. These results contribute to a growing body of evidence for a right hemispheric processing advantage in deaf populations when attending to motion.

Combined frequency-tagging EEG and eye tracking reveal reduced social bias in boys with autism spectrum disorder

Developmental accounts of autism spectrum disorder (ASD) state that infants and children with ASD are spontaneously less attracted by and less proficient in processing social stimuli such as faces. This is hypothesized to partly underlie social communication difficulties in ASD. While in some studies a reduced preference for social stimuli has been shown in individuals with ASD, effect sizes are moderate and vary across studies, stimuli, and designs. Eye tracking, often the methodology of choice to study social preference, conveys information about overt orienting processes but conceals covert attention, possibly resulting in an underestimation of the effects. In this study, we recorded eye tracking and electroencephalography (EEG) during fast periodic visual stimulation to address this issue. We tested 21 boys with ASD (8-12 years old) and 21 typically developing (TD) control boys, matched for age and IQ. Streams of variable images of faces were presented at 6 Hz alongside images of houses presented at 7.5 Hz or vice versa, while children were engaged in an orthogonal task. While frequency-tagged neural responses were larger in response to faces than simultaneously presented houses in both groups, this effect was much larger in TD boys than in boys with ASD. This group difference in saliency of social versus non-social processing is significant after 5 sec of stimulus presentation and holds throughout the entire trial. Although there was no interaction between group and stimulus category for simultaneously recorded eye-tracking data, eye tracking and EEG measures were strongly correlated. We conclude that frequency-tagging EEG, allowing monitoring of both overt and covert processes, provides a fast, objective and reliable measure of decreased preference for social information in ASD.

No intermodal interference effects of threatening information during concurrent audiovisual stimulation

Changes in attention can result in sensory processing trade-off effects, in which sensory cortical responses to attended stimuli are heightened and responses to competing distractors are attenuated. However, it is unclear if competition or facilitation effects will be observed at the level of sensory cortex when attending to competing stimuli in two modalities. The present study used electroencephalogram (EEG) and frequency-tagging to quantitatively assess auditory-visual interactions during sustained multimodal sensory stimulation. The emotional content of a 6.66 Hz rapid serial visual presentation (RSVP) was manipulated to elicit well-established emotional attention effects, while a constant 63 dB tone with a 40.8 Hz modulation served as a concurrent auditory stimulus in two experiments. As a directed attention manipulation, participants were instructed to detect transient sound level events in the auditory stream in Experiment 1. To manipulate attention through threat anticipation, participants were instructed to expect an aversive noise burst after a higher 40.8 Hz modulated tone in Experiment 2. Each stimulus evoked reliable steady-state sensory cortical responses in all participants (n = 30) in both experiments. The visual cortical responses were modulated by the auditory detection task, but not by threat anticipation: Visual responses were smaller during auditory streams with a transient target as compared to uninterrupted auditory streams. Conversely, visual stimulus condition had no significant effects on auditory sensory cortical responses in either experiment. These results indicate that there is neither a competition nor facilitation effect of visual content on concurrent auditory sensory cortical processing. They further indicate that competition effects of auditory stream content on sustained visuocortical responses are limited to auditory target processing.

Attention differentially modulates the amplitude of resonance frequencies in the visual cortex

Rhythmic visual stimuli (flicker) elicit rhythmic brain responses at the frequency of the stimulus, and attention generally enhances these oscillatory brain responses (steady state visual evoked potentials, SSVEPs). Although SSVEP responses have been tested for flicker frequencies up to 100 Hz [Herrmann, 2001], effects of attention on SSVEP amplitude have only been reported for lower frequencies (up to ~30 Hz), with no systematic comparison across a wide, finely sampled frequency range. Does attention modulate SSVEP amplitude at higher flicker frequencies (gamma band, 30-80 Hz), and is attentional modulation constant across frequencies? By isolating SSVEP responses from the broadband EEG signal using a multivariate spatiotemporal source separation method, we demonstrate that flicker in the alpha and gamma bands elicit strongest and maximally phase stable brain responses (resonance), on which the effect of attention is opposite: positive for gamma and negative for alpha. Finding subject-specific gamma resonance frequency and a positive attentional modulation of gamma-band SSVEPs points to the untapped potential of flicker as a non-invasive tool for studying the causal effects of interactions between visual gamma-band rhythmic stimuli and endogenous gamma oscillations on perception and attention.

Probing cortical excitability using rapid frequency tagging

Frequency tagging has been widely used to study the role of visual selective attention. Presenting a visual stimulus flickering at a specific frequency generates so-called steady-state visually evoked responses. However, frequency tagging is mostly done at lower frequencies (<30 Hz). This produces a visible flicker, potentially interfering with both perception and neuronal oscillations in the theta, alpha and beta band. To overcome these problems, we used a newly developed projector with a 1440 Hz refresh rate allowing for frequency tagging at higher frequencies. We asked participants to perform a cued spatial attention task in which imperative pictorial stimuli were presented at 63 Hz or 78 Hz while measuring whole-head magnetoencephalography (MEG). We found posterior sensors to show a strong response at the tagged frequency. Importantly, this response was enhanced by spatial attention. Furthermore, we reproduced the typical modulations of alpha band oscillations, i.e., decrease in the alpha power contralateral to the attentional cue. The decrease in alpha power and increase in frequency tagged signal with attention correlated over subjects. We hereby provide proof-of-principle for the use of high-frequency tagging to study sensory processing and neuronal excitability associated with attention.

IR-camera-based measurements of 2D/3D cognitive fatigue in 2D/3D display system using task-evoked pupillary response

This study was carried out to evaluate a method used to measure three-dimensional (3D) cognitive fatigue based on the pupillary response. This technique was designed to overcome measurement burdens by using non-contact methods. The pupillary response is related to cognitive function by a neural pathway and may be an indicator of 3D cognitive fatigue. Twenty-six undergraduate students (including 14 women) watched both 2D and 3D versions of a video for 70 min. The participants experienced visual fatigue after viewing the 3D content. Measures such as subjective rating, response time, event-related potential latency, heartbeat-evoked potential (HEP) alpha power, and task-evoked pupillary response (TEPR) latency were significantly different. Multitrait-multimethod matrix analysis indicated that HEP and TEPR latency measures had stronger reliability and higher correlations with 3D cognitive fatigue than other measures. TEPR latency may be useful for quantitatively determining 3D visual fatigue, as it can be easily used to evaluate 3D visual fatigue using a non-contact method without measuring burden.

Effect of acceleration of auditory inputs on the primary somatosensory cortex in humans

Cross-modal interaction occurs during the early stages of processing in the sensory cortex; however, its effect on neuronal activity speed remains unclear. We used magnetoencephalography to investigate whether auditory stimulation influences the initial cortical activity in the primary somatosensory cortex. A 25-ms pure tone was randomly presented to the left or right side of healthy volunteers at 1000 ms when electrical pulses were applied to the left or right median nerve at 20 Hz for 1500 ms because we did not observe any cross-modal effect elicited by a single pulse. The latency of N20 m originating from Brodmann's area 3b was measured for each pulse. The auditory stimulation significantly shortened the N20 m latency at 1050 and 1100 ms. This reduction in N20 m latency was identical for the ipsilateral and contralateral sounds for both latency points. Therefore, somatosensory-auditory interaction, such as input to the area 3b from the thalamus, occurred during the early stages of synaptic transmission. Auditory information that converged on the somatosensory system was considered to have arisen from the early stages of the feedforward pathway. Acceleration of information processing through the cross-modal interaction seemed to be partly due to faster processing in the sensory cortex.

Having More Choices Changes How Human Observers Weight Stable Sensory Evidence

Decision-making becomes slower when more choices are available. Existing models attribute this slowing to poor sensory processing, to attenuated rates of sensory evidence accumulation, or to increases in the amount of evidence required before committing to a decision (a higher decision threshold). However, studies have not isolated the effects of having more choices on sensory and decision-related processes from changes in task difficulty and divided attention. Here, we controlled task difficulty while independently manipulating the distribution of attention and the number of choices available to male and female human observers. We used EEG to measure steady-state visually evoked potentials (SSVEPs) and a frontal late positive deflection (LPD), EEG markers of sensory and postsensory decision-related processes, respectively. We found that dividing attention decreased SSVEP and LPD amplitudes, consistent with dampened sensory responses and slower rates of evidence accumulation, respectively. In contrast, having more choices did not alter SSVEP amplitude and led to a larger LPD. These results suggest that having more options largely spares early sensory processing and slows down decision-making via a selective increase in decision thresholds.SIGNIFICANCE STATEMENT When more choices are available, decision-making becomes slower. We tested whether this phenomenon is due to poor sensory processing, to reduced rates of evidence accumulation, or to increases in the amount of evidence required before committing to a decision (a higher decision threshold). We measured choice modulations of sensory and decision-related neural responses using EEG. We also minimized potential confounds from changes in the distribution of attention and task difficulty, which often covary with having more choices. Dividing attention reduced the activity levels of both sensory and decision-related responses. However, having more choices did not change sensory processing and led to larger decision-related responses. These results suggest that having more choices spares sensory processing and selectively increases decision thresholds.

Individual Alpha Peak Frequency Predicts 10 Hz Flicker Effects on Selective Attention

Rhythmic visual stimulation ("flicker") is primarily used to "tag" processing of low-level visual and high-level cognitive phenomena. However, preliminary evidence suggests that flicker may also entrain endogenous brain oscillations, thereby modulating cognitive processes supported by those brain rhythms. Here we tested the interaction between 10 Hz flicker and endogenous alpha-band (∼10 Hz) oscillations during a selective visuospatial attention task. We recorded EEG from human participants (both genders) while they performed a modified Eriksen flanker task in which distractors and targets flickered within (10 Hz) or outside (7.5 or 15 Hz) the alpha band. By using a combination of EEG source separation, time-frequency, and single-trial linear mixed-effects modeling, we demonstrate that 10 Hz flicker interfered with stimulus processing more on incongruent than congruent trials (high vs low selective attention demands). Crucially, the effect of 10 Hz flicker on task performance was predicted by the distance between 10 Hz and individual alpha peak frequency (estimated during the task). Finally, the flicker effect on task performance was more strongly predicted by EEG flicker responses during stimulus processing than during preparation for the upcoming stimulus, suggesting that 10 Hz flicker interfered more with reactive than proactive selective attention. These findings are consistent with our hypothesis that visual flicker entrained endogenous alpha-band networks, which in turn impaired task performance. Our findings also provide novel evidence for frequency-dependent exogenous modulation of cognition that is determined by the correspondence between the exogenous flicker frequency and the endogenous brain rhythms.SIGNIFICANCE STATEMENT Here we provide novel evidence that the interaction between exogenous rhythmic visual stimulation and endogenous brain rhythms can have frequency-specific behavioral effects. We show that alpha-band (10 Hz) flicker impairs stimulus processing in a selective attention task when the stimulus flicker rate matches individual alpha peak frequency. The effect of sensory flicker on task performance was stronger when selective attention demands were high, and was stronger during stimulus processing and response selection compared with the prestimulus anticipatory period. These findings provide novel evidence that frequency-specific sensory flicker affects online attentional processing, and also demonstrate that the correspondence between exogenous and endogenous rhythms is an overlooked prerequisite when testing for frequency-specific cognitive effects of flicker.

Effects of mental workload on involuntary attention: A somatosensory ERP study

Previous psychophysiological assessments of mental workload have relied on the addition of visual or auditory stimuli. This study investigated the tactile ERP and EEG spectral power correlates of mental workload by relating limited-capacity involuntary attention allocation to changes in late positive potential (LPP) amplitude, alpha, and theta powers. We examined whether mental workload (high-level cognitive control) can be evaluated using somatosensory stimuli. Sixteen participants all performed three tasks of varying difficulty. Two dual n-back tasks (n = 1 and 2) were used to investigate the degree to which mental workload affected the LPP amplitudes and EEG spectral powers evoked by ignoring salient tactile stimuli. In control trials, tactile vibrations were applied at random without dual n-back tasks. Subjective mental workload of each task was rated using the NASA Task Load Index. LPP amplitudes at Pz were significantly smaller in the dual-2-back trials compared to control and dual-1-back trials. Significantly increased theta power at Fz and reduced alpha power at Pz were found in the dual-2-back condition compared to control and dual-1-back condition. There was no significant difference between control and dual-1-back trials. The same pattern was found for subjective ratings of cognitive workload. These results indicate that the dual-2-back task imposed a significantly greater mental workload, causing impaired cognitive-control functions. Our findings support the notion that selective attention mechanisms necessary for effectively allocating and modulating attentional resources are temporarily impaired during the mentally overloaded state.

Transcranial direct-current stimulation modulates offline visual oscillatory activity: A magnetoencephalography study

Transcranial direct-current stimulation (tDCS) is a noninvasive neuromodulatory method that involves delivering low amplitude, direct current to specific regions of the brain. While a wealth of literature shows changes in behavior and cognition following tDCS administration, the underlying neuronal mechanisms remain largely unknown. Neuroimaging studies have generally used fMRI and shown only limited consensus to date, while the few electrophysiological studies have reported mostly null or counterintuitive findings. The goal of the current investigation was to quantify tDCS-induced alterations in the oscillatory dynamics of visual processing. To this end, we performed either active or sham tDCS using an occipital-frontal electrode configuration, and then recorded magnetoencephalography (MEG) offline during a visual entrainment task. Significant oscillatory responses were imaged in the time-frequency domain using beamforming, and the effects of tDCS on absolute and relative power were assessed. The results indicated significantly increased basal alpha levels in the occipital cortex following anodal tDCS, as well as reduced occipital synchronization at the second harmonic of the stimulus-flicker frequency relative to sham stimulation. In addition, we found reduced power in brain regions near the cathode (e.g., right inferior frontal gyrus [IFG]) following active tDCS, which was absent in the sham group. Taken together, these results suggest that anodal tDCS of the occipital cortices differentially modulates spontaneous and induced activity, and may interfere with the entrainment of neuronal populations by a visual-flicker stimulus. These findings also demonstrate the importance of electrode configuration on whole-brain dynamics, and highlight the deceptively complicated nature of tDCS in the context of neurophysiology.

Driving steady-state visual evoked potentials at arbitrary frequencies using temporal interpolation of stimulus presentation

Background

Steady-state visual evoked potentials have been utilized widely in basic and applied research in recent years. These oscillatory responses of the visual cortex are elicited by flickering stimuli. They have the same fundamental frequency as the driving stimulus and are highly sensitive to manipulations of attention and stimulus properties. While standard computer monitors offer great flexibility in the choice of visual stimuli for driving SSVEPs, the frequencies that can be elicited are limited to integer divisors of the monitor’s refresh rate.

Results

To avoid this technical constraint, we devised an interpolation technique for stimulus presentation, with which SSVEPs can be elicited at arbitrary frequencies. We tested this technique with monitor refresh rates of 85 and 120 Hz. At a refresh rate of 85 Hz, interpolated presentation produced artifacts in the recorded spectrum in the form of additional peaks not located at the stimulated frequency or its harmonics. However, at a refresh rate of 120 Hz, these artifacts did not occur and the spectrum elicited by an interpolated flicker became indistinguishable from the spectrum obtained by non-interpolated presentation of the same frequency.

Conclusions

Our interpolation technique eliminates frequency limitations of the common non-interpolated presentation technique and has many possible applications for future research.

Reliance on visual attention during visuomotor adaptation: an SSVEP study

Visuomotor adaptation involves the learning of a new mapping between a spatial goal and well-learned movements. In order to learn a new visuomotor transformation, visual attention is needed to monitor movements and their visual consequences. Once a transformation is learnt, it can be executed automatically without attentional control. Using steady-state visual evoked potentials (SSVEPs) measured from EEG activity, we examined how visual attention changes during the early phase of visuomotor adaptation. SSVEPs were elicited by a green disc flickering at 15 Hz which was either the movement target or the cursor that participants controlled. Participants performed an adapted continuous visuomotor adaptation task with either 60° or 120° screen cursor rotation, and changes in 15-Hz SSVEP power were examined. Participants' performance improved over time in all conditions, with the rate of learning significantly influenced by the degree of rotation. SSVEPs at 15 Hz showed a significant change over time with adaptation for 60° rotations, but not for 120° rotations, such that SSVEPs elicited by the stimuli were significantly lower for 60° compared with 120° rotation conditions over the last adaptation blocks. This suggests that visual attention to the movement target and feedback reduces over time as performance improves during visuomotor adaptation for easier rotations, but must be maintained throughout the task for more difficult 120° rotations that might require more strategic control.

The steady-state visual evoked potential in vision research: A review

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.

Exploring how musical rhythm entrains brain activity with electroencephalogram frequency-tagging

The ability to perceive a regular beat in music and synchronize to this beat is a widespread human skill. Fundamental to musical behaviour, beat and meter refer to the perception of periodicities while listening to musical rhythms and often involve spontaneous entrainment to move on these periodicities. Here, we present a novel experimental approach inspired by the frequency-tagging approach to understand the perception and production of rhythmic inputs. This approach is illustrated here by recording the human electroencephalogram responses at beat and meter frequencies elicited in various contexts: mental imagery of meter, spontaneous induction of a beat from rhythmic patterns, multisensory integration and sensorimotor synchronization. Collectively, our observations support the view that entrainment and resonance phenomena subtend the processing of musical rhythms in the human brain. More generally, they highlight the potential of this approach to help us understand the link between the phenomenology of musical beat and meter and the bias towards periodicities arising under certain circumstances in the nervous system. Entrainment to music provides a highly valuable framework to explore general entrainment mechanisms as embodied in the human brain.

Effect of mental fatigue caused by mobile 3D viewing on selective attention: an ERP study

This study investigated behavioral responses to and auditory event-related potential (ERP) correlates of mental fatigue caused by mobile three-dimensional (3D) viewing. Twenty-six participants (14 women) performed a selective attention task in which they were asked to respond to the sounds presented at the attended side while ignoring sounds at the ignored side before and after mobile 3D viewing. Considering different individual susceptibilities to 3D, participants' subjective fatigue data were used to categorize them into two groups: fatigued and unfatigued. The amplitudes of d-ERP components were defined as differences in amplitudes between time-locked brain oscillations of the attended and ignored sounds, and these values were used to calculate the degree to which spatial selective attention was impaired by 3D mental fatigue. The fatigued group showed significantly longer response times after mobile 3D viewing compared to before the viewing. However, response accuracy did not significantly change between the two conditions, implying that the participants used a behavioral strategy to cope with their performance accuracy decrement by increasing their response times. No significant differences were observed for the unfatigued group. Analysis of covariance revealed group differences with significant and trends toward significant decreases in the d-P200 and d-late positive potential (LPP) amplitudes at the occipital electrodes of the fatigued and unfatigued groups. Our findings indicate that mentally fatigued participants did not effectively block out distractors in their information processing mechanism, providing support for the hypothesis that 3D mental fatigue impairs spatial selective attention and is characterized by changes in d-P200 and d-LPP amplitudes.

Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces

Background

The fatigue that users suffer when using steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can cause a number of serious problems such as signal quality degradation and system performance deterioration, users’ discomfort and even risk of photosensitive epileptic seizures, posing heavy restrictions on the applications of SSVEP-based BCIs. Towards alleviating the fatigue, a fundamental step is to measure and evaluate it but most existing works adopt self-reported questionnaire methods which are subjective, offline and memory dependent. This paper proposes an objective and real-time approach based on electroencephalography (EEG) spectral analysis to evaluate the fatigue in SSVEP-based BCIs.

Methods

How the EEG indices (amplitudes in δ, θ, α and β frequency bands), the selected ratio indices (θ/α and (θ + α)/β), and SSVEP properties (amplitude and signal-to-noise ratio (SNR)) changes with the increasing fatigue level are investigated through two elaborate SSVEP-based BCI experiments, one validates mainly the effectiveness and another considers more practical situations. Meanwhile, a self-reported fatigue questionnaire is used to provide a subjective reference. ANOVA is employed to test the significance of the difference between the alert state and the fatigue state for each index.

Results

Consistent results are obtained in two experiments: the significant increases in α and (θ + α)/β, as well as the decrease in θ/α are found associated with the increasing fatigue level, indicating that EEG spectral analysis can provide robust objective evaluation of the fatigue in SSVEP-based BCIs. Moreover, the results show that the amplitude and SNR of the elicited SSVEP are significantly affected by users’ fatigue.

Conclusions

The experiment results demonstrate the feasibility and effectiveness of the proposed method as an objective and real-time evaluation of the fatigue in SSVEP-based BCIs. This method would be helpful in understanding the fatigue problem and optimizing the system design to alleviate the fatigue in SSVEP-based BCIs.

Does visual fatigue from 3D displays affect autonomic regulation and heart rhythm?

Most investigations into the negative effects of viewing stereoscopic 3D content on human health have addressed 3D visual fatigue and visually induced motion sickness (VIMS). Very few, however, have looked into changes in autonomic balance and heart rhythm, which are homeostatic factors that ought to be taken into consideration when assessing the overall impact of 3D video viewing on human health. In this study, 30 participants were randomly assigned to two groups: one group watching a 2D video, (2D-group) and the other watching a 3D video (3D-group). The subjects in the 3D-group showed significantly increased heart rates (HR), indicating arousal, and an increased VLF/HF (Very Low Frequency/High Frequency) ratio (a measure of autonomic balance), compared to those in the 2D-group, indicating that autonomic balance was not stable in the 3D-group. Additionally, a more disordered heart rhythm pattern and increasing heart rate (as determined by the R-peak to R-peak (RR) interval) was observed among subjects in the 3D-group compared to subjects in the 2D-group, further indicating that 3D viewing induces lasting activation of the sympathetic nervous system and interrupts autonomic balance.

The spread of attention across features of a surface

Contrasting theories of visual attention have emphasized selection by spatial location, individual features, and whole objects. We used functional magnetic resonance imaging to ask whether and how attention to one feature of an object spreads to other features of the same object. Subjects viewed two spatially superimposed surfaces of random dots that were segregated by distinct color-motion conjunctions. The color and direction of motion of each surface changed smoothly and in a cyclical fashion. Subjects were required to track one feature (e.g., color) of one of the two surfaces and detect brief moments when the attended feature diverged from its smooth trajectory. To tease apart the effect of attention to individual features on the hemodynamic response, we used a frequency-tagging scheme. In this scheme, the stimulus features (color and direction of motion) are modulated periodically at distinct frequencies so that the contribution of each feature to the hemodynamics can be inferred from the harmonic response at the corresponding frequency. We found that attention to one feature (e.g., color) of one surface increased the response modulation not only to the attended feature but also to the other feature (e.g., motion) of the same surface. This attentional modulation was evident in multiple visual areas and was present as early as V1. The spread of attention to the behaviorally irrelevant features of a surface suggests that attention may automatically select all features of a single object. Thus object-based attention may be supported by an enhancement of feature-specific sensory signals in the visual cortex.

Sustained multifocal attentional enhancement of stimulus processing in early visual areas predicts tracking performance

Keeping track of multiple moving objects is an essential ability of visual perception. However, the mechanisms underlying this ability are not well understood. We instructed human observers to track five or seven independent randomly moving target objects amid identical nontargets and recorded steady-state visual evoked potentials (SSVEPs) elicited by these stimuli. Visual processing of moving targets, as assessed by SSVEP amplitudes, was continuously facilitated relative to the processing of identical but irrelevant nontargets. The cortical sources of this enhancement were located to areas including early visual cortex V1-V3 and motion-sensitive area MT, suggesting that the sustained multifocal attentional enhancement during multiple object tracking already operates at hierarchically early stages of visual processing. Consistent with this interpretation, the magnitude of attentional facilitation during tracking in a single trial predicted the speed of target identification at the end of the trial. Together, these findings demonstrate that attention can flexibly and dynamically facilitate the processing of multiple independent object locations in early visual areas and thereby allow for tracking of these objects.

Keeping focused: sustained spatial selective visual attention is maintained in healthy old age

A better understanding of age-related change in the attentional modulation of perceptual processing may help elucidate cognitive change. For example, increased cognitive interference due to inappropriate processing of irrelevant information has been suggested to contribute to cognitive decline. However, it is not yet clear whether interference effects observed at later stages, such as executive function or response selection, are caused by leaky attentional selection at early, sensory stages of processing. Here, we investigated attentional control of sensory selection by comparing younger and older adults' ability to sustain spatial selective attention to one of two centrally presented, overlapping rapid serial visual presentation (RSVP) letter sequences, one large and one small. These stimuli elicited separable steady-state visual evoked potentials (SSVEP), which provide an index of early visual processing for each stimulus separately and are known to be modulated by selective attention. The condition of most interest required participants to attend to the larger letters while ignoring the smaller letters, as these foveally presented irrelevant stimuli were expected to present the strongest interference. Although the rapid presentation rates made the task demanding, detection ability did not differ between young and older adults. Accordingly, attentional modulation of SSVEP amplitudes was found in both age groups. Neither the magnitude nor the cortical sources of these SSVEP attention effects differed between age groups. Our results thus suggest that in the current task, the effect of voluntary spatial attention on sustained sensory processing in early visual areas is maintained in healthy old age.

Independent effects of attentional gain control and competitive interactions on visual stimulus processing

Attention filters behaviorally relevant stimuli from the constant stream of sensory information comprising our environment. Research into underlying neural mechanisms in humans suggests that visual attention biases mutual suppression between stimuli resulting from competition for limited processing resources. As a consequence, processing of an attended stimulus is facilitated. This account makes 2 assumptions: 1) An attended stimulus is released from mutual suppression with competing stimuli and 2) an attended stimulus experiences greater gain in the presence of competing stimuli than when it is presented alone. Here, we tested these assumptions by recording frequency-tagged potentials elicited in early visual cortex that index stimulus-specific processing. We contrasted the processing of a given stimulus when its location was attended or unattended and in the presence or the absence of a nearby competing stimulus. At variance with previous findings, competition similarly suppressed processing of attended and unattended stimuli. Moreover, the magnitude of attentional gain was comparable in the presence or the absence of competing stimuli. We conclude that visuospatial selective attention does not directly modulate mutual suppression between stimuli but instead acts as a signal gain, which biases processing toward attended stimuli independent of competition.