How microsaccades relate to lateralized ERP components of spatial attention: A co-registration study

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

Ultra-fine resolution of pre-saccadic attention in the fovea

Microsaccades, the tiny gaze relocations that occurr during fixation, have been linked to covert attention deployed degrees away from the center of gaze. However, the link between attention and microsaccades is deeper in that it also unfolds at the foveal scale. Here, we have examined the spatial grain of pre-microsaccadic attention across the 1° foveola. Through the use of high-precision eye-tracking and gaze-contingent display system that achieves arcminute precision in gaze localization, we have shown that the spotlight of attention at this scale can reach a strikingly high resolution, in the order of 0.17°. Further, when a microsaccade occurs, vision is modulated in a peculiar way across the foveola; whereas fine spatial vision is enhanced at the microsaccade goal location, it drops at the very center of gaze, where acuity is normally highest. These results reveal the finesse of the visuomotor system and of the interplay between eye movements and attention.

How far can the self be extended? Automatic attention capture is triggered not only by the self-face

The preferential processing of self-related information is thought to be driven by its high level of familiarity. However, some behavioral studies have shown that people may exhibit a preference for initially unfamiliar stimuli that have been associated with themselves arbitrarily. One of the key questions that needs to be addressed concerns the role of early attention in the prioritization of newly acquired information associated with the self. Another question is whether both highly familiar as well as new information referring to a subjectively significant person (i.e. close-other) benefits from preferential attentional processing. We aimed to tackle both questions by investigating the neural mechanisms involved in processing extremely familiar stimuli, like one's own face or the face of a close-other, as well as stimuli (abstract shapes) that were newly linked to each person. We used a dot-probe paradigm that allowed us to investigate the early stages of attentional prioritization. Our analysis of the N2pc component unveiled that attention was automatically captured by the self-face, a shape associated with oneself, and the face of the close person. However, a shape associated with the close-other did not elicit the same attentional response, as the N2pc was absent. Thus, both the self-face and information referring to the extended self (self-assigned shape, close-other's face) benefit from preferential early and automatic attentional processing.

Involuntary oculomotor inhibition markers of saliency and deviance in response to auditory sequences

Our eyes move constantly but are often inhibited momentarily in response to external stimuli (oculomotor inhibition [OMI]), depending on the stimulus saliency, anticipation, and attention. Previous studies have shown prolonged OMI for auditory oddballs; however, they required counting the oddballs, possibly reflecting voluntary attention. Here, we investigated whether the “passive” OMI response to auditory deviants can provide a quantitative measure of deviance strength (pitch difference) and studied its dependence on the inter-trial interval (ITI). Participants fixated centrally and passively listened to repeated short sequences of pure tones that contained a deviant tone either regularly or with 20% probability (oddballs). In an “active” control experiment, participants counted the deviant or the standard. As in previous studies, the results showed prolonged microsaccade inhibition and increased pupil dilation following the rare deviant tone. Earlier inhibition onset was found in proportion to the pitch deviance (the saliency effect), and a later release was found for oddballs, but only for ITI <2.5 seconds. The active control experiment showed similar results when counting the deviant but longer OMI for the standard when counting it. Taken together, these results suggest that OMI provides involuntary markers of saliency and deviance, which can be obtained without the participant's response.

Gaze-cued shifts of attention and microsaccades are sustained for whole bodies but are transient for body parts

Gaze direction is an evolutionarily important mechanism in daily social interactions. It reflects a person’s internal cognitive state, spatial locus of interest, and predicts future actions. Studies have used static head images presented foveally and simple synthetic tasks to find that gaze orients attention and facilitates target detection at the cued location in a sustained manner. Little is known about how people’s natural gaze behavior, including eyes, head, and body movements, jointly orient covert attention, microsaccades, and facilitate performance in more ecological dynamic scenes. Participants completed a target person detection task with videos of real scenes. The videos showed people looking toward (valid cue) or away from a target (invalid cue) location. We digitally manipulated the individuals in the videos directing gaze to create three conditions: whole-intact (head and body movements), floating heads (only head movements), and headless bodies (only body movements). We assessed their impact on participants’ behavioral performance and microsaccades during the task. We show that, in isolation, an individual’s head or body orienting toward the target-person direction led to facilitation in detection that is transient in time (200 ms). In contrast, only the whole-intact condition led to sustained facilitation (500 ms). Furthermore, observers executed microsaccades more frequently towards the cued direction for valid trials, but this bias was sustained in time only with the joint presence of head and body parts. Together, the results differ from previous findings with foveally presented static heads. In more real-world scenarios and tasks, sustained attention requires the presence of the whole-intact body of the individuals dynamically directing their gaze.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020)

Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.

Sustained spatial attention accounts for the direction bias of human microsaccades

Small ballistic eye movements, so called microsaccades, occur even while foveating an object. Previous studies using covert attention tasks have shown that shortly after a symbolic spatial cue, specifying a behaviorally relevant location, microsaccades tend to be directed toward the cued location. This suggests that microsaccades can serve as an index for the covert orientation of spatial attention. However, this hypothesis faces two major challenges: First, effects associated with visual spatial attention are hard to distinguish from those that associated with the contemplation of foveating a peripheral stimulus. Second, it is less clear whether endogenously sustained attention alone can bias microsaccade directions without a spatial cue on each trial. To address the first issue, we investigated the direction of microsaccades in human subjects while they attended to a behaviorally relevant location and prepared a response eye movement either toward or away from this location. We find that directions of microsaccades are biased toward the attended location rather than towards the saccade target. To tackle the second issue, we verbally indicated the location to attend before the start of each block of trials, to exclude potential visual cue-specific effects on microsaccades. Our results indicate that sustained spatial attention alone reliably produces the microsaccade direction effect. Overall, our findings demonstrate that sustained spatial attention alone, even in the absence of saccade planning or a spatial cue, is sufficient to explain the direction bias observed in microsaccades.

Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing

Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.

Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published reports contain residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were varied orthogonally: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify the correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data, but also distorted neurogenic activity during eye movement-free intervals. However, correction results could be strongly improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact from both viewing paradigms, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.

Predictions as a window into learning: Anticipatory fixation offsets carry more information about environmental statistics than reactive stimulus-responses

A core question underlying neurobiological and computational models of behavior is how individuals learn environmental statistics and use them to make predictions. Most investigations of this issue have relied on reactive paradigms, in which inferences about predictive processes are derived by modeling responses to stimuli that vary in likelihood. Here we deployed a novel anticipatory oculomotor metric to determine how input statistics impact anticipatory behavior that is decoupled from target-driven-response. We implemented transition constraints between target locations, so that the probability of a target being presented on the same side as the previous trial was 70% in one condition (pret70) and 30% in the other (pret30). Rather than focus on responses to targets, we studied subtle endogenous anticipatory fixation offsets (AFOs) measured while participants fixated the screen center, awaiting a target. These AFOs were small (<0.4° from center on average), but strongly tracked global-level statistics. Speaking to learning dynamics, trial-by-trial fluctuations in AFO were well-described by a learning model, which identified a lower learning rate in pret70 than pret30, corroborating prior suggestions that pret70 is subjectively treated as more regular. Most importantly, direct comparisons with saccade latencies revealed that AFOs: (a) reflected similar temporal integration windows, (b) carried more information about the statistical context than did saccade latencies, and (c) accounted for most of the information that saccade latencies also contained about inputs statistics. Our work demonstrates how strictly predictive processes reflect learning dynamics, and presents a new direction for studying learning and prediction.

Finger Temperature: A Psychophysiological Assessment of the Attentional State

Attention is a key cognitive phenomenon that is studied to understand cognitive disorders or even to estimate workloads to prevent accidents. Usually, it is studied using brain activity, even though it has many psychophysiological correlates. In the present study, we aim to evaluate if finger temperature, as a surrogate of peripheral vasoconstriction, can be used to obtain similar and complementary information to electroencephalography (EEG) brain activity measurements. To conduct this, 34 participants were recruited and submitted to performing four tasks-one as a baseline, and three attentional tasks. These three attentional tasks measured sustained attention, resilience to distractors, and attentional resources. During the tasks, the room, forehead, tympanic, and finger temperatures were measured. Furthermore, we included a 32-channel EEG recording. Our results showed a strong monotonic association between the finger temperature and the Alpha and Beta EEG spectral bands. When predicting attentional performance, the finger temperature was complementary to the EEG spectral measurements, through the prediction of aspects of attentional performance that had not been assessed by spectral EEG activity, or through the improvement of the model's fit. We also found that during the baseline task (non-goal-oriented task), the spectral EEG activity has an inverted correlation, as compared to a goal-oriented task. Our current results suggest that the psychophysiological assessment of attention is complementary to classic EEG approach, while also having the advantage of easy implementation of analysis tools in environments of reducing control (workplaces, student classrooms).

Large lateralized EDAN-like brain potentials in a gaze-shift detection task

Attentional cueing tasks using gaze direction as spatial cues have sometimes yielded an early directing attention negativity (EDAN) component in the ERP, presumably reflecting the initial orienting toward the cued location. However, other studies have failed to identify an EDAN component for gaze cues, yielding an inconsistent picture. In the present study, we re-examined the EDAN to gaze cueing, using a continuous task where the specific direction of the gaze changes was task irrelevant. Face stimuli changed gaze direction several times during each trial between direct, left-, and right-averted positions. Participants counted the number of gaze shifts during the trial. Results showed an unusually large EDAN-like ERP asymmetry at posterior scalp sites that was of similar amplitude for large and small gaze shifts into the periphery. Shifts from an averted position toward a direct gaze elicited a qualitatively similar but smaller effect than shifts into the periphery. Together, these findings shed new light on gaze-elicited spatial attention as they indicate a reflexive attention orienting, following the direction of gaze motion, even when the gaze direction itself is irrelevant for the task.

Parafoveal previews and lexical frequency in natural reading: Evidence from eye movements and fixation-related potentials

Participants' eye movements and electroencephalogram (EEG) signal were recorded as they read sentences displayed according to the gaze-contingent boundary paradigm. Two target words in each sentence were manipulated for lexical frequency (high vs. low frequency) and parafoveal preview of each target word (identical vs. string of random letters vs. string of Xs). Eye movement data revealed visual parafoveal-on-foveal (PoF) effects, as well as foveal visual and orthographic preview effects and word frequency effects. Fixation-related potentials (FRPs) showed visual and orthographic PoF effects as well as foveal visual and orthographic preview effects. Our results replicated the early preview positivity effect (Dimigen, Kliegl, & Sommer, 2012) in the X-string preview condition, and revealed different neural correlates associated with a preview comprised of a string of random letters relative to a string of Xs. The former effects seem likely to reflect difficulty associated with the integration of parafoveal and foveal information, as well as feature overlap, while the latter reflect inhibition, and potentially disruption, to processing underlying reading. Interestingly, and consistent with Kretzschmar, Schlesewsky, and Staub (2015), no frequency effect was reflected in the FRP measures. The findings provide insight into the neural correlates of parafoveal processing and written word recognition in reading and demonstrate the value of utilizing ecologically valid paradigms to study well established phenomena that occur as text is read naturally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Unconscious Detection of One's Own Image

A key mechanism behind preferential processing of self-related information might be an early and automatic capture of attention. Therefore, the present study tested a hypothesis that one's own face will attract bottom-up attention even without conscious identification. To test this, we used a dot-probe paradigm with electrophysiological recordings, in which participants ( N = 18) viewed masked and unmasked pairs of faces (other, self) presented laterally. Analysis of the sensitivity measure d ' indicated that faces were not consciously identified in the masked condition. A clear N2 posterior-contralateral (N2pc) component (a neural marker of attention shifts) was found in both the masked and unmasked conditions, revealing that one's own face automatically captures attention when processed unconsciously. Therefore, our study (a) demonstrates that self-related information is boosted at an early (preconscious) stage of processing, (b) identifies further features (beyond simple physical ones) that cause automatic attention capture, and (c) provides further evidence for the dissociative nature of attention and consciousness.

Auditory spatial attention modulates the unmasking effect of perceptual separation in a "cocktail party" environment

The perceptual separation between a signal speech and a competing speech (masker), induced by the precedence effect, plays an important role in releasing the signal speech from the masker, especially in a reverberant environment. The perceptual-separation-induced unmasking effect has been suggested to involve multiple cognitive processes, such as selective attention. However, whether listeners' spatial attention modulate the perceptual-separation-induced unmasking effect is not clear. The present study investigated how perceptual separation and auditory spatial attention interact with each other to facilitate speech perception under a simulated noisy and reverberant environment by analyzing the cortical auditory evoked potentials to the signal speech. The results showed that the N1 wave was significantly enhanced by perceptual separation between the signal and masker regardless of whether the participants' spatial attention was directed to the signal or not. However, the P2 wave was significantly enhanced by perceptual separation only when the participants attended to the signal speech. The results indicate that the perceptual-separation-induced facilitation of P2 needs more attentional resource than that of N1. The results also showed that the signal speech caused an enhanced N1 in the contralateral hemisphere regardless of whether participants' attention was directed to the signal or not. In contrast, the signal speech caused an enhanced P2 in the contralateral hemisphere only when the participant attended to the signal. The results indicate that the hemispheric distribution of N1 is mainly affected by the perceptual features of the acoustic stimuli, while that of P2 is affected by the listeners' attentional status.

Electrophysiological evidence for changes in attentional orienting and selection in functional somatic symptoms

OBJECTIVE

We investigated changes in attention mechanisms in people who report a high number of somatic symptoms which cannot be associated with a physical cause.

METHOD

Based on scores on the Somatoform Disorder Questionnaire (SDQ-20; Nijenhuis et al., 1996) we compared two non-clinical groups, one with high symptoms on the SDQ-20 and a control group with low or no symptoms. We recorded EEG whilst participants performed an exogenous tactile attention task where they had to discriminate between tactile targets following a tactile cue to the same or opposite hand.

RESULTS

The neural marker of attentional orienting to the body, the Late Somatosensory Negativity (LSN), was diminished in the high symptoms group and attentional modulation of touch processing was prolonged at mid and enhanced at later latency stages in this group.

CONCLUSION

These results confirm that attentional processes are altered in people with somatic symptoms, even in a non-clinical group. Furthermore, the observed pattern fits explanations of changes in prior beliefs or expectations leading to diminished amplitudes of the marker of attentional orienting to the body (i.e. the LSN) and enhanced attentional gain of touch processing.

SIGNIFICANCE

This study shows that high somatic symptoms are associated with neurocognitive attention changes.

Self-face Captures, Holds, and Biases Attention

The implicit self-recognition process may take place already in the pre-attentive stages of perception. After a silent stimulus has captured attention, it is passed on to the attentive stage where it can affect decision making and responding. Numerous studies show that the presence of self-referential information affects almost every cognitive level. These effects may share a common and fundamental basis in an attentional mechanism, conceptualized as attentional bias: the exaggerated deployment of attentional resources to a salient stimulus. A gold standard in attentional bias research is the dot-probe paradigm. In this task, a prominent stimulus (cue) and a neutral stimulus are presented in different spatial locations, followed by the presentation of a target. In the current study we aimed at investigating whether the self-face captures, holds and biases attention when presented as a task-irrelevant stimulus. In two dot-probe experiments coupled with the event-related potential (ERP) technique we analyzed the following relevant ERPs components: N2pc and SPCN which reflect attentional shifts and the maintenance of attention, respectively. An inter-stimulus interval separating face-cues and probes (800 ms) was introduced only in the first experiment. In line with our predictions, in Experiment 1 the self-face elicited the N2pc and the SPCN component. In Experiment 2 in addition to N2pc, an attentional bias was observed. Our results indicate that unintentional self-face processing disables the top-down control setting to filter out distractors, thus leading to the engagement of attentional resources and visual short-term memory.