Saccade onset and offset lambda waves: relation to pattern movement visually evoked potentials

Fixation-related potentials during a virtual navigation task: The influence of image statistics on early cortical processing

Historically, electrophysiological correlates of scene processing have been studied with experiments using static stimuli presented for discrete timescales where participants maintain a fixed eye position. Gaps remain in generalizing these findings to real-world conditions where eye movements are made to select new visual information and where the environment remains stable but changes with our position and orientation in space, driving dynamic visual stimulation. Co-recording of eye movements and electroencephalography (EEG) is an approach to leverage fixations as time-locking events in the EEG recording under free-viewing conditions to create fixation-related potentials (FRPs), providing a neural snapshot in which to study visual processing under naturalistic conditions. The current experiment aimed to explore the influence of low-level image statistics-specifically, luminance and a metric of spatial frequency (slope of the amplitude spectrum)-on the early visual components evoked from fixation onsets in a free-viewing visual search and navigation task using a virtual environment. This research combines FRPs with an optimized approach to remove ocular artifacts and deconvolution modeling to correct for overlapping neural activity inherent in any free-viewing paradigm. The results suggest that early visual components-namely, the lambda response and N1-of the FRPs are sensitive to luminance and spatial frequency around fixation, separate from modulation due to underlying differences in eye-movement characteristics. Together, our results demonstrate the utility of studying the influence of image statistics on FRPs using a deconvolution modeling approach to control for overlapping neural activity and oculomotor covariates.

Take it sitting down: the effect of body posture on cortical potentials during free viewing-A mobile EEG recording study

Brain imaging performed in natural settings is known as mobile brain and body imaging (MoBI). One of the features which distinguishes MoBI and laboratory-based experiments is the body posture. Previous studies pointed to mechanical, autonomic, cortical and cognitive differences between upright stance and sitting or reclining. The purpose of this study was to analyse effects of posture on eye-movement related potentials (EMRP) recorded during free viewing of human faces. A 64-channel wireless EEG was recorded from 14 participants in either standing or reclining postures while they freely viewed pictures of emotional faces displaying fear, anger, sadness, and a neutral emotional state. Eye tracking data was used to insert triggers corresponding to the instant at which the gaze first landed on a face. Spatial filtering of the EEG data was performed using a group independent component analysis (ICA). Grand average EMRPs displayed the post-saccadic lambda component and the face-sensitive N170/vertex positive potential (VPP) complex. The lambda component but not the N170 component was stronger during reclining than upright posture. Emotional expression of faces showed no effects on EMRP components or subjective ratings. Results suggest that posture primarily affects early components of EMRPs recorded using wireless EEG recordings during free viewing of faces. Thus, findings from evoked potential data obtained in seated individuals, e.g., in laboratory experiments, should be interpreted with caution in MoBI experiments with posture affecting primarily the early latency component.

Saccade size predicts onset time of object processing during visual search of an open world virtual environment

OBJECTIVE

To date the vast majority of research in the visual neurosciences have been forced to adopt a highly constrained perspective of the vision system in which stimuli are processed in an open-loop reactive fashion (i.e., abrupt stimulus presentation followed by an evoked neural response). While such constraints enable high construct validity for neuroscientific investigation, the primary outcomes have been a reductionistic approach to isolate the component processes of visual perception. In electrophysiology, of the many neural processes studied under this rubric, the most well-known is, arguably, the P300 evoked response. There is, however, relatively little known about the real-world corollary of this component in free-viewing paradigms where visual stimuli are connected to neural function in a closed-loop. While growing evidence suggests that neural activity analogous to the P300 does occur in such paradigms, it is an open question when this response occurs and what behavioral or environmental factors could be used to isolate this component.

APPROACH

The current work uses convolutional networks to decode neural signals during a free-viewing visual search task in a closed-loop paradigm within an open-world virtual environment. From the decoded activity we construct fixation-locked response profiles that enable estimations of the variable latency of any P300 analogue around the moment of fixation. We then use these estimates to investigate which factors best reduce variable latency and, thus, predict the onset time of the response. We consider measurable, search-related factors encompassing top-down (i.e., goal driven) and bottom-up (i.e., stimulus driven) processes, such as fixation duration and salience. We also consider saccade size as an intermediate factor reflecting the integration of these two systems.

MAIN RESULTS

The results show that of these factors only saccade size reliably determines the onset time of P300 analogous activity for this task. Specifically, we find that for large saccades the variability in response onset is small enough to enable analysis using traditional ensemble averaging methods.

SIGNIFICANCE

The results show that P300 analogous activity does occur during closed-loop, free-viewing visual search while highlighting distinct differences between the open-loop version of this response and its real-world analogue. The results also further establish saccades, and saccade size, as a key factor in real-world visual processing.

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

Refixation behavior in naturalistic viewing: Methods, mechanisms, and neural correlates

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations.

Normal variants and artifacts: Importance in EEG interpretation

Overinterpretation of EEG is an important contributor to the misdiagnosis of epilepsy. For the EEG to have a high diagnostic value and high specificity, it is critical to recognize waveforms that can be mistaken for abnormal patterns. This article describes artifacts, normal rhythms, and normal patterns that are prone to being misinterpreted as abnormal. Artifacts are potentials generated outside the brain. They are divided into physiologic and extraphysiologic. Physiologic artifacts arise from the body and include EMG, eyes, various movements, EKG, pulse, and sweat. Some physiologic artifacts can be useful for interpretation such as EMG and eye movements. Extraphysiologic artifacts arise from outside the body, and in turn can be divided into the environments (electrodes, equipment, and cellphones) and devices within the body (pacemakers and neurostimulators). Normal rhythms can be divided into awake patterns (alpha rhythm and its variants, mu rhythm, lambda waves, posterior slow waves of youth, HV-induced slowing, photic driving, and photomyogenic response) and sleep patterns (POSTS, vertex waves, spindles, K complexes, sleep-related hypersynchrony, and frontal arousal rhythm). Breach can affect both awake and sleep rhythms. Normal variants or variants of uncertain clinical significance include variants that may have been considered abnormal in the early days of EEG but are now considered normal. These include wicket spikes and wicket rhythms (the most common normal pattern overread as epileptiform), small sharp spikes (aka benign epileptiform transients of sleep), rhythmic midtemporal theta of drowsiness (aka psychomotor variant), Cigánek rhythm (aka midline theta), 6 Hz phantom spike-wave, 14 and 6 Hz positive spikes, subclinical rhythmic epileptiform discharges of adults (SREDA), slow-fused transients, occipital spikes of blindness, and temporal slowing of the elderly. Correctly identifying artifacts and normal patterns can help avoid overinterpretation and misdiagnosis. This is an educational review paper addressing a learning objective of the International League Against Epilepsy (ILAE) curriculum.

Gaze onsets during naturalistic infant-caregiver interaction associate with 'sender' but not 'receiver' neural responses, and do not lead to changes in inter-brain synchrony

Temporal coordination during infant-caregiver social interaction is thought to be crucial for supporting early language acquisition and cognitive development. Despite a growing prevalence of theories suggesting that increased inter-brain synchrony associates with many key aspects of social interactions such as mutual gaze, little is known about how this arises during development. Here, we investigated the role of mutual gaze onsets as a potential driver of inter-brain synchrony. We extracted dual EEG activity around naturally occurring gaze onsets during infant-caregiver social interactions in N = 55 dyads (mean age 12 months). We differentiated between two types of gaze onset, depending on each partners' role. 'Sender' gaze onsets were defined at a time when either the adult or the infant made a gaze shift towards their partner at a time when their partner was either already looking at them (mutual) or not looking at them (non-mutual). 'Receiver' gaze onsets were defined at a time when their partner made a gaze shift towards them at a time when either the adult or the infant was already looking at their partner (mutual) or not (non-mutual). Contrary to our hypothesis we found that, during a naturalistic interaction, both mutual and non-mutual gaze onsets were associated with changes in the sender, but not the receiver's brain activity and were not associated with increases in inter-brain synchrony above baseline. Further, we found that mutual, compared to non-mutual gaze onsets were not associated with increased inter brain synchrony. Overall, our results suggest that the effects of mutual gaze are strongest at the intra-brain level, in the 'sender' but not the 'receiver' of the mutual gaze.

Episodic memory formation in unrestricted viewing

The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior.

Revealing Whole-Brain Causality Networks During Guided Visual Searching

In our daily lives, we use eye movements to actively sample visual information from our environment ("active vision"). However, little is known about how the underlying mechanisms are affected by goal-directed behavior. In a study of 31 participants, magnetoencephalography was combined with eye-tracking technology to investigate how interregional interactions in the brain change when engaged in two distinct forms of active vision: freely viewing natural images or performing a guided visual search. Regions of interest with significant fixation-related evoked activity (FRA) were identified with spatiotemporal cluster permutation testing. Using generalized partial directed coherence, we show that, in response to fixation onset, a bilateral cluster consisting of four regions (posterior insula, transverse temporal gyri, superior temporal gyrus, and supramarginal gyrus) formed a highly connected network during free viewing. A comparable network also emerged in the right hemisphere during the search task, with the right supramarginal gyrus acting as a central node for information exchange. The results suggest that all four regions are vital to visual processing and guiding attention. Furthermore, the right supramarginal gyrus was the only region where activity during fixations on the search target was significantly negatively correlated with search response times. Based on our findings, we hypothesize that, following a fixation, the right supramarginal gyrus supplies the right supplementary eye field (SEF) with new information to update the priority map guiding the eye movements during the search task.

During natural viewing, neural processing of visual targets continues throughout saccades

Relatively little is known about visual processing during free-viewing visual search in realistic dynamic environments. Free-viewing is characterized by frequent saccades. During saccades, visual processing is thought to be suppressed, yet we know that the presaccadic visual content can modulate postsaccadic processing. To better understand these processes in a realistic setting, we study here saccades and neural responses elicited by the appearance of visual targets in a realistic virtual environment. While subjects were being driven through a 3D virtual town, they were asked to discriminate between targets that appear on the road. Using a system identification approach, we separated overlapping and correlated activity evoked by visual targets, saccades, and button presses. We found that the presence of a target enhances early occipital as well as late frontocentral saccade-related responses. The earlier potential, shortly after 125 ms post-saccade onset, was enhanced for targets that appeared in the peripheral vision as compared to the central vision, suggesting that fast peripheral processing initiated before saccade onset. The later potential, at 195 ms post-saccade onset, was strongly modulated by the visibility of the target. Together these results suggest that, during natural viewing, neural processing of the presaccadic visual stimulus continues throughout the saccade, apparently unencumbered by saccadic suppression.

I think I'm sleepy, therefore I am - Awareness of sleepiness while driving: A systematic review

Driver drowsiness contributes to 10-20% of motor vehicle crashes. To reduce crash risk, ideally drivers would be aware of the drowsy state and cease driving. The extent to which drivers can accurately identify sleepiness remains under much debate. We systematically examined whether individuals are aware of sleepiness while driving, and whether this accurately reflects driving impairment, using meta-analyses and narrative review. Within this scope, there is high variability in measures of subjective sleepiness, driving performance and physiologically-derived drowsiness, and statistical analyses. Thirty-four simulated/naturalistic driving studies were reviewed. To summarise, drivers were aware of sleepiness, and this was associated to physiological drowsiness and driving impairment, such that high levels of sleepiness significantly predicted crash events and lane deviations. Subjective sleepiness was more strongly correlated (i) with physiological drowsiness compared to driving outcomes; (ii) under simulated driving conditions compared to naturalistic drives; and (iii) when examined using the Karolinska sleepiness scale (KSS) compared to other measures. Gaps remain in relation to how age, sex, and varying degrees of sleep loss may influence this association. This review provides evidence that drivers are aware of drowsiness while driving, and stopping driving when feeling 'sleepy' may significantly reduce crash risk.

Eye-Tracking for Clinical Ophthalmology with Virtual Reality (VR): A Case Study of the HTC Vive Pro Eye's Usability

BACKGROUND

A case study is proposed to empirically test and discuss the eye-tracking status-quo hardware capabilities and limitations of an off-the-shelf virtual reality (VR) headset with embedded eye-tracking for at-home ready-to-go online usability in ophthalmology applications.

METHODS

The eye-tracking status-quo data quality of the HTC Vive Pro Eye is investigated with novel testing specific to objective online VR perimetry. Testing was done across a wide visual field of the head-mounted-display's (HMD) screen and in two different moving conditions. A new automatic and low-cost Raspberry Pi system is introduced for VR temporal precision testing for assessing the usability of the HTC Vive Pro Eye as an online assistance tool for visual loss.

RESULTS

The target position on the screen and head movement evidenced limitations of the eye-tracker capabilities as a perimetry assessment tool. Temporal precision testing showed the system's latency of 58.1 milliseconds (ms), evidencing its good potential usage as a ready-to-go online assistance tool for visual loss.

CONCLUSIONS

The test of the eye-tracking data quality provides novel analysis useful for testing upcoming VR headsets with embedded eye-tracking and opens discussion regarding expanding future introduction of these HMDs into patients' homes for low-vision clinical usability.

A passive BCI for monitoring the intentionality of the gaze-based moving object selection

OBJECTIVE

The use of an electroencephalogram (EEG) anticipation-related component, the expectancy wave (E-wave), in brain-machine interaction was proposed more than 50 years ago. This possibility was not explored for decades, but recently it was shown that voluntary attempts to select items using eye fixations, but not spontaneous eye fixations, are accompanied by the E-wave. Thus, the use of the E-wave detection was proposed for the enhancement of gaze interaction technology, which has a strong need for a mean to decide if a gaze behaviour is voluntary or not. Here, we attempted at estimating whether this approach can be used in the context of moving object selection through smooth pursuit eye movements.

APPROACH

18 participants selected, one by one, items which moved on a computer screen, by gazing at them. In separate runs, the participants performed tasks not related to voluntary selection but also provoking smooth pursuit. A low-cost consumer-grade eye tracker was used for item selection.

MAIN RESULTS

A component resembling the E-wave was found in the averaged EEG segments time-locked to voluntary selection events of every participant. Linear discriminant analysis with shrinkage regularization (sLDA) classified the intentional and spontaneous smooth pursuit eye movements, using single-trial 300 ms long EEG segments, significantly above chance in eight participants. When the classifier output was averaged over ten subsequent data segments, median group ROC AUC of 0.75 was achieved.

SIGNIFICANCE

The results suggest the possible usefulness of the E-wave detection in the gaze-based selection of moving items, e.g., in video games. This technique might be more effective when trial data can be averaged, thus it could be considered for use in passive interfaces, for example, in estimating the degree of the user's involvement during gaze-based interaction.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Fixation-related Brain Potentials during Semantic Integration of Object–Scene Information

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t − 1). Object–scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object–scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t − 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object–scene integration during natural viewing.

The effects of cognitive distraction on behavioural, oculomotor and electrophysiological metrics during a driving hazard perception task

Previous research has demonstrated that the distraction caused by holding a mobile telephone conversation is not limited to the period of the actual conversation (Haigney, 1995; Redelmeier & Tibshirani, 1997; Savage et al., 2013). In a prior study we identified potential eye movement and EEG markers of cognitive distraction during driving hazard perception. However the extent to which these markers are affected by the demands of the hazard perception task are unclear. Therefore in the current study we assessed the effects of secondary cognitive task demand on eye movement and EEG metrics separately for periods prior to, during and after the hazard was visible. We found that when no hazard was present (prior and post hazard windows), distraction resulted in changes to various elements of saccadic eye movements. However, when the target was present, distraction did not affect eye movements. We have previously found evidence that distraction resulted in an overall decrease in theta band output at occipital sites of the brain. This was interpreted as evidence that distraction results in a reduction in visual processing. The current study confirmed this by examining the effects of distraction on the lambda response component of subjects eye fixation related potentials (EFRPs). Furthermore, we demonstrated that although detections of hazards were not affected by distraction, both eye movement and EEG metrics prior to the onset of the hazard were sensitive to changes in cognitive workload. This suggests that changes to specific aspects of the saccadic eye movement system could act as unobtrusive markers of distraction even prior to a breakdown in driving performance.

Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing

Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

Transsacadic Information and Corollary Discharge in Local Field Potentials of Macaque V1

Approximately three times per second, human visual perception is interrupted by a saccadic eye movement. In addition to taking the eyes to a new location, several lines of evidence suggest that the saccades play multiple roles in visual perception. Indeed, it may be crucial that visual processing is informed about movements of the eyes in order to analyze visual input distinctly and efficiently on each fixation and preserve stable visual perception of the world across saccades. A variety of studies has demonstrated that activity in multiple brain areas is modulated by saccades. The hypothesis tested here is that these signals carry significant information that could be used in visual processing. To test this hypothesis, local field potentials (LFPs) were simultaneously recorded from multiple electrodes in macaque primary visual cortex (V1); support vector machines (SVMs) were used to classify the peri-saccadic LFPs. We find that LFPs in area V1 carry information that can be used to distinguish neural activity associated with fixations from saccades, precisely estimate the onset time of fixations, and reliably infer the directions of saccades. This information may be used by the brain in processes including visual stability, saccadic suppression, receptive field (RF) remapping, fixation amplification, and trans-saccadic visual perception.

Transient and sustained effects of stimulus properties on the generation of microsaccades

Saccades shift the gaze rapidly every few hundred milliseconds from one fixated location to the next, producing a flow of visual input into the visual system even in the absence of changes in the environment. During fixation, small saccades called microsaccades are produced 1-3 times per second, generating a flow of visual input. The characteristics of this visual flow are determined by the timings of the saccades and by the characteristics of the visual stimuli on which they are performed. Previous models of microsaccade generation have accounted for the effects of external stimulation on the production of microsaccades, but they have not considered the effects of the prolonged background stimulus on which microsaccades are performed. The effects of this stimulus on the process of microsaccade generation could be sustained, following its prolonged presentation, or transient, through the visual transients produced by the microsaccades themselves. In four experiments, we varied the properties of the constant displays and examined the resulting modulation of microsaccade properties: their sizes, their timings, and the correlations between properties of consecutive microsaccades. Findings show that displays of higher spatial frequency and contrast produce smaller microsaccades and longer minimal intervals between consecutive microsaccades; and smaller microsaccades are followed by smaller and delayed microsaccades. We explain these findings in light of previous models and suggest a conceptual model by which both sustained and transient effects of the stimulus have central roles in determining the generation of microsaccades.

Improving free-viewing fixation-related EEG potentials with continuous-time regression

BACKGROUND

In the analysis of combined ET-EEG data, there are several issues with estimating FRPs by averaging. Neural responses associated with fixations will likely overlap with one another in the EEG recording and neural responses change as a function of eye movement characteristics. Especially in tasks that do not constrain eye movements in any way, these issues can become confounds.

NEW METHOD

Here, we propose the use of regression based estimates as an alternative to averaging. Multiple regression can disentangle different influences on the EEG and correct for overlap. It thereby accounts for potential confounds in a way that averaging cannot. Specifically, we test the applicability of the rERP framework, as proposed by Smith and Kutas (2015b), (2017), or Sassenhagen (2018) to combined eye tracking and EEG data from a visual search and a scene memorization task.

RESULTS

Results show that the method successfully estimates eye movement related confounds in real experimental data, so that these potential confounds can be accounted for when estimating experimental effects.

COMPARISON WITH EXISTING METHODS

The rERP method successfully corrects for overlapping neural responses in instances where averaging does not. As a consequence, baselining can be applied without risking distortions. By estimating a known experimental effect, we show that rERPs provide an estimate with less variance and more accuracy than averaged FRPs. The method therefore provides a practically feasible and favorable alternative to averaging.

CONCLUSIONS

We conclude that regression based ERPs provide novel opportunities for estimating fixation related EEG in free-viewing experiments.

Tracking Economic Value of Products in Natural Settings: A Wireless EEG Study

Economic decision making refers to the process of individuals translating their preference into subjective value (SV). Little is known about the dynamics of the neural processes that underpin this form of value-based decision making and no studies have investigated these processes outside of controlled laboratory settings. The current study investigated the spatio-temporal dynamics that accompany economic valuation of products using mobile electroencephalography (EEG) and eye tracking techniques. Participants viewed and rated images of household products in a gallery setting while EEG and eye tracking data were collected wirelessly. A Becker-DeGroot-Marschak (BDM) auction task was subsequently used to quantify the individual's willingness to pay (WTP) for each product. WTP was used to classify products into low, low medium, high medium and high economic value conditions. Eye movement related potentials (EMRP) were examined, and independent component analysis (ICA) was used to separate sources of activity from grand averaged EEG data. Four independent components (ICs) of EMRPs were modulated by WTP (i.e., SV) in the latency range of 150-250 ms. Of the four value-sensitive ICs, one IC displayed enhanced amplitude for all value conditions excluding low value, and another IC presented enhanced amplitude for low value products only. The remaining two value-sensitive ICs resolved inter-mediate levels of SV. Our study quantified, for the first time, the neural processes involved in economic value based decisions in a natural setting. Results suggest that multiple spatio-temporal brain activation patterns mediate the attention and aversion of products which could reflect an early valuation system. The EMRP parietal P200 component could reflect an attention allocation mechanism that separates the lowest-value products (IC7) from products of all other value (IC4), suggesting that low-value items are categorized early on as being aversive. While none of the ICs showed linear amplitude changes that parallel SV's of products, results suggest that a combination of multiple components may sub-serve a fine-grained resolution of the SV of products.

Refixation control in free viewing: a specialized mechanism divulged by eye-movement-related brain activity

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eye-movement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. NEW & NOTEWORTHY A substantial proportion of eye fixations in human natural viewing behavior are revisits of recently visited locations, i.e., refixations. Our recently developed methods enabled us to study refixations in a free viewing visual search task, using combined eye movement and EEG recording. We identified in the EEG a distinctive refixation-related signal, signifying a control mechanism specific to refixations as opposed to ordinary eye fixations.

Brain Responses to Emotional Faces in Natural Settings: A Wireless Mobile EEG Recording Study

The detection of a human face in a visual field and correct reading of emotional expression of faces are important elements in everyday social interactions, decision making and emotional responses. Although brain correlates of face processing have been established in previous fMRI and electroencephalography (EEG)/MEG studies, little is known about how the brain representation of faces and emotional expressions of faces in freely moving humans. The present study aimed to detect brain electrical potentials that occur during the viewing of human faces in natural settings. 64-channel wireless EEG and eye-tracking data were recorded in 19 participants while they moved in a mock art gallery and stopped at times to evaluate pictures hung on the walls. Positive, negative and neutral valence pictures of objects and human faces were displayed. The time instants in which pictures first occurred in the visual field were identified in eye-tracking data and used to reconstruct the triggers in continuous EEG data after synchronizing the time axes of the EEG and eye-tracking device. EEG data showed a clear face-related event-related potential (ERP) in the latency interval ranging from 165 to 210 ms (N170); this component was not seen whilst participants were viewing non-living objects. The face ERP component was stronger during viewing disgusted compared to neutral faces. Source dipole analysis revealed an equivalent current dipole in the right fusiform gyrus (BA37) accounting for N170 potential. Our study demonstrates for the first time the possibility of recording brain responses to human faces and emotional expressions in natural settings. This finding opens new possibilities for clinical, developmental, social, forensic, or marketing research in which information about face processing is of importance.

Target probability modulates fixation-related potentials in visual search

This study investigated the influence of target probability on the neural response to target detection in free viewing visual search. Participants were asked to indicate the number of targets (one or two) among distractors in a visual search task while EEG and eye movements were co-registered. Target probability was manipulated by varying the set size of the displays between 10, 22, and 30 items. Fixation-related potentials time-locked to first target fixations revealed a pronounced P300 at the centro-parietal cortex with larger amplitudes for set sizes 22 and 30 than for set size 10. With increasing set size, more distractor fixations preceded the detection of the target, resulting in a decreased target probability and, consequently, a larger P300. For distractors, no increase of P300 amplitude with set size was observed. The findings suggest that set size specifically affects target but not distractor processing in overt serial visual search.

Presaccadic EEG activity predicts visual saliency in free-viewing contour integration

While viewing a scene, the eyes are attracted to salient stimuli. We set out to identify the brain signals controlling this process. In a contour integration task, in which participants searched for a collinear contour in a field of randomly oriented Gabor elements, a previously established model was applied to calculate a visual saliency value for each fixation location. We studied brain activity related to the modeled saliency values, using coregistered eye tracking and EEG. To disentangle EEG signals reflecting salience in free viewing from overlapping EEG responses to sequential eye movements, we adopted generalized additive mixed modeling (GAMM) to single epochs of saccade-related EEG. We found that, when saliency at the next fixation location was high, amplitude of the presaccadic EEG activity was low. Since presaccadic activity reflects covert attention to the saccade target, our results indicate that larger attentional effort is needed for selecting less salient saccade targets than more salient ones. This effect was prominent in contour-present conditions (half of the trials), but ambiguous in the contour-absent condition. Presaccadic EEG activity may thus be indicative of bottom-up factors in saccade guidance. The results underscore the utility of GAMM for EEG-eye movement coregistration research.

Large-Scale Traveling Waves in EEG Activity Following Eye Movement

In spontaneous, stimulus-evoked, and eye-movement evoked EEG, the oscillatory signal shows large scale, dynamically organized patterns of phase. We investigated eye-movement evoked patterns in free-viewing conditions. Participants viewed photographs of natural scenes in anticipation of a memory test. From 200 ms intervals following saccades, we estimated the EEG phase gradient over the entire scalp, and the wave activity, i.e. the goodness of fit of a wave model involving a phase gradient assumed to be smooth over the scalp. In frequencies centered at 6.5 Hz, large-scale phase organization occurred, peaking around 70 ms after fixation onset and taking the form of a traveling wave. According to the wave gradient, most of the times the wave spreads from the posterior-inferior to anterior-superior direction. In these directions, the gradients depended on the size and direction of the saccade. Wave propagation velocity decreased in the course of the fixation, particularly in the interval from 50 to 150 ms after fixation onset. This interval corresponds to the fixation-related lambda activity, which reflects early perceptual processes following fixation onset. We conclude that lambda activity has a prominent traveling wave component. This component consists of a short-term whole-head phase pattern of specific direction and velocity, which may reflect feedforward propagation of visual information at fixation.

Estimation of overlapped Eye Fixation Related Potentials: The General Linear Model, a more flexible framework than the ADJAR algorithm

The Eye Fixation Related Potential (EFRP) estimation is the average of EEG signals across epochs at ocular fixation onset. Its main limitation is the overlapping issue. Inter Fixation Intervals (IFI) - typically around 300 ms in the case of unrestricted eye movement- depend on participants’ oculomotor patterns, and can be shorter than the latency of the components of the evoked potential. If the duration of an epoch is longer than the IFI value, more than one fixation can occur, and some overlapping between adjacent neural responses ensues. The classical average does not take into account either the presence of several fixations during an epoch or overlapping. The Adjacent Response algorithm (ADJAR), which is popular for event-related potential estimation, was compared to the General Linear Model (GLM) on a real dataset from a conjoint EEG and eye-tracking experiment to address the overlapping issue. The results showed that the ADJAR algorithm was based on assumptions that were too restrictive for EFRP estimation. The General Linear Model appeared to be more robust and efficient. Different configurations of this model were compared to estimate the potential elicited at image onset, as well as EFRP at the beginning of exploration. These configurations took into account the overlap between the event-related potential at stimulus presentation and the following EFRP, and the distinction between the potential elicited by the first fixation onset and subsequent ones. The choice of the General Linear Model configuration was a tradeoff between assumptions about expected behavior and the quality of the EFRP estimation: the number of different potentials estimated by a given model must be controlled to avoid erroneous estimations with large variances.

Task modulates functional connectivity networks in free viewing behavior

In free visual exploration, eye-movement is immediately followed by dynamic reconfiguration of brain functional connectivity. We studied the task-dependency of this process in a combined visual search-change detection experiment. Participants viewed two (nearly) same displays in succession. First time they had to find and remember multiple targets among distractors, so the ongoing task involved memory encoding. Second time they had to determine if a target had changed in orientation, so the ongoing task involved memory retrieval. From multichannel EEG recorded during 200 ms intervals time-locked to fixation onsets, we estimated the functional connectivity using a weighted phase lag index at the frequencies of theta, alpha, and beta bands, and derived global and local measures of the functional connectivity graphs. We found differences between both memory task conditions for several network measures, such as mean path length, radius, diameter, closeness and eccentricity, mainly in the alpha band. Both the local and the global measures indicated that encoding involved a more segregated mode of operation than retrieval. These differences arose immediately after fixation onset and persisted for the entire duration of the lambda complex, an evoked potential commonly associated with early visual perception. We concluded that encoding and retrieval differentially shape network configurations involved in early visual perception, affecting the way the visual input is processed at each fixation. These findings demonstrate that task requirements dynamically control the functional connectivity networks involved in early visual perception.

Isolating Discriminant Neural Activity in the Presence of Eye Movements and Concurrent Task Demands

A growing number of studies use the combination of eye-tracking and electroencephalographic (EEG) measures to explore the neural processes that underlie visual perception. In these studies, fixation-related potentials (FRPs) are commonly used to quantify early and late stages of visual processing that follow the onset of each fixation. However, FRPs reflect a mixture of bottom-up (sensory-driven) and top-down (goal-directed) processes, in addition to eye movement artifacts and unrelated neural activity. At present there is little consensus on how to separate this evoked response into its constituent elements. In this study we sought to isolate the neural sources of target detection in the presence of eye movements and over a range of concurrent task demands. Here, participants were asked to identify visual targets (Ts) amongst a grid of distractor stimuli (Ls), while simultaneously performing an auditory N-back task. To identify the discriminant activity, we used independent components analysis (ICA) for the separation of EEG into neural and non-neural sources. We then further separated the neural sources, using a modified measure-projection approach, into six regions of interest (ROIs): occipital, fusiform, temporal, parietal, cingulate, and frontal cortices. Using activity from these ROIs, we identified target from non-target fixations in all participants at a level similar to other state-of-the-art classification techniques. Importantly, we isolated the time course and spectral features of this discriminant activity in each ROI. In addition, we were able to quantify the effect of cognitive load on both fixation-locked potential and classification performance across regions. Together, our results show the utility of a measure-projection approach for separating task-relevant neural activity into meaningful ROIs within more complex contexts that include eye movements.

The Impact of Task Demands on Fixation-Related Brain Potentials during Guided Search

Recording synchronous data from EEG and eye-tracking provides a unique methodological approach for measuring the sensory and cognitive processes of overt visual search. Using this approach we obtained fixation related potentials (FRPs) during a guided visual search task specifically focusing on the lambda and P3 components. An outstanding question is whether the lambda and P3 FRP components are influenced by concurrent task demands. We addressed this question by obtaining simultaneous eye-movement and electroencephalographic (EEG) measures during a guided visual search task while parametrically modulating working memory load using an auditory N-back task. Participants performed the guided search task alone, while ignoring binaurally presented digits, or while using the auditory information in a 0, 1, or 2-back task. The results showed increased reaction time and decreased accuracy in both the visual search and N-back tasks as a function of auditory load. Moreover, high auditory task demands increased the P3 but not the lambda latency while the amplitude of both lambda and P3 was reduced during high auditory task demands. The results show that both early and late stages of visual processing indexed by FRPs are significantly affected by concurrent task demands imposed by auditory working memory.

Classification of Eye Fixation Related Potentials for Variable Stimulus Saliency

OBJECTIVE

Electroencephalography (EEG) and eye tracking can possibly provide information about which items displayed on the screen are relevant for a person. Exploiting this implicit information promises to enhance various software applications. The specific problem addressed by the present study is that items shown in real applications are typically diverse. Accordingly, the saliency of information, which allows to discriminate between relevant and irrelevant items, varies. As a consequence, recognition can happen in foveal or in peripheral vision, i.e., either before or after the saccade to the item. Accordingly, neural processes related to recognition are expected to occur with a variable latency with respect to the eye movements. The aim was to investigate if relevance estimation based on EEG and eye tracking data is possible despite of the aforementioned variability.

APPROACH

Sixteen subjects performed a search task where the target saliency was varied while the EEG was recorded and the unrestrained eye movements were tracked. Based on the acquired data, it was estimated which of the items displayed were targets and which were distractors in the search task.

RESULTS

Target prediction was possible also when the stimulus saliencies were mixed. Information contained in EEG and eye tracking data was found to be complementary and neural signals were captured despite of the unrestricted eye movements. The classification algorithm was able to cope with the experimentally induced variable timing of neural activity related to target recognition.

SIGNIFICANCE

It was demonstrated how EEG and eye tracking data can provide implicit information about the relevance of items on the screen for potential use in online applications.

Sequential effects in continued visual search: using fixation-related potentials to compare distractor processing before and after target detection

To search for a target in a complex environment is an everyday behavior that ends with finding the target. When we search for two identical targets, however, we must continue the search after finding the first target and memorize its location. We used fixation-related potentials to investigate the neural correlates of different stages of the search, that is, before and after finding the first target. Having found the first target influenced subsequent distractor processing. Compared to distractor fixations before the first target fixation, a negative shift was observed for three subsequent distractor fixations. These results suggest that processing a target in continued search modulates the brain's response, either transiently by reflecting temporary working memory processes or permanently by reflecting working memory retention.

Co-registration of eye movements and event-related potentials in connected-text paragraph reading

Eyetracking during reading has provided a critical source of on-line behavioral data informing basic theory in language processing. Similarly, event-related potentials (ERPs) have provided an important on-line measure of the neural correlates of language processing. Recently there has been strong interest in co-registering eyetracking and ERPs from simultaneous recording to capitalize on the strengths of both techniques, but a challenge has been devising approaches for controlling artifacts produced by eye movements in the EEG waveform. In this paper we describe our approach to correcting for eye movements in EEG and demonstrate its applicability to reading. The method is based on independent components analysis, and uses three criteria for identifying components tied to saccades: (1) component loadings on the surface of the head are consistent with eye movements; (2) source analysis localizes component activity to the eyes, and (3) the temporal activation of the component occurred at the time of the eye movement and differed for right and left eye movements. We demonstrate this method's applicability to reading by comparing ERPs time-locked to fixation onset in two reading conditions. In the text-reading condition, participants read paragraphs of text. In the pseudo-reading control condition, participants moved their eyes through spatially similar pseudo-text that preserved word locations, word shapes, and paragraph spatial structure, but eliminated meaning. The corrected EEG, time-locked to fixation onsets, showed effects of reading condition in early ERP components. The results indicate that co-registration of eyetracking and EEG in connected-text paragraph reading is possible, and has the potential to become an important tool for investigating the cognitive and neural bases of on-line language processing in reading.

Speed of mental addition in an abacus expert, estimated by eye movements and neural activities

A grand expert of abacus looked at a display on which three-digit numbers were aligned vertically with constant spacing, for sequentially adding the numbers mentally. His eye regularly moved downward by alternating a fixation to one of the numbers with a saccade to another below it, with the average period of the alternations at 271 msec. His magnetoencephalogram averaged with respect to start of the fixations revealed activity in the right superior parietal cortex. This finding not only supported the previous view that abacus experts mentally calculate by manipulating spatial representations of numbers, but also showed that the calculation was synchronized with the periodic eye movements. Thus, each process of the mental sequential addition, which starts with visual recognition of an addend, was estimated to require less than 271 msec. on average.

Neural saccadic response estimation during natural viewing

Studying neural activity during natural viewing conditions is not often attempted. Isolating the neural response of a single saccade is necessary to study neural activity during natural viewing; however, the close temporal spacing of saccades that occurs during natural viewing makes it difficult to determine the response to a single saccade. Herein, a general linear model (GLM) approach is applied to estimate the EEG neural saccadic response for different segments of the saccadic main sequence separately. It is determined that, in visual search conditions, neural responses estimated by conventional event-related averaging are significantly and systematically distorted relative to GLM estimates due to the close temporal spacing of saccades during visual search. Before the GLM is applied, analyses are applied that demonstrate that saccades during visual search with intersaccadic spacings as low as 100-150 ms do not exhibit significant refractory effects. Therefore, saccades displaying different intersaccadic spacings during visual search can be modeled using the same regressor in a GLM. With the use of the GLM approach, neural responses were separately estimated for five different ranges of saccade amplitudes during visual search. Occipital responses time locked to the onsets of saccades during visual search were found to account for, on average, 79 percent of the variance of EEG activity in a window 90-200 ms after the onsets of saccades for all five saccade amplitude ranges that spanned a range of 0.2-6.0 degrees. A GLM approach was also used to examine the lateralized ocular artifacts associated with saccades. Possible extensions of the methods presented here to account for the superposition of microsaccades in event-related EEG studies conducted in nominal fixation conditions are discussed.

Superposition model predicts EEG occipital activity during free viewing of natural scenes

Visual event-related potentials (ERPs) produced by a stimulus are thought to reflect either an increase of synchronized activity or a phase realignment of ongoing oscillatory activity, with both mechanisms sharing the assumption that ERPs are independent of the current state of the brain at the time of stimulation. In natural viewing, however, visual inputs occur one after another at specific subject-paced intervals through unconstrained eye movements. We conjecture that during natural viewing, ERPs generated after each fixation are better explained by a superposition of ongoing oscillatory activity related to the processing of previous fixations, with new activity elicited by the visual input at the current fixation. We examined the electroencephalography (EEG) signals that occur in humans at the onset of each visual fixation, both while subjects freely viewed natural scenes and while they viewed a black or gray background. We found that the fixation ERPs show visual components that are absent when subjects move their eyes on a homogeneous gray or black screen. Single-trial EEG signals that comprise the ERP are predicted more accurately by a model of superposition than by either phase resetting or the addition of evoked responses and stimulus-independent noise. The superposition of ongoing oscillatory activity and the visually evoked response results in a modification of the ongoing oscillation phase. The results presented suggest that the observed EEG signals reflect changes occurring in a common neuronal substrate rather than a simple summation at the scalp of signals from independent sources.

Human microsaccade-related visual brain responses

Microsaccades are very small, involuntary flicks in eye position that occur on average once or twice per second during attempted visual fixation. Microsaccades give rise to EMG eye muscle spikes that can distort the spectrum of the scalp EEG and mimic increases in gamma band power. Here we demonstrate that microsaccades are also accompanied by genuine and sizeable cortical activity, manifested in the EEG. In three experiments, high-resolution eye movements were corecorded with the EEG: during sustained fixation of checkerboard and face stimuli and in a standard visual oddball task that required the counting of target stimuli. Results show that microsaccades as small as 0.15 degrees generate a field potential over occipital cortex and midcentral scalp sites 100-140 ms after movement onset, which resembles the visual lambda response evoked by larger voluntary saccades. This challenges the standard assumption of human brain imaging studies that saccade-related brain activity is precluded by fixation, even when fully complied with. Instead, additional cortical potentials from microsaccades were present in 86% of the oddball task trials and of similar amplitude as the visual response to stimulus onset. Furthermore, microsaccade probability varied systematically according to the proportion of target stimuli in the oddball task, causing modulations of late stimulus-locked event-related potential (ERP) components. Microsaccades present an unrecognized source of visual brain signal that is of interest for vision research and may have influenced the data of many ERP and neuroimaging studies.

Pattern-induced negative occipital potentials (PINOP)

SUMMARY

: We describe 24 subjects (16 female, 8 male; age range, 3 to 36 years [mean, 15.7]) who exhibited a hitherto inadequately described EEG response to visual scanning of geometric patterns. Like lambda waves (the normal physiologic response to visual scanning of pictures), the responses to pattern scanning were occipitally distributed and disappeared on eye closure. In contrast to lambda waves, they were surface negative, briefer in duration, and sharper in contour. To designate them, we have coined the term pattern-induced negative occipital potentials (PINOP). The prevalence of epilepsy was not significantly different among subjects with PINOP than among age- and sex-matched control subjects who did not have PINOP. None of the patients with PINOP exhibited paroxysmal epileptiform responses to intermittent photic stimulation or pattern activation. We conclude that PINOP represent an unusual benign physiologic phenomenon. The restricted distribution over the posterior head region and absence of associated, more widely distributed epileptiform responses during photic stimulation and pattern testing are clues to the diagnosis. We hypothesize that a differential area of activation in the occipital lobe during pattern viewing versus picture viewing and orientation of the activated areas to the scalp electrodes may account for the differences in the morphology of the wave forms observed during conventional EEG recording.

Spatio-temporal brain dynamics underlying saccade execution, suppression, and error-related feedback

Human and nonhuman animal research has outlined the neural regions that support saccadic eye movements. The aim of the current work was to outline the sequence by which distinct neural regions come on-line to support goal-directed saccade execution and error-related feedback. To achieve this, we obtained behavioral responses via eye movement recordings and neural responses via magnetoencephalography (MEG), concurrently, while participants performed an antisaccade task. Neural responses were examined with respect to the onset of the saccadic eye movements. Frontal eye field and visual cortex activity distinguished subsequently successful goal-directed saccades from (correct and erroneous) reflexive saccades prior to the deployment of the eye movement. Activity in the same neural regions following the saccadic movement distinguished correct from incorrect saccadic responses. Error-related activity in the frontal eye fields preceded that from visual regions, suggesting a potential feedback network that may drive corrective eye movements. This work provides the first empirical demonstration of simultaneous remote eyetracking and MEG recording. The coupling of behavioral and neuroimaging technologies, used here to characterize dynamic brain networks underlying saccade execution and error-related feedback, demonstrates a novel within-paradigm converging evidence approach by which to outline the neural underpinnings of cognition.

Single-trial lambda wave identification using a fuzzy inference system and predictive statistical diagnosis

The aim of the study was to automate the identification of a saccade-related visual evoked potential (EP) called the lambda wave. The lambda waves were extracted from single trials of electroencephalogram (EEG) waveforms using independent component analysis (ICA). A trial was a set of EEG waveforms recorded from 64 scalp electrode locations while a saccade was performed. Forty saccade-related EEG trials (recorded from four normal subjects) were used in the study. The number of waveforms per trial was reduced from 64 to 22 by pre-processing. The application of ICA to the resulting waveforms produced 880 components (i.e. 4 subjects x 10 trials per subject x 22 components per trial). The components were divided into 373 lambda and 507 nonlambda waves by visual inspection and then they were represented by one spatial and two temporal features. The classification performance of a Bayesian approach called predictive statistical diagnosis (PSD) was compared with that of a fuzzy logic approach called a fuzzy inference system (FIS). The outputs from the two classification approaches were then combined and the resulting discrimination accuracy was evaluated. For each approach, half the data from the lambda and nonlambda wave categories were used to determine the operating parameters of the classification schemes while the rest (i.e. the validation set) were used to evaluate their classification accuracies. The sensitivity and specificity values when the classification approaches were applied to the lambda wave validation data set were as follows: for the PSD 92.51% and 91.73% respectively, for the FIS 95.72% and 89.76% respectively, and for the combined FIS and PSD approach 97.33% and 97.24% respectively (classification threshold was 0.5). The devised signal processing techniques together with the classification approaches provided for an effective extraction and classification of the single-trial lambda waves. However, as only four subjects were included, it will be valuable to further evaluate the methods on a larger group of subjects.

Comparison between the lambda response of eye-fixation-related potentials and the P100 component of pattern-reversal visual evoked potentials

The purpose of this study was to compare the lambda response of eye-fixation-related potentials (EFRPs) with the P100 component of pattern-reversal visual-evoked potentials. EFRPs were obtained by averaging EEGs time-locked to the offset of the saccade. The dipole of the lambda response and that of the P100 component were estimated by the dipole-tracing method (Musha & Homma, 1990). The locations of their dipoles at the occipital sites were very close to each other when the difference waveform, which was calculated by subtracting the EFRP to the patternless stimulus from the EFRP to the patterned stimulus, was used for the lambda response. This finding implies that the lambda response and P100 have a common neural generator in the visual cortex. However, the peak latency of the lambda response was shorter than that of P100. The saccades in the EFRP trial were considered to be the cause of the difference.