Saccade size and lambda complex in man

Saccade size predicts onset time of object processing during visual search of an open world virtual environment

OBJECTIVE

To date the vast majority of research in the visual neurosciences have been forced to adopt a highly constrained perspective of the vision system in which stimuli are processed in an open-loop reactive fashion (i.e., abrupt stimulus presentation followed by an evoked neural response). While such constraints enable high construct validity for neuroscientific investigation, the primary outcomes have been a reductionistic approach to isolate the component processes of visual perception. In electrophysiology, of the many neural processes studied under this rubric, the most well-known is, arguably, the P300 evoked response. There is, however, relatively little known about the real-world corollary of this component in free-viewing paradigms where visual stimuli are connected to neural function in a closed-loop. While growing evidence suggests that neural activity analogous to the P300 does occur in such paradigms, it is an open question when this response occurs and what behavioral or environmental factors could be used to isolate this component.

APPROACH

The current work uses convolutional networks to decode neural signals during a free-viewing visual search task in a closed-loop paradigm within an open-world virtual environment. From the decoded activity we construct fixation-locked response profiles that enable estimations of the variable latency of any P300 analogue around the moment of fixation. We then use these estimates to investigate which factors best reduce variable latency and, thus, predict the onset time of the response. We consider measurable, search-related factors encompassing top-down (i.e., goal driven) and bottom-up (i.e., stimulus driven) processes, such as fixation duration and salience. We also consider saccade size as an intermediate factor reflecting the integration of these two systems.

MAIN RESULTS

The results show that of these factors only saccade size reliably determines the onset time of P300 analogous activity for this task. Specifically, we find that for large saccades the variability in response onset is small enough to enable analysis using traditional ensemble averaging methods.

SIGNIFICANCE

The results show that P300 analogous activity does occur during closed-loop, free-viewing visual search while highlighting distinct differences between the open-loop version of this response and its real-world analogue. The results also further establish saccades, and saccade size, as a key factor in real-world visual processing.

Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

Under natural viewing conditions, complex stimuli such as human faces are typically looked at several times in succession, implying that their recognition may unfold across multiple eye fixations. Although electrophysiological (EEG) experiments on face recognition typically prohibit eye movements, participants still execute frequent (micro)saccades on the face, each of which generates its own visuocortical response. This finding raises the question of whether the fixation-related potentials (FRPs) evoked by these tiny gaze shifts also contain psychologically valuable information about face processing. Here, we investigated this question by corecording EEG and eye movements in an experiment with emotional faces (happy, angry, neutral). Deconvolution modeling was used to separate the stimulus ERPs to face onset from the FRPs generated by subsequent microsaccades-induced refixations on the face. As expected, stimulus ERPs exhibited typical emotion effects, with a larger early posterior negativity (EPN) for happy/angry compared with neutral faces. Eye tracking confirmed that participants made small saccades in 98% of the trials, which were often aimed at the left eye of the stimulus face. However, while each saccade produced a strong response over visual areas, this response was unaffected by the face's emotional expression, both for the first and for subsequent (micro)saccades. This finding suggests that the face's affective content is rapidly evaluated after stimulus onset, leading to only a short-lived sensory enhancement by arousing stimuli that does not repeat itself during immediate refixations. Methodologically, our work demonstrates how eye tracking and deconvolution modeling can be used to extract several brain responses from each EEG trial, providing insights into neural processing at different latencies after stimulus onset.

Semantic novelty modulates neural responses to visual change across the human brain

Our continuous visual experience in daily life is dominated by change. Previous research has focused on visual change due to stimulus motion, eye movements or unfolding events, but not their combined impact across the brain, or their interactions with semantic novelty. We investigate the neural responses to these sources of novelty during film viewing. We analyzed intracranial recordings in humans across 6328 electrodes from 23 individuals. Responses associated with saccades and film cuts were dominant across the entire brain. Film cuts at semantic event boundaries were particularly effective in the temporal and medial temporal lobe. Saccades to visual targets with high visual novelty were also associated with strong neural responses. Specific locations in higher-order association areas showed selectivity to either high or low-novelty saccades. We conclude that neural activity associated with film cuts and eye movements is widespread across the brain and is modulated by semantic novelty.

Eye-Fixation-Related Potentials (EFRPs) As a Predictor of Human Error Occurrences During a Visual Inspection Task

Estimation of human attentional states using an electroencephalogram (EEG) has been demonstrated to help prevent human errors associated with the degradation. Since the use of the lambda response -one of eye-fixation-related potentials time-locked to the saccade offset- enables such estimation without external triggers, the measurements are compatible for an application in a real-world environment. With aiming to apply the lambda response as an index of human errors during the visual inspection, the current research elucidated whether the mean amplitude of the lambda response was a predictor of the number of inspection errors. EEGs were measured from 50 participants while inspecting the differences between two images of the circuit board. Twenty percent of the total number of image pairs included differences. The lambda response was obtained relative to a saccade offset starting a fixation of the inspection image. Participants conducted four sessions over two days (625 trials/ session, 2 sessions/ day). A Poisson regression of the number of inspection errors using a generalized linear mixed model showed that a coefficient of the mean amplitude of the lambda response was significant , suggesting that the response has a role in th$(\hat \beta = 0.24,p < 0.01)$e prediction of the number of human error occurrences in the visual inspection.

Neural correlates of intra-saccadic motion perception

Retinal motion of the visual scene is not consciously perceived during ocular saccades in normal everyday conditions. It has been suggested that extra-retinal signals actively suppress intra-saccadic motion perception to preserve stable perception of the visual world. However, using stimuli optimized to preferentially activate the M-pathway, Castet and Masson (2000) demonstrated that motion can be perceived during a saccade. Based on this psychophysical paradigm, we used electroencephalography and eye-tracking recordings to investigate the neural correlates related to the conscious perception of intra-saccadic motion. We demonstrated the effective involvement during saccades of the cortical areas V1-V2 and MT-V5, which convey motion information along the M-pathway. We also showed that individual motion perception was related to retinal temporal frequency.

During natural viewing, neural processing of visual targets continues throughout saccades

Relatively little is known about visual processing during free-viewing visual search in realistic dynamic environments. Free-viewing is characterized by frequent saccades. During saccades, visual processing is thought to be suppressed, yet we know that the presaccadic visual content can modulate postsaccadic processing. To better understand these processes in a realistic setting, we study here saccades and neural responses elicited by the appearance of visual targets in a realistic virtual environment. While subjects were being driven through a 3D virtual town, they were asked to discriminate between targets that appear on the road. Using a system identification approach, we separated overlapping and correlated activity evoked by visual targets, saccades, and button presses. We found that the presence of a target enhances early occipital as well as late frontocentral saccade-related responses. The earlier potential, shortly after 125 ms post-saccade onset, was enhanced for targets that appeared in the peripheral vision as compared to the central vision, suggesting that fast peripheral processing initiated before saccade onset. The later potential, at 195 ms post-saccade onset, was strongly modulated by the visibility of the target. Together these results suggest that, during natural viewing, neural processing of the presaccadic visual stimulus continues throughout the saccade, apparently unencumbered by saccadic suppression.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Neural correlates of task-related refixation behavior

Eye movement research has shown that attention shifts from the currently fixated location to the next before a saccade is executed. We investigated whether the cost of the attention shift depends on higher-order processing at the time of fixation, in particular on visual working memory load differences between fixations and refixations on task-relevant items. The attention shift is reflected in EEG activity in the saccade-related potential (SRP). In a free viewing task involving visual search and memorization of multiple targets amongst distractors, we compared the SRP in first fixations versus refixations on targets and distractors. The task-relevance of targets implies that more information will be loaded in memory (e.g. both identity and location) than for distractors (e.g. location only). First fixations will involve greater memory load than refixations, since first fixations involve loading of new items, while refixations involve rehearsal of previously visited items. The SRP in the interval preceding the saccade away from a target or distractor revealed that saccade preparation is affected by task-relevance and refixation behavior. For task-relevant items only, we found longer fixation duration and higher SRP amplitudes for first fixations than for refixations over the occipital region and the opposite effect over the frontal region. Our findings provide first neurophysiological evidence that working memory loading of task-relevant information at fixation affects saccade planning.

Eye contact in active and passive viewing: Event-related brain potential evidence from a combined eye tracking and EEG study

Eye contact is a salient social cue, which is assumed to influence already early neural correlates of face perception. Specifically, the N170 component of the event-related potential (ERP) has often been found to be larger for faces with an averted gaze as compared to faces that directly look at the observer. In most existing ERP studies, effects of eye contact were investigated under comparatively artificial conditions where participants were instructed to maintain a steady fixation while they passively observed gaze changes in the stimulus face. It is therefore unclear to what extent neural correlates of eye contact generalize to more naturalistic situations that involve a continuous interplay between directed and averted gaze between the communication partners. To start bridging this gap, the present study compared the passive viewing of gaze changes to an active condition in which the participant's own gaze (measured online with an eye tracker) interacted with the gaze position of a continuously presented stimulus face. We also investigated whether eye contact effects were modulated by the face's emotional expression. In both the passive and the active viewing condition, N170 amplitudes were larger when the gaze of the stimulus faces was averted rather than directed towards the participant. Furthermore, eye contact decreased P300 amplitudes in both conditions. The emotional expression of the face also modulated the N170, but this effect did not interact with that of gaze direction. We conclude that the neural correlates of gaze perception during active gaze interactions are comparable to those found during passive viewing, encouraging the further study of eye contact effects in more naturalistic settings.

The effects of cognitive distraction on behavioural, oculomotor and electrophysiological metrics during a driving hazard perception task

Previous research has demonstrated that the distraction caused by holding a mobile telephone conversation is not limited to the period of the actual conversation (Haigney, 1995; Redelmeier & Tibshirani, 1997; Savage et al., 2013). In a prior study we identified potential eye movement and EEG markers of cognitive distraction during driving hazard perception. However the extent to which these markers are affected by the demands of the hazard perception task are unclear. Therefore in the current study we assessed the effects of secondary cognitive task demand on eye movement and EEG metrics separately for periods prior to, during and after the hazard was visible. We found that when no hazard was present (prior and post hazard windows), distraction resulted in changes to various elements of saccadic eye movements. However, when the target was present, distraction did not affect eye movements. We have previously found evidence that distraction resulted in an overall decrease in theta band output at occipital sites of the brain. This was interpreted as evidence that distraction results in a reduction in visual processing. The current study confirmed this by examining the effects of distraction on the lambda response component of subjects eye fixation related potentials (EFRPs). Furthermore, we demonstrated that although detections of hazards were not affected by distraction, both eye movement and EEG metrics prior to the onset of the hazard were sensitive to changes in cognitive workload. This suggests that changes to specific aspects of the saccadic eye movement system could act as unobtrusive markers of distraction even prior to a breakdown in driving performance.

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.

The impact of driver sleepiness on fixation-related brain potentials

The effects of driver sleepiness are often quantified as deteriorated driving performance, increased blink durations and high levels of subjective sleepiness. Driver sleepiness has also been associated with increasing levels of electroencephalogram (EEG) power, especially in the alpha range. The present exploratory study investigated a new measure of driver sleepiness, the EEG fixation-related lambda response. Thirty young male drivers (23.6 ± 1.7 years old) participated in a driving simulator experiment in which they drove on rural and suburban roads in simulated daylight versus darkness during both the daytime (full sleep) and night-time (sleep deprived). The results show lower lambda responses during night driving and with longer time on task, indicating that sleep deprivation and time on task cause a general decrement in cortical responsiveness to incoming visual stimuli. Levels of subjective sleepiness and line crossings were higher under the same conditions. Furthermore, results of a linear mixed-effects model showed that low lambda responses are associated with high subjective sleepiness and more line crossings. We suggest that the fixation-related lambda response can be used to investigate driving impairment induced by sleep deprivation while driving and that, after further refinement, it may be useful as an objective measure of driver sleepiness.

Transsacadic Information and Corollary Discharge in Local Field Potentials of Macaque V1

Approximately three times per second, human visual perception is interrupted by a saccadic eye movement. In addition to taking the eyes to a new location, several lines of evidence suggest that the saccades play multiple roles in visual perception. Indeed, it may be crucial that visual processing is informed about movements of the eyes in order to analyze visual input distinctly and efficiently on each fixation and preserve stable visual perception of the world across saccades. A variety of studies has demonstrated that activity in multiple brain areas is modulated by saccades. The hypothesis tested here is that these signals carry significant information that could be used in visual processing. To test this hypothesis, local field potentials (LFPs) were simultaneously recorded from multiple electrodes in macaque primary visual cortex (V1); support vector machines (SVMs) were used to classify the peri-saccadic LFPs. We find that LFPs in area V1 carry information that can be used to distinguish neural activity associated with fixations from saccades, precisely estimate the onset time of fixations, and reliably infer the directions of saccades. This information may be used by the brain in processes including visual stability, saccadic suppression, receptive field (RF) remapping, fixation amplification, and trans-saccadic visual perception.

Brain Responses to Emotional Faces in Natural Settings: A Wireless Mobile EEG Recording Study

The detection of a human face in a visual field and correct reading of emotional expression of faces are important elements in everyday social interactions, decision making and emotional responses. Although brain correlates of face processing have been established in previous fMRI and electroencephalography (EEG)/MEG studies, little is known about how the brain representation of faces and emotional expressions of faces in freely moving humans. The present study aimed to detect brain electrical potentials that occur during the viewing of human faces in natural settings. 64-channel wireless EEG and eye-tracking data were recorded in 19 participants while they moved in a mock art gallery and stopped at times to evaluate pictures hung on the walls. Positive, negative and neutral valence pictures of objects and human faces were displayed. The time instants in which pictures first occurred in the visual field were identified in eye-tracking data and used to reconstruct the triggers in continuous EEG data after synchronizing the time axes of the EEG and eye-tracking device. EEG data showed a clear face-related event-related potential (ERP) in the latency interval ranging from 165 to 210 ms (N170); this component was not seen whilst participants were viewing non-living objects. The face ERP component was stronger during viewing disgusted compared to neutral faces. Source dipole analysis revealed an equivalent current dipole in the right fusiform gyrus (BA37) accounting for N170 potential. Our study demonstrates for the first time the possibility of recording brain responses to human faces and emotional expressions in natural settings. This finding opens new possibilities for clinical, developmental, social, forensic, or marketing research in which information about face processing is of importance.

Presaccadic EEG activity predicts visual saliency in free-viewing contour integration

While viewing a scene, the eyes are attracted to salient stimuli. We set out to identify the brain signals controlling this process. In a contour integration task, in which participants searched for a collinear contour in a field of randomly oriented Gabor elements, a previously established model was applied to calculate a visual saliency value for each fixation location. We studied brain activity related to the modeled saliency values, using coregistered eye tracking and EEG. To disentangle EEG signals reflecting salience in free viewing from overlapping EEG responses to sequential eye movements, we adopted generalized additive mixed modeling (GAMM) to single epochs of saccade-related EEG. We found that, when saliency at the next fixation location was high, amplitude of the presaccadic EEG activity was low. Since presaccadic activity reflects covert attention to the saccade target, our results indicate that larger attentional effort is needed for selecting less salient saccade targets than more salient ones. This effect was prominent in contour-present conditions (half of the trials), but ambiguous in the contour-absent condition. Presaccadic EEG activity may thus be indicative of bottom-up factors in saccade guidance. The results underscore the utility of GAMM for EEG-eye movement coregistration research.

Temporal Dynamics of Natural Static Emotional Facial Expressions Decoding: A Study Using Event- and Eye Fixation-Related Potentials

This study aims at examining the precise temporal dynamics of the emotional facial decoding as it unfolds in the brain, according to the emotions displayed. To characterize this processing as it occurs in ecological settings, we focused on unconstrained visual explorations of natural emotional faces (i.e., free eye movements). The General Linear Model (GLM; Smith and Kutas, 2015a,b; Kristensen et al., 2017a) enables such a depiction. It allows deconvolving adjacent overlapping responses of the eye fixation-related potentials (EFRPs) elicited by the subsequent fixations and the event-related potentials (ERPs) elicited at the stimuli onset. Nineteen participants were displayed with spontaneous static facial expressions of emotions (Neutral, Disgust, Surprise, and Happiness) from the DynEmo database (Tcherkassof et al., 2013). Behavioral results on participants' eye movements show that the usual diagnostic features in emotional decoding (eyes for negative facial displays and mouth for positive ones) are consistent with the literature. The impact of emotional category on both the ERPs and the EFRPs elicited by the free exploration of the emotional faces is observed upon the temporal dynamics of the emotional facial expression processing. Regarding the ERP at stimulus onset, there is a significant emotion-dependent modulation of the P2-P3 complex and LPP components' amplitude at the left frontal site for the ERPs computed by averaging. Yet, the GLM reveals the impact of subsequent fixations on the ERPs time-locked on stimulus onset. Results are also in line with the valence hypothesis. The observed differences between the two estimation methods (Average vs. GLM) suggest the predominance of the right hemisphere at the stimulus onset and the implication of the left hemisphere in the processing of the information encoded by subsequent fixations. Concerning the first EFRP, the Lambda response and the P2 component are modulated by the emotion of surprise compared to the neutral emotion, suggesting an impact of high-level factors, in parieto-occipital sites. Moreover, no difference is observed on the second and subsequent EFRP. Taken together, the results stress the significant gain obtained in analyzing the EFRPs using the GLM method and pave the way toward efficient ecological emotional dynamic stimuli analyses.

Large-Scale Traveling Waves in EEG Activity Following Eye Movement

In spontaneous, stimulus-evoked, and eye-movement evoked EEG, the oscillatory signal shows large scale, dynamically organized patterns of phase. We investigated eye-movement evoked patterns in free-viewing conditions. Participants viewed photographs of natural scenes in anticipation of a memory test. From 200 ms intervals following saccades, we estimated the EEG phase gradient over the entire scalp, and the wave activity, i.e. the goodness of fit of a wave model involving a phase gradient assumed to be smooth over the scalp. In frequencies centered at 6.5 Hz, large-scale phase organization occurred, peaking around 70 ms after fixation onset and taking the form of a traveling wave. According to the wave gradient, most of the times the wave spreads from the posterior-inferior to anterior-superior direction. In these directions, the gradients depended on the size and direction of the saccade. Wave propagation velocity decreased in the course of the fixation, particularly in the interval from 50 to 150 ms after fixation onset. This interval corresponds to the fixation-related lambda activity, which reflects early perceptual processes following fixation onset. We conclude that lambda activity has a prominent traveling wave component. This component consists of a short-term whole-head phase pattern of specific direction and velocity, which may reflect feedforward propagation of visual information at fixation.

Task modulates functional connectivity networks in free viewing behavior

In free visual exploration, eye-movement is immediately followed by dynamic reconfiguration of brain functional connectivity. We studied the task-dependency of this process in a combined visual search-change detection experiment. Participants viewed two (nearly) same displays in succession. First time they had to find and remember multiple targets among distractors, so the ongoing task involved memory encoding. Second time they had to determine if a target had changed in orientation, so the ongoing task involved memory retrieval. From multichannel EEG recorded during 200 ms intervals time-locked to fixation onsets, we estimated the functional connectivity using a weighted phase lag index at the frequencies of theta, alpha, and beta bands, and derived global and local measures of the functional connectivity graphs. We found differences between both memory task conditions for several network measures, such as mean path length, radius, diameter, closeness and eccentricity, mainly in the alpha band. Both the local and the global measures indicated that encoding involved a more segregated mode of operation than retrieval. These differences arose immediately after fixation onset and persisted for the entire duration of the lambda complex, an evoked potential commonly associated with early visual perception. We concluded that encoding and retrieval differentially shape network configurations involved in early visual perception, affecting the way the visual input is processed at each fixation. These findings demonstrate that task requirements dynamically control the functional connectivity networks involved in early visual perception.

The Impact of Task Demands on Fixation-Related Brain Potentials during Guided Search

Recording synchronous data from EEG and eye-tracking provides a unique methodological approach for measuring the sensory and cognitive processes of overt visual search. Using this approach we obtained fixation related potentials (FRPs) during a guided visual search task specifically focusing on the lambda and P3 components. An outstanding question is whether the lambda and P3 FRP components are influenced by concurrent task demands. We addressed this question by obtaining simultaneous eye-movement and electroencephalographic (EEG) measures during a guided visual search task while parametrically modulating working memory load using an auditory N-back task. Participants performed the guided search task alone, while ignoring binaurally presented digits, or while using the auditory information in a 0, 1, or 2-back task. The results showed increased reaction time and decreased accuracy in both the visual search and N-back tasks as a function of auditory load. Moreover, high auditory task demands increased the P3 but not the lambda latency while the amplitude of both lambda and P3 was reduced during high auditory task demands. The results show that both early and late stages of visual processing indexed by FRPs are significantly affected by concurrent task demands imposed by auditory working memory.

Sequential effects in continued visual search: using fixation-related potentials to compare distractor processing before and after target detection

To search for a target in a complex environment is an everyday behavior that ends with finding the target. When we search for two identical targets, however, we must continue the search after finding the first target and memorize its location. We used fixation-related potentials to investigate the neural correlates of different stages of the search, that is, before and after finding the first target. Having found the first target influenced subsequent distractor processing. Compared to distractor fixations before the first target fixation, a negative shift was observed for three subsequent distractor fixations. These results suggest that processing a target in continued search modulates the brain's response, either transiently by reflecting temporary working memory processes or permanently by reflecting working memory retention.

Early event-related cortical activity originating in the frontal eye fields and inferior parietal lobe predicts the occurrence of correct and error saccades

Although the cortical circuitry underlying saccade execution has well been specified by neurophysiological and functional imaging studies, the temporal dynamics of cortical activity predicting the occurrence of voluntary or reflexive saccades in humans are largely unknown. Here, we examined electrophysiological activity preceding the onset of correct (i.e., voluntary) or error (i.e., reflexive) saccades in an oculomotor capture task. Participants executed saccades to lateralized visual targets while attempting to inhibit reflexive glances to abruptly appearing distracters. Since the visual display was identical for both types of saccades, different electrophysiological patterns preceding correct and error saccades could not be explained by low-level perceptual differences. Compared to correct saccades electrophysiological activity preceding error saccades showed significant differences of the scalp electric field and of voltage amplitudes at posterior electrodes. In addition, though error saccades had significantly shorter latency than correct saccades a prolonged topographic configuration of electric potentials prior to error saccades was found ∼120-140 ms following target onset. In agreement with the known asymmetry in hemispheric dominance for spatial attention, distinct electrophysiological patterns were only found for leftward saccades. While error saccades were associated with stronger activity in the right Frontal Eye Field, correct saccades were preceded by stronger activity in the inferior parietal lobule. These findings suggest that selection of the saccade target in a conflicting situation is determined by early top-down biases originating in frontal and parietal cortical regions critical for spatial attention and saccade programming.

Saccadic context indicates information processing within visual fixations: evidence from event-related potentials and eye-movements analysis of the distractor effect

Attention, visual information processing, and oculomotor control are integrated functions of closely related brain mechanisms. Recently, it was shown that the processing of visual distractors appearing during a fixation is modulated by the amplitude of its preceding saccade (Pannasch & Velichkovsky, 2009). So far, this was demonstrated only at the behavioral level in terms of saccadic inhibition. The present study investigated distractor-related brain activity with cortical eye fixation-related potentials (EFRPs). Moreover, the following saccade was included as an additional classification criterion. Eye movements and EFRPs were recorded during free visual exploration of paintings. During some of the fixations, a visual distractor was shown as an annulus around the fixation position, 100 ms after the fixation onset. The saccadic context of a fixation was classified by its preceding and following saccade amplitudes with the cut-off criterion set to 4° of visual angle. The prolongation of fixation duration induced by distractors was largest for fixations preceded and followed by short saccades. EFRP data revealed a difference in distractor-related P2 amplitude between the saccadic context conditions, following the same trend as in eye movements. Furthermore, influences of the following saccade amplitude on the latency of the saccadic inhibition and on the N1 amplitude were found. The EFRP results cannot be explained by the influence of saccades per se since this bias was removed by subtracting the baseline from the distractor EFRP. Rather, the data suggest that saccadic context indicates differences in how information is processed within single visual fixations.

Lambda response by orientation of striped patterns

A lambda response is an averaged occipital EEG potential associated with the offsets of saccadic eye movements. In the present experiment, two participants were asked to make horizontal saccades across horizontal or vertical white-black stripes. Analysis of orientations of striped patterns showed horizontal saccades across vertical stripes produced larger amplitudes of the lambda response than did horizontal saccades across horizontal stripes. Therefore, when the lambda response is used as the index of mental workload, it is necessary to take notice of the orientation of stimulus pattern.

Integrated effect of stimulation at fixation points on EFRP (eye-fixation related brain potentials)

The purpose of this study was to investigate the integrated effect of stimulation at the fixation points just before and just after saccadic eye-movement (saccade) on eye-fixation related brain potentials (EFRP: P75 and N105). Checkerboard patterns were used as stimuli. In Experiment 1, changes in check sizes between two fixation points enhanced the amplitude of P75, while changes in the phases of patterns between the two points did not affect EFRP. This result showed that EFRP was affected by two fixation points, and that changes in the retinal image between the two points did not necessarily affect EFRP. In Experiment 2, the relationship between EFRP and check size was investigated in detail. A second order relationship between logarithm of check size and the latency of P75, and a linear relationship between logarithm of check size and the amplitude of N105 were found. The effect of check size on the amplitude of P75 which might explain the increased amplitude of P75 observed in Experiment 1 did not appear. These results suggest that EFRP might reflect relative higher processing than peripheral stimulation at one fixation point.

Brain potentials associated with eye fixations during visual tasks under different lighting systems

The variations of eye fixation related potentials (EFRPs) were examined in two tasks under three lighting conditions for assessment of lighting environments. Sixteen subjects participated in two tasks; a difficult and an easy reading task under three lighting conditions: Spot light (S), General light (G) and Mixed light (M). EEG (Oz) and EOG were recorded. EEG epochs time-locked to onset of eye fixations were collected at random and averaged separately in two arrays to obtain a pair of EFRPs. Two wave forms under the S were similar, although those under the G showed the disparity, the largest disparity being in the easy task under the G. Under the S, wave forms of EFRPs were stable in the difficult task. The amplitude changed with the task load. The results suggested that EFRPs might be an index of the work load under lighting conditions.

Saccade onset and offset lambda waves: relation to pattern movement visually evoked potentials

The lambda (lambda) wave is an occipital EEG potential which occurs when saccadic eye movements are made against an illuminated contrast background. There is some disagreement concerning the presence of sub-components to the lambda-wave, and its relationship to visually evoked potentials. In the present study, lambda-waves were recorded with saccades of different durations (30-110 ms) and compared to VEPs associated with pattern movements of similar durations and velocity. It was found that the lambda-wave consisted of a saccade onset component with positive sub-components at 59 and 100 ms after saccade onset, and a saccade offset component with a positive potential at 74 ms after saccade offset. With small saccades of 30 ms duration or less, these components superimposed to form a single lambda-wave. In the case of pattern movement VEPs, a movement onset component of latency 110 ms following movement onset, and a movement offset component at 89 ms after movement offset, were identified. The similar behaviour of the lambda-wave and VEP under these conditions supports the view that the lambda-wave is a visually evoked potential resulting from movement of the visual field across the retina during a saccadic eye movement.

Evoked potentials during REM sleep reflect dreaming

Polygraphic recordings were collected for 11 normal subjects during sleep and wakefulness in order to investigate characteristics of the rapid eye movement (REM) associated potentials. EEGs were averaged using 5 different triggering points: (1) saccade onset under the normal ambient illumination, (2) saccade onset in the total darkness, (3) onset of REMs during REM sleep, (4) flash during REM sleep, and (5) flash during stage 2 sleep. In the central area, positive potentials appeared with waking saccades under the normal ambient illumination (P240L) and REMs (P185R). The latency of P185R associated with REMs was significantly shorter than that of P240L associated with waking saccades. These findings suggest that P185R is evoked by PGO waves occurring just before the REM. A small positive potential appeared in the occipital area with waking saccade under the normal ambient illumination (P260L) and REMs in the total darkness (P250R). Conspicuous absence of these waves for waking saccades in the total darkness suggests that P250R accompanied with REMs reflects activities involved with the cognitive processes occurring when a subject scans a dream image during REM sleep.

Phasic EEG activities associated with rapid eye movements during REM sleep in man

Human cortical potentials associated with rapid eye movements (REMs) during REM sleep were recorded in 6 normal young adults by using the averaging technique. The potentials were compared with those recorded during voluntary saccades in the waking state in a dark or an illuminated room. A positive potential of relatively long duration with peak latency of about 200 msec and a wide spread bilateral distribution and of highest amplitude in the mid-parietal area appeared only during REM sleep. It is suggested that this positive potential is phasic EEG activity appearing in association with REMs during REM sleep in man. Comparison of this positive potential with P300s induced by task-irrelevant visual stimuli disclosed that they resemble each other. The possibility was discussed that this positive potential occurring in association with REMs during REM sleep is indeed a P300 appearing in response to visual images in dream.

On the origin of the presaccadic spike potential

The spike potential (SP) accompanying the onset of saccadic eye movements has been reported to originate near the orbital region. Its dependence on saccade size, however, does not correlate with the behaviour of any of the possible sources of the potential available in the orbital region. The exact size dependence cannot be studied from results obtained with classical ENG methods to detect the saccadic onset and furthermore without removal of the artefact in the signal caused by the corneo-retinal potential. We recorded the SP by monitoring the eye movements with an infrared scleral reflection method (IRIS), and carefully studied the SP amplitude as a function of saccade size, and revealed a more realistic function. Furthermore, removal of the artefact of the corneo-retinal potential revealed a biphasic wave shape of the SP instead of the usually observed monophasic peak. These results support the hypothesis that this electrical activity accompanying the onset of saccadic eye movements originates in the oculomotor neurones innervating the ocular muscle units.

Are eye movement evoked potentials different from pattern reversal evoked potentials?

The averaged lambda wave elicited by saccadic eye movements across a checkerboard pattern has been reported to differ from checkerboard reversal evoked responses, even when the electroencephalographic responses were corrected for the artefact caused by the movement of the dipole moment of the eye itself. Because of these differences it was suggested that the recording of the lambda wave might provide extra information in pathological circumstances. We performed experiments in which the parameters of the pattern (high contrast checkerboard pattern, 20' checks, large field 72 x 72 degrees) shift across the retina were carefully adjusted. For instance, eye movements were made across an integer and odd number of checks in order to mimic the pattern reversal. Furthermore, the timing of the pattern movement in the pattern reversal condition was adjusted so as to simulate the saccadic eye movement. The results seem to suggest that the reported dissimilarities between pattern reversal and eye movement evoked responses can be accounted for by the small differences of the retinal shift in the two conditions.

Electrophysiological evidence for dreaming: human cerebral potentials associated with rapid eye movement during REM sleep

In order to examine the relationship between rapid eye movement (REM) during REM sleep and dreaming, scalp EEGs (Fz, Cz, Pz and Oz) of 4 normal human subjects, time-locked to REM onset, to saccade onset toward fixation targets and to saccade onset in total darkness, were averaged. The results include the following: Three positive potentials were associated with REM: a sharp potential in the parieto-occipital area just before REM onset; a large, slow potential in the vertex area 140-180 msec after REM onset; and a potential in the occipital area 210-280 msec after REM onset. Three positive potentials, one being the so-called EM-antecedent potential and the others being the lambda response, were associated with the waking saccades toward targets: a sharp potential in the parieto-occipital area just before the saccade onset and two potentials in the occipital area with latencies of 140-150 and 260-310 msec from the saccade onset. Only the EM-antecedent potential appearing just before saccade onset was found in association with saccades in total darkness. The similarities between the 3 positive potentials during REM sleep and the lambda response during wakefulness, and the relationship between those potentials and dreaming, are discussed in terms of the neural processes occurring during REM sleep.

A comparison of saccade evoked potentials recorded during reading and tracking tasks

The influence of visual processing demands on saccade-triggered evoked potentials was investigated at P3, P4 and Oz recording sites during reading and tracking tasks. To maximize the physical similarities between tasks, subjects tracked a series of lights that flashed in a stereotypic reading pattern behind a page of text; eye movements recorded during reading initiated the light sequence. In the first experiment, a significant decrease observed in the latency of the major positive peak recorded from Oz during tracking was attributed to the smaller amplitude of tracking, relative to reading, saccades. To confirm this interpretation, the experiment was repeated with modification to the light display. As anticipated, equating saccade amplitudes across tasks eliminated waveform differences in the second experiment. Although peak latencies and amplitudes were not influenced reliably by visual processing demands, tracking potentials exhibited a negative DC shift relative to reading waveforms that was significant at 174 msec at the Oz site. These data suggest that the saccade-triggered evoked potential components generally are insensitive to task differences within the visual modality when visual configuration and eye movement parameters are controlled.

Electrocortical signs of word categorization in saccade-related brain potentials and visual evoked potentials

Visual evoked potentials (VEPs) elicited by foveal presentation of words were compared to brain potentials evoked by the same words in a condition where subjects had to make a saccadic eye movement in order to perceive the words (saccade-related brain potentials, SRPs). Subjects had to categorize the words responding with a button press to stimuli belonging to the target (infrequent, P = 0.2) category. The VEP and SRP waveforms showed divergences in the early (up to 250 ms) components, but a marked similarity between the late components. Principal Component Analysis also revealed the same relationship between the two types of brain responses. Peak latency of the late SRP components measured from saccade offset showed an apparent processing advantage over the corresponding late components of VEPs. The N3 component, indexing semantic processing of visual patterns, peaked between 310 and 375 ms in the SRPs, while in the VEPs it appeared between 410 and 470 ms. The P4 component, associated with final stimulus evaluation, showed a similar latency benefit in favour of SRPs (420-500 ms vs 530-590 ms in VEPs). The mean reaction time was 74 ms shorter in the eye movement condition (measured from saccade offset) than in the VEP condition (703 vs 777 ms). The question of what kind of processes may contribute to the differences in mean RTs and to the latencies of the late components between the two conditions are discussed. We suggest that the late components (P3, N3 and P4) of the VEP and the SRP, respectively, index identical brain processes.

Saccade-related brain potentials in semantic categorization tasks

Saccade-related brain potentials (SRPs) were recorded in word categorization tasks in which subjects had to perform a saccade in order to perceive the stimulus. For all three conditions representing different degrees of complexity of semantic categorization, the stimuli belonged to one of two categories which appeared with the respective probabilities of either 0.20 or 0.80. The late positivity (P4) of the SRPs to infrequent stimuli appeared systematically later as the complexity of stimulus evaluation increased: The easiest categorization was accompanied by a P4 at 400 msec, in the more complex condition it peaked at 600 msec, and in the most difficult semantic categorization the P4 peaked even later, at 680 msec. This shift in peak latency with increasing complexity of categorization is in agreement with the results for traditional ERPs (e.g. Kutas and Donchin, 1978). The possible overlap of the late components was investigated by applying Principal Component-Varimax Analysis to the SRPs.

Late components of saccade-related brain potentials in guessing tasks

The late positive components of lambda responses were studied in a guessing task modified in such a way that subjects had to perform a saccadic eye movement in order to perceive task-relevant information. Responses from 6 scalp areas were investigated in 9 subjects and in 2 conditions: control and guessing. In both conditions subjects performed two consecutive saccades in a given trial: the first to the middle light, the second to the target area. In the control condition the subjects knew beforehand what the 'target' would be. In the guessing condition they had to make a guess before each trial as to which of the three target stimuli would appear. The target stimuli occurred with unequal probability and were presented in a randomized sequence. Unlike the control condition, the guessing task led to the appearance of a late positive component in the lambda response. Similar to traditional ERP findings, this late positivity showed an amplitude maximum at the parietal area and a peak latency at 375 msec from saccade onset. Furthermore, Principal Component-Varimax Analysis (PCVA) of the lambda responses revealed a first factor giving the strongest loading in the latency range of the P300, and a second factor which was identified as the slow wave. These factors are quite similar to the factors found in the PCVA of ERPs. Our results suggest that the late components of lambda responses reflect the effects of information processing in cognitive tasks similarly to the way the late positive components of ERPs do.

Human cerebral potentials associated with REM sleep rapid eye movements: links to PGO waves and waking potentials

Eye movement triggered averaging and topographic display techniques indicate the presence of parieto-occipital potentials that precede the rapid eye movements of human REM sleep. Since these potentials have strong similarities with PGO waves in animals, including lateralization according to eye movement (EM) direction, and with waking EM-antecedent potentials in man, this suggests that PGO-like activity both exists in man, and may be functionally related to EM-antecedent potentials in waking. The ability to detect such central potentials opens the possibility of studying REM sleep central physiological structure in a variety of normal and pathological conditions in humans.

Visual signal detection and lambda responses

On the basis of the result in earlier studies that the lambda response was associated with offset of the saccade, e.g., onset of the fixation pause, the relationship between the lambda response and the visual information processing load was examined. Eleven subjects were presented simultaneously random patterns including visual signals on Goldman perimeter and tone pips including auditory signals at random. In the visual task, the subject was instructed to search on the random pattern and detect the visual signals. In the auditory task, he was instructed to listen to tone pips without ceasing eye movements and detect the tone signals. EEGs from Oz and Cz, and the vertical and the horizontal EOGs were recorded. The vector EOG was calculated from both EOGs for detecting the absolute value of the saccade size. The occipital EEGs time-locked to offset of saccades were averaged 100 times to obtain lambda responses. The vertex EEGs time-locked to tone signals were averaged 100 times to obtain vertex potentials to auditory stimuli (AEP). Amplitudes of the lambda response (NOP100 and P100-N180) in the visual task were significantly higher than in the auditory task. Amplitudes of AEP (N1 and N1-P2) in the auditory task were enhanced significantly more than in the visual task. The results showed that the lambda response varied with information-processing load. The lambda response can be applicable as an index of visual information processing in eye movement situations.