Dissociating parafoveal preview benefit and parafovea-on-fovea effects during reading: A combined eye tracking and EEG study

Direct lexical control of eye movements in Chinese reading: Evidence from the co-registration of EEG and eye tracking

The direct-lexical-control hypothesis stipulates that some aspect of a word's processing determines the duration of the fixation on that word and/or the next. Although the direct lexical control is incorporated into most current models of eye-movement control in reading, the precise implementation varies and the assumptions of the hypothesis may not be feasible given that lexical processing must occur rapidly enough to influence fixation durations. Conclusive empirical evidence supporting this hypothesis is therefore lacking. In this article, we report the results of an eye-tracking experiment using the boundary paradigm in which native speakers of Chinese read sentences in which target words were either high- or low-frequency and preceded by a valid or invalid preview. Eye movements were co-registered with electroencephalography, allowing standard analyses of eye-movement measures, divergence point analyses of fixation-duration distributions, and fixated-related potentials on the target words. These analyses collectively provide strong behavioral and neural evidence of early lexical processing and thus strong support for the direct-lexical-control hypothesis. We discuss the implications of the findings for our understanding of how the hypothesis might be implemented, the neural systems that support skilled reading, and the nature of eye-movement control in the reading of Chinese versus alphabetic scripts.

Neural correlates for word-frequency effect in Chinese natural reading

Word frequency effect has always been of interest for reading research because of its critical role in exploring mental processing underlying reading behaviors. Access to word frequency information has long been considered an indicator of the beginning of lexical processing and the most sensitive marker for studying when the brain begins to extract semantic information Sereno & Rayner, Brain and Cognition, 42, 78-81, (2000), Trends in Cognitive Sciences, 7, 489-493, (2003). While the word frequency effect has been extensively studied in numerous eye-tracking and traditional EEG research using the RSVP paradigm, there is a lack of corresponding evidence in studies of natural reading. To find the neural correlates of the word frequency effect, we conducted a study of Chinese natural reading using EEG and eye-tracking coregistration to examine the time course of lexical processing. Our results reliably showed that the word frequency effect first appeared in the N200 time window and the bilateral occipitotemporal regions. Additionally, the word frequency effect was reflected in the N400 time window, spreading from the occipital region to the central parietal and frontal regions. Our current study provides the first neural correlates for word-frequency effect in natural Chinese reading so far, shedding new light on understanding lexical processing in natural reading and could serve as an important basis for further reading study when considering neural correlates in a realistic manner.

An early effect of the parafoveal preview on post-saccadic processing of English words

A key aspect of efficient visual processing is to use current and previous information to make predictions about what we will see next. In natural viewing, and when looking at words, there is typically an indication of forthcoming visual information from extrafoveal areas of the visual field before we make an eye movement to an object or word of interest. This "preview effect" has been studied for many years in the word reading literature and, more recently, in object perception. Here, we integrated methods from word recognition and object perception to investigate the timing of the preview on neural measures of word recognition. Through a combined use of EEG and eye-tracking, a group of multilingual participants took part in a gaze-contingent, single-shot saccade experiment in which words appeared in their parafoveal visual field. In valid preview trials, the same word was presented during the preview and after the saccade, while in the invalid condition, the saccade target was a number string that turned into a word during the saccade. As hypothesized, the valid preview greatly reduced the fixation-related evoked response. Interestingly, multivariate decoding analyses revealed much earlier preview effects than previously reported for words, and individual decoding performance correlated with participant reading scores. These results demonstrate that a parafoveal preview can influence relatively early aspects of post-saccadic word processing and help to resolve some discrepancies between the word and object literatures.

Neural mechanism underlying preview effects and masked priming effects in visual word processing

Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between the two paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. To facilitate direct comparisons, we used a simplified version of the boundary paradigm - the single word boundary paradigm. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger with the single word boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms, showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a single saccade, although such effects appear to be stronger in the latter case.

(The limits of) eye-tracking with iPads

Applications for eye-tracking-particularly in the clinic-are limited by a reliance on dedicated hardware. Here we compare eye-tracking implemented on an Apple iPad Pro 11" (third generation)-using the device's infrared head-tracking and front-facing camera-with a Tobii 4c infrared eye-tracker. We estimated gaze location using both systems while 28 observers performed a variety of tasks. For estimating fixation, gaze position estimates from the iPad were less accurate and precise than the Tobii (mean absolute error of 3.2° ± 2.0° compared with 0.75° ± 0.43°), but fixation stability estimates were correlated across devices (r = 0.44, p < 0.05). For tasks eliciting saccades >1.5°, estimated saccade counts (r = 0.4-0.73, all p < 0.05) were moderately correlated across devices. For tasks eliciting saccades >8° we observed moderate correlations in estimated saccade speed and amplitude (r = 0.4-0.53, all p < 0.05). We did, however, note considerable variation in the vertical component of estimated smooth pursuit speed from the iPad and a catastrophic failure of tracking on the iPad in 5% to 20% of observers (depending on the test). Our findings sound a note of caution to researchers seeking to use iPads for eye-tracking and emphasize the need to properly examine their eye-tracking data to remove artifacts and outliers.

Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Impact of observational and direct learning on fear conditioning generalization in humans

Humans gain knowledge about threats not only from their own experiences but also from observing others' behavior. A neutral stimulus is associated with a threat stimulus for several times and the neutral stimulus will evoke fear responses, which is known as fear conditioning. When encountering a new event that is similar to one previously associated with a threat, one may feel afraid and produce fear responses. This is called fear generalization. Previous studies have mostly focused on fear conditioning and generalization based on direct learning, but few have explored how observational fear learning affects fear conditioning and generalization. To the best of our knowledge, no previous study has focused on the neural correlations of fear conditioning and generalization based on observational learning. In the present study, 58 participants performed a differential conditioning paradigm in which they learned the associations between neutral cues (i.e., geometric figures) and threat stimuli (i.e., electric shock). The learning occurred on their own (i.e., direct learning) and by observing other participant's responses (i.e., observational learning); the study used a within-subjects design. After each learning condition, a fear generalization paradigm was conducted by each participant independently while their behavioral responses (i.e., expectation of a shock) and electroencephalography (EEG) recordings or responses were recorded. The shock expectancy ratings showed that observational learning, compared to direct learning, reduced the differentiation between the conditioned threatening stimuli and safety stimuli and the increased shock expectancy to the generalization stimuli. The EEG indicated that in fear learning, threatening conditioned stimuli in observational and direct learning increased early discrimination (P1) and late motivated attention (late positive potential [LPP]), compared with safety conditioned stimuli. In fear generalization, early discrimination, late motivated attention, and orienting attention (alpha-event-related desynchronization [alpha-ERD]) to generalization stimuli were reduced in the observational learning condition. These findings suggest that compared to direct learning, observational learning reduces differential fear learning and increases the generalization of fear, and this might be associated with reduced discrimination and attentional function related to generalization stimuli.

The pattern of intra-/inter-hemispheric interactions of left and right hemispheres in visual word processing

This study aimed to investigate the intra-/inter-hemispheric interactions during visual word processing, by manipulating stimulus onset asynchrony (SOA) in a primed-lateralized lexical decision task. To assess intra-/inter-hemispheric priming effects, identical prime-target pairs were presented in the same or opposite unilateral visual fields. The study found that the right visual field advantage (RVFA) was observed when Korean words were presented sequentially within hemispheres, indicating that the inherent characteristics of the two hemispheres, rather than differences in memory or linguistic aspects of lexical processing, contributed to the hemispheric asymmetry. Additionally, intra-hemispheric priming effects were symmetrical in both hemispheres, with similar increases in priming for words and nonwords from SOA 120 ms to SOA 600 ms. Furthermore, inter-hemispheric priming effects were asymmetrical, with stronger priming when stimuli were presented in a sequence of LH→RH than in RH→LH. These findings suggest that the intrinsic differences in lexical processing between the two hemispheres may be related to the asymmetric pattern of hemispheric interactions in visual word processing.

Saccades are locked to the phase of alpha oscillations during natural reading

We saccade 3 to 5 times per second when reading. However, little is known about the neuronal mechanisms coordinating the oculomotor and visual systems during such rapid processing. Here, we ask if brain oscillations play a role in the temporal coordination of the visuomotor integration. We simultaneously acquired MEG and eye-tracking data while participants read sentences silently. Every sentence was embedded with a target word of either high or low lexical frequency. Our key finding demonstrated that saccade onsets were locked to the phase of alpha oscillations (8 to 13 Hz), and in particular, for saccades towards low frequency words. Source modelling demonstrated that the alpha oscillations to which the saccades were locked, were generated in the right-visual motor cortex (BA 7). Our findings suggest that the alpha oscillations serve to time the processing between the oculomotor and visual systems during natural reading, and that this coordination becomes more pronounced for demanding words.

Examining the extraction of parafoveal semantic information in Tibetan

This study conducted two experiments to investigate the extraction of semantic preview information from the parafovea in Tibetan reading. In Experiment 1, a single-factor (preview type: identical vs. semantically related vs. unrelated) within-subject experimental design was used to investigate whether there is a parafoveal semantic preview effect (SPE) in Tibetan reading. Experiment 2 used a 2 (contextual constraint: high vs. low) × 3 (preview type: identical vs. semantically related vs. unrelated) within-subject experimental design to investigate the influence of contextual constraint on the parafoveal semantic preview effect in Tibetan reading. Supporting the E-Z reader model, the experimental results showed that in Tibetan reading, readers could not obtain semantic preview information from the parafovea, and contextual constraint did not influence this process. However, comparing high- and low-constrained contexts, the latter might be more conducive to extracting semantic preview information from the parafovea.

Lateralization of orthographic processing in fixed-gaze and natural reading conditions

Lateralized processing of orthographic information is a hallmark of proficient reading. However, how this finding obtained for fixed-gaze processing of orthographic stimuli translates to ecologically valid reading conditions remained to be clarified. To address this shortcoming, here we assessed the lateralization of early orthographic processing in fixed-gaze and natural reading conditions using concurrent eye-tracking and EEG data recorded from young adults without reading difficulties. Sensor-space analyses confirmed the well-known left-lateralized negative-going deflection of fixed-gaze EEG activity throughout the period of early orthographic processing. At the same time, fixation-related EEG activity exhibited left-lateralized followed by right-lateralized processing of text stimuli during natural reading. A strong positive relationship was found between the early leftward lateralization in fixed-gaze and natural reading conditions. Using source-space analyses, early left-lateralized brain activity was obtained in lateraloccipital and posterior ventral occipito-temporal cortices reflecting letter-level processing in both conditions. In addition, in the same time interval, left-lateralized source activity was found also in premotor and parietal brain regions during natural reading. While brain activity remained left-lateralized in later stages representing word-level processing in posterior and middle ventral temporal regions in the fixed-gaze condition, fixation-related source activity became stronger in the right hemisphere in medial and more anterior ventral temporal brain regions indicating higher-level processing of orthographic information. Although our results show a strong positive relationship between the lateralization of letter-level processing in the two reading modes and suggest lateralized brain activity as a general marker for processing of orthographic information, they also clearly indicate the need for reading research in ecologically valid conditions to identify the neural basis of visuospatial attentional, oculomotor and higher-level processes specific to natural reading.

Face familiarity revealed by fixational eye movements and fixation-related potentials in free viewing

Event-related potentials (ERPs) and the oculomotor inhibition (OMI) in response to visual transients are known to be sensitive to stimulus properties, attention, and expectation. We have recently found that the OMI is also sensitive to face familiarity. In natural vision, stimulation of the visual cortex is generated primarily by saccades, and it has been recently suggested that fixation-related potentials (FRPs) share similar components with the ERPs. Here, we investigated whether FRPs and microsaccade inhibition (OMI) in free viewing are sensitive to face familiarity. Observers freely watched a slideshow of seven unfamiliar and one familiar facial images presented randomly for 4-s periods, with multiple images per identity. We measured the occipital fixation-related N1 relative to the P1 magnitude as well as the associated fixation-triggered OMI. We found that the average N1-P1 was significantly smaller and the OMI was shorter for the familiar face, compared with any of the seven unfamiliar faces. Moreover, the P1 was suppressed across saccades for the familiar but not for the unfamiliar faces. Our results highlight the sensitivity of the occipital FRPs to stimulus properties such as face familiarity and advance our understanding of the integration process across successive saccades in natural vision.

Fixation-related fMRI analysis reveals the neural basis of parafoveal processing in self-paced reading of Chinese words

While parafoveal word processing plays an important role in natural reading, the underlying neural mechanism remains unclear. The present study investigated the neural basis of parafoveal processing during Chinese word reading with the co-registration of eye-tracking and functional magnetic resonance imaging (fMRI) using fixation-related fMRI analysis. In the gaze-contingent boundary paradigm, preview conditions (words that are identical, orthographically similar, and unrelated to target words), pre-target word frequency and target word frequency were manipulated. When fixating the pre-target word, the identical preview condition elicited lower brain activation in the left fusiform gyrus relative to unrelated and orthographically similar preview conditions and there were significant interactions of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus, left fusiform gyrus and supplementary motor area. When fixating the target word, there was a significant main effect of preview condition on brain activation of the right fusiform gyrus and a significant interaction of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus. These results suggest that fixation-related brain activation provides immediate measures and new perspectives to understand the mechanism of parafoveal processing in self-paced reading.

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Preview frequency effects in reading: evidence from Chinese

Studies about sentence reading have shown that visual and lexical information beyond the currently fixated word can be integrated across fixations. The gaze-contingent boundary paradigm has been used widely to explore the extent to which parafoveal information can be processed before a word is fixated on. However, a critical review of the current literature suggests that unrelated mask previews are an unlikely baseline control with zero lexical activation, blurring the nature of experimental effects observed in the paradigm. The present study, therefore, aimed at shedding light on the effect of parafoveal mask properties through a manipulation of preview word frequency. Low-frequency preview words that are unrelated to target words elicited a larger interference than high-frequency preview words. We discuss implications of the preview frequency effect for computational models of eye-movement control in reading.

Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements

Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing.

Neural evidence for lexical parafoveal processing

In spite of the reduced visual acuity, parafoveal information plays an important role in natural reading. However, competing models on reading disagree on whether words are previewed parafoveally at the lexical level. We find neural evidence for lexical parafoveal processing by combining a rapid invisible frequency tagging (RIFT) approach with magnetoencephalography (MEG) and eye-tracking. In a silent reading task, target words are tagged (flickered) subliminally at 60 Hz. The tagging responses measured when fixating on the pre-target word reflect parafoveal processing of the target word. We observe stronger tagging responses during pre-target fixations when followed by low compared with high lexical frequency targets. Moreover, this lexical parafoveal processing is associated with individual reading speed. Our findings suggest that reading unfolds in the fovea and parafovea simultaneously to support fluent reading.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Fixation-related Brain Potentials during Semantic Integration of Object–Scene Information

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t − 1). Object–scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object–scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t − 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object–scene integration during natural viewing.

Parafoveal-on-foveal repetition effects in sentence reading: A co-registered eye-tracking and electroencephalogram study

When reading, can the next word in the sentence (word n + 1) influence how you read the word you are currently looking at (word n)? Serial models of sentence reading state that this generally should not be the case, whereas parallel models predict that this should be the case. Here we focus on perhaps the simplest and the strongest Parafoveal‐on‐Foveal (PoF) manipulation: word n + 1 is either the same as word n or a different word. Participants read sentences for comprehension and when their eyes left word n, the repeated or unrelated word at position n + 1 was swapped for a word that provided a syntactically correct continuation of the sentence. We recorded electroencephalogram and eye‐movements, and time‐locked the analysis of fixation‐related potentials (FRPs) to fixation of word n. We found robust PoF repetition effects on gaze durations on word n, and also on the initial landing position on word n. Most important is that we also observed significant effects in FRPs, reaching significance at 260 ms post‐fixation of word n. Repetition of the target word n at position n + 1 caused a widely distributed reduced negativity in the FRPs. Given the timing of this effect, we argue that it is driven by orthographic processing of word n + 1, while readers were still looking at word n, plus the spatial integration of orthographic information extracted from these two words in parallel.

A hotly debated issue in reading research concerns the serial versus parallel nature of word identification processes. Evidence in favor of parallel processing has been criticized because it has often been obtained with artificial reading paradigms. Here we show that fixation‐related potentials elicited by a target word embedded in a normal sentence are impacted by the nature of the word immediately to the right of the fixated target word (n), and that this impact became significant 260 ms after fixation of word n. This is in line with parallel orthographic processing across adjacent words during sentence reading.

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.

A co-registration investigation of inter-word spacing and parafoveal preview: Eye movements and fixation-related potentials

Participants' eye movements (EMs) and EEG signal were simultaneously recorded to examine foveal and parafoveal processing during sentence reading. All the words in the sentence were manipulated for inter-word spacing (intact spaces vs. spaces replaced by a random letter) and parafoveal preview (identical preview vs. random letter string preview). We observed disruption for unspaced text and invalid preview conditions in both EMs and fixation-related potentials (FRPs). Unspaced and invalid preview conditions received longer reading times than spaced and valid preview conditions. In addition, the FRP data showed that unspaced previews disrupted reading in earlier time windows of analysis, compared to string preview conditions. Moreover, the effect of parafoveal preview was greater for spaced relative to unspaced conditions, in both EMs and FRPs. These findings replicate well-established preview effects, provide novel insight into the neural correlates of reading with and without inter-word spacing and suggest that spatial selection precedes lexical processing.

Event-related brain potentials reveal how multiple aspects of semantic processing unfold across parafoveal and foveal vision during sentence reading

Recent event-related brain potential (ERP) experiments have demonstrated parafoveal N400 expectancy and congruity effects, showing that semantic information can be accessed from words in parafoveal vision (a conclusion also supported by some eye-tracking work). At the same time, it is unclear how higher-order integrative aspects of language comprehension unfold across the visual field during reading. In the current study, we recorded ERPs in a parafoveal flanker paradigm, while readers were instructed to read passively for comprehension or to judge the plausibility of sentences in which target words varied in their semantic expectancy and congruity. We directly replicated prior work showing graded N400 effects for parafoveal viewing, which are then not duplicated when the target words are processed foveally. Critically, although N400 effects were not modulated by task goals, a posteriorly distributed late positive component thought to reflect semantic integration processes was observed to semantic incongruities only in the plausibility judgment task. However, this effect was observed at a considerable delay, appearing only after words had moved into foveal vision. Our findings thus suggest that semantic access can be initiated in parafoveal vision, whereas central foveal vision may be necessary to enact higher-order (and task-dependent) integrative processing.

The peripheral preview effect with faces: Combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing

The world appears stable despite saccadic eye-movements. One possible explanation for this phenomenon is that the visual system predicts upcoming input across saccadic eye-movements based on peripheral preview of the saccadic target. We tested this idea using concurrent electroencephalography (EEG) and eye-tracking. Participants made cued saccades to peripheral upright or inverted face stimuli that changed orientation (invalid preview) or maintained orientation (valid preview) while the saccade was completed. Experiment 1 demonstrated better discrimination performance and a reduced fixation-locked N170 component (fN170) with valid than with invalid preview, demonstrating integration of pre- and post-saccadic information. Moreover, the early fixation-related potentials (FRP) showed a preview face inversion effect suggesting that some pre-saccadic input was represented in the brain until around 170 ms post fixation-onset. Experiment 2 replicated Experiment 1 and manipulated the proportion of valid and invalid trials to test whether the preview effect reflects context-based prediction across trials. A whole-scalp Bayes factor analysis showed that this manipulation did not alter the fN170 preview effect but did influence the face inversion effect before the saccade. The pre-saccadic inversion effect declined earlier in the mostly invalid block than in the mostly valid block, which is consistent with the notion of pre-saccadic expectations. In addition, in both studies, we found strong evidence for an interaction between the pre-saccadic preview stimulus and the post-saccadic target as early as 50 ms (Experiment 2) or 90 ms (Experiment 1) into the new fixation. These findings suggest that visual stability may involve three temporal stages: prediction about the saccadic target, integration of pre-saccadic and post-saccadic information at around 50-90 ms post fixation onset, and post-saccadic facilitation of rapid categorization.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

An electrophysiological investigation of orthographic spatial integration in reading

During reading, word recognition speed is influenced by the amount of orthographic overlap with surrounding words. The nature of this phenomenon is not understood: some theories attribute it to low-level visual operations (i.e., parafoveal feature detectors influencing foveal letter detectors), whereas other theories assume that orthographic processing (i.e., letter position coding and word activation) occurs across multiple words in parallel. To arbitrate between these theories, we used electroencephalography to reveal the time course of orthographic spatial integration in a lexical decision task. Foveal target words were flanked on each side by parafoveal words, manipulated across three conditions: repetition flankers (e.g. rock rock rock), unrelated flankers (step rock step) and a no-flanker condition. Linear mixed-effect models were constructed to analyze EEG data on a trial-by-trial basis. Word recognition was worse in the unrelated flanker condition than in the repetition and no-flanker conditions. This behavioral pattern was accompanied by increased negativity in the N250 and N400 windows, associated with the activation of sub-lexical and lexico-semantic representations, respectively. Crucially, the absence of effects prior to 200 ms post-stimulus onset provides evidence against the involvement of low-level visual processes. We conclude that orthographic spatial integration is driven by parallel processing of multiple words, which leads to the activation of a larger set of sub-lexical nodes and more difficult processing at the lexical level when those words are orthographically unrelated.

•

Words are recognized faster when they share many letters with surrounding words.

•

EEG measurements suggest that this effect unfolds 200 ms after stimulus onset.

•

The locus of effects contradicts the involvement of low-level visual processing.

•

Instead, the phenomenon appears to be driven by parallel orthographic processing.

Parafoveal previews and lexical frequency in natural reading: Evidence from eye movements and fixation-related potentials

Participants' eye movements and electroencephalogram (EEG) signal were recorded as they read sentences displayed according to the gaze-contingent boundary paradigm. Two target words in each sentence were manipulated for lexical frequency (high vs. low frequency) and parafoveal preview of each target word (identical vs. string of random letters vs. string of Xs). Eye movement data revealed visual parafoveal-on-foveal (PoF) effects, as well as foveal visual and orthographic preview effects and word frequency effects. Fixation-related potentials (FRPs) showed visual and orthographic PoF effects as well as foveal visual and orthographic preview effects. Our results replicated the early preview positivity effect (Dimigen, Kliegl, & Sommer, 2012) in the X-string preview condition, and revealed different neural correlates associated with a preview comprised of a string of random letters relative to a string of Xs. The former effects seem likely to reflect difficulty associated with the integration of parafoveal and foveal information, as well as feature overlap, while the latter reflect inhibition, and potentially disruption, to processing underlying reading. Interestingly, and consistent with Kretzschmar, Schlesewsky, and Staub (2015), no frequency effect was reflected in the FRP measures. The findings provide insight into the neural correlates of parafoveal processing and written word recognition in reading and demonstrate the value of utilizing ecologically valid paradigms to study well established phenomena that occur as text is read naturally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Stable preview difficulty effects in reading with an improved variant of the boundary paradigm

Using gaze-contingent display changes in the boundary paradigm during sentence reading, it has recently been shown that parafoveal word-processing difficulties affect fixations on words to the right of the boundary. Current interpretations of this post-boundary preview difficulty effect range from delayed parafoveal-on-foveal effects in parallel word-processing models to forced fixations in serial word-processing models. However, these findings are based on an experimental design that, while allowing to isolate preview difficulty effects, might have established a bias with respect to asymmetries in parafoveal preview benefit for high-frequent and low-frequent target words. Here, we present a revision of this paradigm varying the preview's lexical frequency and keeping the target word constant. We found substantial effects of the preview difficulty in fixation durations after the boundary confirming that preview processing affects the oculomotor decisions not only via trans-saccadic integration of preview and target word information. An additional time-course analysis showed that the preview difficulty effect was significant across the full fixation duration distribution on the target word without any evidence on the pretarget word before the boundary. We discuss implications of the accumulating evidence of post-boundary preview difficulty effects for models of eye movement control during reading.

Decoding Trans-Saccadic Memory

We examine whether peripheral information at a planned saccade target affects immediate postsaccadic processing at the fovea on saccade landing. Current neuroimaging research suggests that presaccadic stimulation has a late effect on postsaccadic processing, in contrast to the early effect seen in behavioral studies. Human participants (both male and female) were instructed to saccade toward a face or a house that, on different trials, remained the same, changed, or disappeared during the saccade. We used a multivariate pattern analysis of electroencephalography data to decode face versus house processing directly after the saccade. The classifier was trained on separate trials without a saccade, where a house or face was presented at the fovea. When the saccade target remained the same across the saccade, we could reliably decode the target 123 ms after saccade offset. In contrast, when the target was changed during the saccade, the new target was decoded at a later time-point, 151 ms after saccade offset. The "same" condition advantage suggests that congruent presaccadic information facilitates processing of the postsaccadic stimulus compared with incongruent information. Finally, the saccade target could be decoded above chance even when it had been removed during the saccade, albeit with a slower time course (162 ms) and poorer signal strength. These findings indicate that information about the (peripheral) presaccadic stimulus is transferred across the saccade so that it becomes quickly available and influences processing at its expected new retinal position (the fovea).SIGNIFICANCE STATEMENT Here we provide neural evidence for early information transfer across saccades. Specifically, we examined the effect of presaccadic sensory information on the initial neuronal processing of a postsaccadic stimuli. Using electroencephalography and multivariate pattern analysis, we found the following: (1) that the identity of the presaccadic stimulus modulated the postsaccadic latency of stimulus relevant information; and (2) that a saccadic neural marker for a saccade target stimulus could be detected even when the stimulus had been removed during saccade. These results demonstrate that information about the peripheral presaccadic stimulus was transferred across the saccade and influenced processing at a new retinal position (the fovea) directly after the saccade landed.