Parafoveal previews and lexical frequency in natural reading: Evidence from eye movements and fixation-related potentials

Understanding words in context: A naturalistic EEG study of children's lexical processing

When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.

Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Establishing gaze markers of perceptual load during multi-target visual search

Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.

General cognitive processing for orthographic discrepancy engages foveal attention during sentence comprehension

Words are processed in the parafovea and fovea in succession during natural reading, but the classic rapid serial visual presentation (RSVP) paradigm presents words only in the fovea. Unlike the RSVP paradigm, the RSVP with flanker (RSVP-flanker) paradigm is similar to natural text reading. Previous studies using the RSVP-flanker paradigm have suggested that high-level semantic/syntactic integration engages foveal fixation after parafoveal semantic access. However, it is less clear how general cognitive processing, such as discrepancy monitoring and error correction, unfolds across the parafoveal and foveal visual fields. In the current study, Chinese sentences were presented with the RSVP-flanker paradigm. Critical words were manipulated so that they were expected (EXP), semantically violated (VIO), or orthographically similar (ORT). Because of the nuanced differences between the EXP and ORT conditions, the processing of ORT critical words required more general cognitive processing, such as discrepancy monitoring and error correction. In contrast, the processing of VIO words may require more semantic integration. The ERP results showed more positive foveal late positive component (LPC) in ORT versus VIO. Moreover, there was no parafoveal LPC effect, but a robust foveal LPC effect, in the contrasts of VIO/ORT versus EXP, replicating previous results. Together, the results provide substantial evidence that general cognitive processing for orthographic discrepancy occurs at the foveal perception.

Parallel word reading revealed by fixation-related brain potentials

During reading, the brain is confronted with many relevant objects at once. But does lexical processing occur for multiple words simultaneously? Cognitive science has yet to answer this prominent question. Recently it has been argued that the issue warrants supplementing the field's traditional toolbox (response times, eye-tracking) with neuroscientific techniques (EEG, fMRI). Indeed, according to the OB1-reader model, upcoming words need not impact oculomotor behavior per se, but parallel processing of these words must nonetheless be reflected in neural activity. Here we combined eye-tracking with EEG, time-locking the neural window of interest to the fixation on target words in sentence reading. During these fixations, we manipulated the identity of the subsequent word so that it posed either a syntactically legal or illegal continuation of the sentence. In line with previous research, oculomotor measures were unaffected. Yet, syntax impacted brain potentials as early as 100 ms after the target fixation onset. Given the EEG literature on syntax processing, the presently observed timings suggest parallel word reading. We reckon that parallel word processing typifies reading, and that OB1-reader offers a good platform for theorizing about the reading brain.

Positive emotions and their upregulation increase willingness to consume healthy foods

While highly relevant for everyday life, it is unclear whether experiencing incidental positive or negative emotional states, and active emotion regulation, influence the weighting of perceived taste and health in food choices. In Experiment 1, we examined two emotion regulation strategies, reappraisal and distraction, used to decrease negative emotions. Participants were cued to experience or decrease their emotional response for either neutral or negative incidental emotion-inducing images. They subsequently rated their willingness to consume foods, which varied in their taste and health attributes. Mixed-effects model analysis showed that compared to neutral, negative emotions decreased willingness to consume, regardless of perceived taste and health, but neither emotion regulation strategy had a significant effect. Experiment 2 used images inducing incidental positive emotions in combination with three emotion regulation strategies: reappraisal, distraction, and increasing positive emotions. Experiencing positive emotions generally increased willingness to consume, with stronger effects for tasty and healthy foods. Decreasing positive emotions via reappraisal decreased willingness to consume, particularly for healthy foods. Increasing positive emotion intensity further increased willingness to consume, with stronger effects for healthy foods. The results suggest that experiencing positive emotions increases desire particularly strongly for healthy foods, which can additionally be modulated via emotion regulation. This has important implications for designing health-related interventions targeting mood improvement.

The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.

Dynamic attentional bias for pictorial and textual food cues in the visual search paradigm

Previous studies have found that individuals have an attentional bias for food cues, which may be related to the energy level or the type of stimulus (e.g., pictorial or textual food cues) of the food cues. However, the available evidence is inconsistent, and there is no consensus about how the type of stimulus and food energy modulate food-related attentional bias. Searching for food is one of the most important daily behaviors. In this study, a modified visual search paradigm was used to explore the attentional bias for food cues, and eye movements were recorded. Food cues consisted of both food words and food pictures with different energy levels (i.e., high- and low-calorie foods). The results showed that there was an attentional avoidance in the early stage but a later-stage attentional approach for all food cues in the pictorial condition. This was especially true for high-calorie food pictures. Participants showed a later-stage conflicting attentional bias for foods with different energy levels in the textual condition. They showed an attentional approach to high-calorie food words but an attentional avoidance of low-calorie food words. These data show that food-related attentional bias varied along with different time courses, which was also modulated by the type of stimulus and food energy. These findings regarding dynamic attentional bias could be explained using the Goal Conflict Model of eating behavior.

Saccades are locked to the phase of alpha oscillations during natural reading

We saccade 3 to 5 times per second when reading. However, little is known about the neuronal mechanisms coordinating the oculomotor and visual systems during such rapid processing. Here, we ask if brain oscillations play a role in the temporal coordination of the visuomotor integration. We simultaneously acquired MEG and eye-tracking data while participants read sentences silently. Every sentence was embedded with a target word of either high or low lexical frequency. Our key finding demonstrated that saccade onsets were locked to the phase of alpha oscillations (8 to 13 Hz), and in particular, for saccades towards low frequency words. Source modelling demonstrated that the alpha oscillations to which the saccades were locked, were generated in the right-visual motor cortex (BA 7). Our findings suggest that the alpha oscillations serve to time the processing between the oculomotor and visual systems during natural reading, and that this coordination becomes more pronounced for demanding words.

Emerging Wearable Biosensor Technologies for Stress Monitoring and Their Real-World Applications

Wearable devices are being developed faster and applied more widely. Wearables have been used to monitor movement-related physiological indices, including heartbeat, movement, and other exercise metrics, for health purposes. People are also paying more attention to mental health issues, such as stress management. Wearable devices can be used to monitor emotional status and provide preliminary diagnoses and guided training functions. The nervous system responds to stress, which directly affects eye movements and sweat secretion. Therefore, the changes in brain potential, eye potential, and cortisol content in sweat could be used to interpret emotional changes, fatigue levels, and physiological and psychological stress. To better assess users, stress-sensing devices can be integrated with applications to improve cognitive function, attention, sports performance, learning ability, and stress release. These application-related wearables can be used in medical diagnosis and treatment, such as for attention-deficit hyperactivity disorder (ADHD), traumatic stress syndrome, and insomnia, thus facilitating precision medicine. However, many factors contribute to data errors and incorrect assessments, including the various wearable devices, sensor types, data reception methods, data processing accuracy and algorithms, application reliability and validity, and actual user actions. Therefore, in the future, medical platforms for wearable devices and applications should be developed, and product implementations should be evaluated clinically to confirm product accuracy and perform reliable research.

Fixation-related fMRI analysis reveals the neural basis of parafoveal processing in self-paced reading of Chinese words

While parafoveal word processing plays an important role in natural reading, the underlying neural mechanism remains unclear. The present study investigated the neural basis of parafoveal processing during Chinese word reading with the co-registration of eye-tracking and functional magnetic resonance imaging (fMRI) using fixation-related fMRI analysis. In the gaze-contingent boundary paradigm, preview conditions (words that are identical, orthographically similar, and unrelated to target words), pre-target word frequency and target word frequency were manipulated. When fixating the pre-target word, the identical preview condition elicited lower brain activation in the left fusiform gyrus relative to unrelated and orthographically similar preview conditions and there were significant interactions of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus, left fusiform gyrus and supplementary motor area. When fixating the target word, there was a significant main effect of preview condition on brain activation of the right fusiform gyrus and a significant interaction of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus. These results suggest that fixation-related brain activation provides immediate measures and new perspectives to understand the mechanism of parafoveal processing in self-paced reading.

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Parafoveal processing of repeated words during reading

In an eye-tracking experiment during reading, we examined the repetition effect, whereby words that are repeated in the same paragraph receive shorter fixation durations. Target words that were either high-frequency or low-frequency words and of which the parafoveal preview was either correct or with all letters replaced were embedded three times in the same paragraph. Shorter fixation times and higher skipping rates were observed for high-frequency compared to low-frequency words, words for which the parafoveal preview was correct versus incorrect, and as the word was being repeated more often. An interaction between frequency and repetition indicated that the reduction in fixation times due to repetition was more pronounced for low-frequency words. We also observed influences of word repetition on parafoveal processing, as repeated words were skipped more often. An interaction between parafoveal preview and repetition indicated an absent repetition effect when the preview was incorrect, but this effect was short lived, as it was restricted to the first fixation duration on the target word.

Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements

Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing.

The association of eye movements and performance accuracy in a novel sight-reading task

The present study investigated how eye movements were associated with performance accuracy during sight-reading. Participants performed a complex span task in which sequences of single quarter note symbols that either enabled chunking or did not enable chunking were presented for subsequent serial recall. In between the presentation of each note, participants sight-read a notated melody on an electric piano in the tempo of 70 bpm. All melodies were unique but contained four types of note pairs: eighth-eighth, eighthquarter, quarter-eighth, quarter-quarter. Analyses revealed that reading with fewer fixations was associated with a more accurate note onset. Fewer fixations might be advantageous for sight-reading as fewer saccades have to be planned and less information has to be integrated. Moreover, the quarter-quarter note pair was read with a larger number of fixations and the eighth-quarter note pair was read with a longer gaze duration. This suggests that when rhythm is processed, additional beats might trigger re-fixations and unconventional rhythmical patterns might trigger longer gazes. Neither recall accuracy nor chunking processes were found to explain additional variance in the eye movement data.

Neural evidence for lexical parafoveal processing

In spite of the reduced visual acuity, parafoveal information plays an important role in natural reading. However, competing models on reading disagree on whether words are previewed parafoveally at the lexical level. We find neural evidence for lexical parafoveal processing by combining a rapid invisible frequency tagging (RIFT) approach with magnetoencephalography (MEG) and eye-tracking. In a silent reading task, target words are tagged (flickered) subliminally at 60 Hz. The tagging responses measured when fixating on the pre-target word reflect parafoveal processing of the target word. We observe stronger tagging responses during pre-target fixations when followed by low compared with high lexical frequency targets. Moreover, this lexical parafoveal processing is associated with individual reading speed. Our findings suggest that reading unfolds in the fovea and parafovea simultaneously to support fluent reading.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Individuals with dyslexia use a different visual sampling strategy to read text

Individuals with dyslexia present with reading-related deficits including inaccurate and/or less fluent word recognition and poor decoding abilities. Slow reading speed and worse text comprehension can occur as secondary consequences of these deficits. Reports of visual symptoms such as atypical eye movements during reading gave rise to a search for these deficits' underlying mechanisms. This study sought to replicate established behavioral deficits in reading and cognitive processing speed while investigating their underlying mechanisms in more detail by developing a comprehensive profile of eye movements specific to reading in adult dyslexia. Using a validated standardized reading assessment, our findings confirm a reading speed deficit among adults with dyslexia. We observed different eye movements in readers with dyslexia across numerous eye movement metrics including the duration of a stop (i.e., fixation), the length of jumps (i.e., saccades), and the number of times a reader's eyes expressed a jump atypical for reading. We conclude that individuals with dyslexia visually sample written information in a laborious and more effortful manner that is fundamentally different from those without dyslexia. Our findings suggest a mix of aberrant cognitive linguistic and oculomotor processes being present in adults with dyslexia.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Fixation-related NIRS indexes retinotopic occipital processing of parafoveal preview during natural reading

While word frequency and predictability effects have been examined extensively, any evidence on interactive effects as well as parafoveal influences during whole sentence reading remains inconsistent and elusive. Novel neuroimaging methods utilize eye movement data to account for the hemodynamic responses of very short events such as fixations during natural reading. In this study, we used the rapid sampling frequency of near-infrared spectroscopy (NIRS) to investigate neural responses in the occipital and orbitofrontal cortex to word frequency and predictability. We observed increased activation in the right ventral occipital cortex when the fixated word N was of low frequency, which we attribute to an enhanced cost during saccade planning. Importantly, unpredictable (in contrast to predictable) low frequency words increased the activity in the left dorsal occipital cortex at the fixation of the preceding word N-1, presumably due to an upcoming breach of top-down modulated expectation. Opposite to studies that utilized a serial presentation of words (e.g. Hofmann et al., 2014), we did not find such an interaction in the orbitofrontal cortex, implying that top-down timing of cognitive subprocesses is not required during natural reading. We discuss the implications of an interactive parafoveal-on-foveal effect for current models of eye movements.

Best practices in eye tracking research

This guide describes best practices in using eye tracking technology for research in a variety of disciplines. A basic outline of the anatomy and physiology of the eyes and of eye movements is provided, along with a description of the sorts of research questions eye tracking can address. We then explain how eye tracking technology works and what sorts of data it generates, and provide guidance on how to select and use an eye tracker as well as selecting appropriate eye tracking measures. Challenges to the validity of eye tracking studies are described, along with recommendations for overcoming these challenges. We then outline correct reporting standards for eye tracking studies.

Parafoveal-on-foveal repetition effects in sentence reading: A co-registered eye-tracking and electroencephalogram study

When reading, can the next word in the sentence (word n + 1) influence how you read the word you are currently looking at (word n)? Serial models of sentence reading state that this generally should not be the case, whereas parallel models predict that this should be the case. Here we focus on perhaps the simplest and the strongest Parafoveal‐on‐Foveal (PoF) manipulation: word n + 1 is either the same as word n or a different word. Participants read sentences for comprehension and when their eyes left word n, the repeated or unrelated word at position n + 1 was swapped for a word that provided a syntactically correct continuation of the sentence. We recorded electroencephalogram and eye‐movements, and time‐locked the analysis of fixation‐related potentials (FRPs) to fixation of word n. We found robust PoF repetition effects on gaze durations on word n, and also on the initial landing position on word n. Most important is that we also observed significant effects in FRPs, reaching significance at 260 ms post‐fixation of word n. Repetition of the target word n at position n + 1 caused a widely distributed reduced negativity in the FRPs. Given the timing of this effect, we argue that it is driven by orthographic processing of word n + 1, while readers were still looking at word n, plus the spatial integration of orthographic information extracted from these two words in parallel.

A hotly debated issue in reading research concerns the serial versus parallel nature of word identification processes. Evidence in favor of parallel processing has been criticized because it has often been obtained with artificial reading paradigms. Here we show that fixation‐related potentials elicited by a target word embedded in a normal sentence are impacted by the nature of the word immediately to the right of the fixated target word (n), and that this impact became significant 260 ms after fixation of word n. This is in line with parallel orthographic processing across adjacent words during sentence reading.

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.

A co-registration investigation of inter-word spacing and parafoveal preview: Eye movements and fixation-related potentials

Participants' eye movements (EMs) and EEG signal were simultaneously recorded to examine foveal and parafoveal processing during sentence reading. All the words in the sentence were manipulated for inter-word spacing (intact spaces vs. spaces replaced by a random letter) and parafoveal preview (identical preview vs. random letter string preview). We observed disruption for unspaced text and invalid preview conditions in both EMs and fixation-related potentials (FRPs). Unspaced and invalid preview conditions received longer reading times than spaced and valid preview conditions. In addition, the FRP data showed that unspaced previews disrupted reading in earlier time windows of analysis, compared to string preview conditions. Moreover, the effect of parafoveal preview was greater for spaced relative to unspaced conditions, in both EMs and FRPs. These findings replicate well-established preview effects, provide novel insight into the neural correlates of reading with and without inter-word spacing and suggest that spatial selection precedes lexical processing.

Speed-Accuracy Tradeoffs in Brain and Behavior: Testing the Independence of P300 and N400 Related Processes in Behavioral Responses to Sentence Categorization

Although the N400 was originally discovered in a paradigm designed to elicit a P300 (Kutas and Hillyard, 1980), its relationship with the P300 and how both overlapping event-related potentials (ERPs) determine behavioral profiles is still elusive. Here we conducted an ERP (N = 20) and a multiple-response speed-accuracy tradeoff (SAT) experiment (N = 16) on distinct participant samples using an antonym paradigm (The opposite of black is white/nice/yellow with acceptability judgment). We hypothesized that SAT profiles incorporate processes of task-related decision-making (P300) and stimulus-related expectation violation (N400). We replicated previous ERP results (Roehm et al., 2007): in the correct condition (white), the expected target elicits a P300, while both expectation violations engender an N400 [reduced for related (yellow) vs. unrelated targets (nice)]. Using multivariate Bayesian mixed-effects models, we modeled the P300 and N400 responses simultaneously and found that correlation between residuals and subject-level random effects of each response window was minimal, suggesting that the components are largely independent. For the SAT data, we found that antonyms and unrelated targets had a similar slope (rate of increase in accuracy over time) and an asymptote at ceiling, while related targets showed both a lower slope and a lower asymptote, reaching only approximately 80% accuracy. Using a GLMM-based approach (Davidson and Martin, 2013), we modeled these dynamics using response time and condition as predictors. Replacing the predictor for condition with the averaged P300 and N400 amplitudes from the ERP experiment, we achieved identical model performance. We then examined the piecewise contribution of the P300 and N400 amplitudes with partial effects (see Hohenstein and Kliegl, 2015). Unsurprisingly, the P300 amplitude was the strongest contributor to the SAT-curve in the antonym condition and the N400 was the strongest contributor in the unrelated condition. In brief, this is the first demonstration of how overlapping ERP responses in one sample of participants predict behavioral SAT profiles of another sample. The P300 and N400 reflect two independent but interacting processes and the competition between these processes is reflected differently in behavioral parameters of speed and accuracy.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

An electrophysiological investigation of orthographic spatial integration in reading

During reading, word recognition speed is influenced by the amount of orthographic overlap with surrounding words. The nature of this phenomenon is not understood: some theories attribute it to low-level visual operations (i.e., parafoveal feature detectors influencing foveal letter detectors), whereas other theories assume that orthographic processing (i.e., letter position coding and word activation) occurs across multiple words in parallel. To arbitrate between these theories, we used electroencephalography to reveal the time course of orthographic spatial integration in a lexical decision task. Foveal target words were flanked on each side by parafoveal words, manipulated across three conditions: repetition flankers (e.g. rock rock rock), unrelated flankers (step rock step) and a no-flanker condition. Linear mixed-effect models were constructed to analyze EEG data on a trial-by-trial basis. Word recognition was worse in the unrelated flanker condition than in the repetition and no-flanker conditions. This behavioral pattern was accompanied by increased negativity in the N250 and N400 windows, associated with the activation of sub-lexical and lexico-semantic representations, respectively. Crucially, the absence of effects prior to 200 ms post-stimulus onset provides evidence against the involvement of low-level visual processes. We conclude that orthographic spatial integration is driven by parallel processing of multiple words, which leads to the activation of a larger set of sub-lexical nodes and more difficult processing at the lexical level when those words are orthographically unrelated.

•

Words are recognized faster when they share many letters with surrounding words.

•

EEG measurements suggest that this effect unfolds 200 ms after stimulus onset.

•

The locus of effects contradicts the involvement of low-level visual processing.

•

Instead, the phenomenon appears to be driven by parallel orthographic processing.