Trans-saccadic parafoveal preview benefits in fluent reading: A study with fixation-related brain potentials

An early effect of the parafoveal preview on post-saccadic processing of English words

A key aspect of efficient visual processing is to use current and previous information to make predictions about what we will see next. In natural viewing, and when looking at words, there is typically an indication of forthcoming visual information from extrafoveal areas of the visual field before we make an eye movement to an object or word of interest. This "preview effect" has been studied for many years in the word reading literature and, more recently, in object perception. Here, we integrated methods from word recognition and object perception to investigate the timing of the preview on neural measures of word recognition. Through a combined use of EEG and eye-tracking, a group of multilingual participants took part in a gaze-contingent, single-shot saccade experiment in which words appeared in their parafoveal visual field. In valid preview trials, the same word was presented during the preview and after the saccade, while in the invalid condition, the saccade target was a number string that turned into a word during the saccade. As hypothesized, the valid preview greatly reduced the fixation-related evoked response. Interestingly, multivariate decoding analyses revealed much earlier preview effects than previously reported for words, and individual decoding performance correlated with participant reading scores. These results demonstrate that a parafoveal preview can influence relatively early aspects of post-saccadic word processing and help to resolve some discrepancies between the word and object literatures.

Neural mechanism underlying preview effects and masked priming effects in visual word processing

Two classic experimental paradigms - masked repetition priming and the boundary paradigm - have played a pivotal role in understanding the process of visual word recognition. Traditionally, these paradigms have been employed by different communities of researchers, with their own long-standing research traditions. Nevertheless, a review of the literature suggests that the brain-electric correlates of word processing established with both paradigms may show interesting similarities, in particular with regard to the location, timing, and direction of N1 and N250 effects. However, as of yet, no direct comparison has been undertaken between the two paradigms. In the current study, we used combined eye-tracking/EEG to perform such a within-subject comparison using the same materials (single Chinese characters) as stimuli. To facilitate direct comparisons, we used a simplified version of the boundary paradigm - the single word boundary paradigm. Our results show the typical early repetition effects of N1 and N250 for both paradigms. However, repetition effects in N250 (i.e., a reduced negativity following identical-word primes/previews as compared to different-word primes/previews) were larger with the single word boundary paradigm than with masked priming. For N1 effects, repetition effects were similar across the two paradigms, showing a larger N1 after repetitions as compared to alternations. Therefore, the results indicate that at the neural level, a briefly presented and masked foveal prime produces qualitatively similar facilitatory effects on visual word recognition as a parafoveal preview before a single saccade, although such effects appear to be stronger in the latter case.

Repetition Suppression for Familiar Visual Words Through Acceleration of Early Processing

The visual N1 (N170) component with occipito-temporal negativity and fronto-central positivity is sensitive to visual expertise for print. Slightly later, an N200 component with an increase after stimulus repetition was reported to be specific for Chinese, but found at centro-parietal electrodes against a mastoid reference. Given the unusual location, temporal proximity to the N1, and atypical repetition behavior, we aimed at clarifying the relation between the two components. We collected 128-channel EEG data from 18 native Chinese readers during a script decision experiment. Familiar Chinese one- and two-character words were presented among unfamiliar Korean control stimuli with half of the stimuli immediately repeated. Stimulus repetition led to a focal increase in the N1 onset and to a wide-spread decrease in the N1 offset, especially for familiar Chinese and also prominently near the mastoids. A TANOVA analysis corroborated robust repetition effects in the N1 offset across ERP maps with a modulation by script familiarity around 300 ms. Microstate analyses revealed a shorter N1 microstate duration after repetitions, especially for Chinese. The results demonstrate that the previously reported centro-parietal N200 effects after repetitions reflect changes during the N1 offset at occipito-temporal electrodes including the mastoids. Although larger for Chinese, repetition effects could also be found for two-character Korean words, suggesting that they are not specific for Chinese. While the decrease of the N1 offset after repetition is in agreement with a repetition suppression effect, the microstate findings suggest that at least part of the facilitation is due to accelerated processing after repetition.

Expectation Modulates Repetition Suppression at Late But Not Early Stages during Visual Word Recognition: Evidence from Event-related Potentials

Visual word recognition is commonly rapid and efficient, incorporating top-down predictive processing mechanisms. Neuroimaging studies with face stimuli suggest that repetition suppression (RS) reflects predictive processing at the neural level, as this effect is larger when repetitions are more frequent, that is, more expected. It remains unclear, however, at the temporal level whether and how RS and its modulation by expectation occur in visual word recognition. To address this gap, the present study aimed to investigate the presence and time course of these effects during visual word recognition using EEG. Thirty-six native Cantonese speakers were presented with pairs of Chinese written words and performed a nonlinguistic oddball task. The second word of a pair was either a repetition of the first or a different word (alternation). In repetition blocks, 75% of trials were repetitions and 25% were alternations, whereas the reverse was true in alternation blocks. Topographic analysis of variance of EEG at each time point showed robust RS effects in three time windows (141-227 msec, 242-445 msec, and 467-513 msec) reflecting facilitation of visual word recognition. Importantly, the modulation of RS by expectation was observed at the late rather than early intervals (334-387 msec, 465-550 msec, and 559-632 msec) and more than 100 msec after the first RS effects. In the predictive coding view of RS, only late repetition effects are modulated by expectation, whereas early RS effects may be mediated by lower-level predictions. Taken together, our findings provide the first EEG evidence revealing distinct temporal dynamics of RS effects and repetition probability on RS effects in visual processing of Chinese words.

Parafoveal and foveal N400 effects in natural reading: A timeline of semantic processing from fixation-related potentials

The depth at which parafoveal words are processed during reading is an ongoing topic of debate. Recent studies using RSVP-with-flanker paradigms have shown that implausible words within sentences elicit an N400 component while they are still in parafoveal vision, suggesting that the semantics of parafoveal words can be accessed to rapidly update the sentence representation. To study this effect in natural reading, we combined the coregistration of eye movements and EEG with the deconvolution modeling of fixation-related potentials (FRPs) to test whether semantic plausibility is processed parafoveally during Chinese sentence reading. For one target word per sentence, both its parafoveal and foveal plausibility were orthogonally manipulated using the boundary paradigm. Consistent with previous eye movement studies, we observed a delayed effect of parafoveal plausibility on fixation durations that only emerged on the foveal word. Crucially, in FRPs aligned to the pretarget fixation, a clear N400 effect emerged already based on parafoveal plausibility, with more negative voltages for implausible previews. Once participants fixated the target, we again observed an N400 effect of foveal plausibility. Interestingly, this foveal N400 was absent whenever the preview had been implausible, indicating that when a word's (im)plausibility is already processed in parafoveal vision, this information is not revised anymore upon direct fixation. Implausible words also elicited a late positive component (LPC), but exclusively when in foveal vision. Our results not only provide convergent neural and behavioral evidence for the parafoveal uptake of semantic information, but also indicate different contributions of parafoveal versus foveal information toward higher level sentence processing.

Saccade execution increases the preview effect with faces: An EEG and eye-tracking coregistration study

Under naturalistic viewing conditions, humans conduct about three to four saccadic eye movements per second. These dynamics imply that in real life, humans rarely see something completely new; there is usually a preview of the upcoming foveal input from extrafoveal regions of the visual field. In line with results from the field of reading research, we have shown with EEG and eye-tracking coregistration that an extrafoveal preview also affects postsaccadic visual object processing and facilitates discrimination. Here, we ask whether this preview effect in the fixation-locked N170, and in manual responses to the postsaccadic target face (tilt discrimination), requires saccade execution. Participants performed a gaze-contingent experiment in which extrafoveal face images could change their orientation during a saccade directed to them. In a control block, participants maintained stable gaze throughout the experiment and the extrafoveal face reappeared foveally after a simulated saccade latency. Compared with this no-saccade condition, the neural and the behavioral preview effects were much larger in the saccade condition. We also found shorter first fixation durations after an invalid preview, which is in contrast to reading studies. We interpret the increased preview effect under saccade execution as the result of the additional sensorimotor processes that come with gaze behavior compared with visual perception under stable fixation. In addition, our findings call into question whether EEG studies with fixed gaze capture key properties and dynamics of active, natural vision.

Emoji-SP, the Spanish emoji database: Visual complexity, familiarity, frequency of use, clarity, and emotional valence and arousal norms for 1031 emojis

This article presents subjective norms for 1031 emojis in six dimensions: visual complexity, familiarity, frequency of use, clarity, emotional valence, and emotional arousal. This is the largest normative study conducted so far that relies on subjective ratings. Unlike the few existing normative studies, which mainly comprise face emojis, here we present a wide range of emoji categories. We also examine the correlations between the dimensions assessed. Our results show that, in terms of their affective properties, emojis are analogous to other stimuli, such as words, showing the expected U-shaped relationship between valence and arousal. The relationship between affective properties and other dimensions (e.g., between valence and familiarity) is also similar to the relationship observed in words, in the sense that positively valenced emojis are more familiar than negative ones. These findings suggest that emojis are suitable stimuli for studying affective processing. Emoji-SP will be highly valuable for researchers of various fields interested in emojis, including computer science, communication, linguistics, and psychology. The full set of norms is available at: https://osf.io/dtfjv/ .

Parallel word reading revealed by fixation-related brain potentials

During reading, the brain is confronted with many relevant objects at once. But does lexical processing occur for multiple words simultaneously? Cognitive science has yet to answer this prominent question. Recently it has been argued that the issue warrants supplementing the field's traditional toolbox (response times, eye-tracking) with neuroscientific techniques (EEG, fMRI). Indeed, according to the OB1-reader model, upcoming words need not impact oculomotor behavior per se, but parallel processing of these words must nonetheless be reflected in neural activity. Here we combined eye-tracking with EEG, time-locking the neural window of interest to the fixation on target words in sentence reading. During these fixations, we manipulated the identity of the subsequent word so that it posed either a syntactically legal or illegal continuation of the sentence. In line with previous research, oculomotor measures were unaffected. Yet, syntax impacted brain potentials as early as 100 ms after the target fixation onset. Given the EEG literature on syntax processing, the presently observed timings suggest parallel word reading. We reckon that parallel word processing typifies reading, and that OB1-reader offers a good platform for theorizing about the reading brain.

Event-related potentials show that parafoveal vision is insufficient for semantic integration

Readers extract information from a word from parafoveal vision prior to looking at it. It has been argued that parafoveal perception allows readers to initiate linguistic processes, but it is unclear which stages of word processing are engaged: the process of extracting letter information to recognize words, or the process of extracting meaning to comprehend them. This study used the event-related brain potential (ERP) technique to investigate how word recognition (indexed by the N400 effect for unexpected or anomalous compared to expected words) and semantic integration (indexed by the Late-positive component; LPC effect for anomalous compared to expected words) are or are not elicited when the word is perceived only in parafoveal vision. Participants read a target word following a sentence that made it expected, unexpected, or anomalous, and read the sentences presented three words at a time in the Rapid Serial Visual Presentation (RSVP) with flankers paradigm so that words were perceived in parafoveal and foveal vision. We orthogonally manipulated whether the target word was masked in parafoveal and/or foveal vision to dissociate the processing associated with perception of the target word from either location. We found that the N400 effect was generated from parafoveally perceived words, and was reduced for foveally perceived words if they were previously perceived parafoveally. In contrast, the LPC effect was only elicited if the word was perceived foveally, suggesting that readers must attend to a word directly in foveal vision in order to attempt to integrate its meaning into the sentence context.

Lateralization of orthographic processing in fixed-gaze and natural reading conditions

Lateralized processing of orthographic information is a hallmark of proficient reading. However, how this finding obtained for fixed-gaze processing of orthographic stimuli translates to ecologically valid reading conditions remained to be clarified. To address this shortcoming, here we assessed the lateralization of early orthographic processing in fixed-gaze and natural reading conditions using concurrent eye-tracking and EEG data recorded from young adults without reading difficulties. Sensor-space analyses confirmed the well-known left-lateralized negative-going deflection of fixed-gaze EEG activity throughout the period of early orthographic processing. At the same time, fixation-related EEG activity exhibited left-lateralized followed by right-lateralized processing of text stimuli during natural reading. A strong positive relationship was found between the early leftward lateralization in fixed-gaze and natural reading conditions. Using source-space analyses, early left-lateralized brain activity was obtained in lateraloccipital and posterior ventral occipito-temporal cortices reflecting letter-level processing in both conditions. In addition, in the same time interval, left-lateralized source activity was found also in premotor and parietal brain regions during natural reading. While brain activity remained left-lateralized in later stages representing word-level processing in posterior and middle ventral temporal regions in the fixed-gaze condition, fixation-related source activity became stronger in the right hemisphere in medial and more anterior ventral temporal brain regions indicating higher-level processing of orthographic information. Although our results show a strong positive relationship between the lateralization of letter-level processing in the two reading modes and suggest lateralized brain activity as a general marker for processing of orthographic information, they also clearly indicate the need for reading research in ecologically valid conditions to identify the neural basis of visuospatial attentional, oculomotor and higher-level processes specific to natural reading.

Parafoveal preview benefit in a conflicting sentential context: Evidence from ERPs

In natural reading, the reader's processing of a word starts when the word is located in parafoveal vision. Relative to a situation with an invalid preview, fixations are significantly shorter after a fully valid, identical preview. Although research on the identity preview benefit has been ongoing for more than 40 years, the mechanism of this preview benefit, the level of lexical processing it occurs, and its relationship to the sentential context remain unclear. This study employed EEG brain component analysis technology to address these questions in Chinese sentence reading. We manipulated the sentential context to make the target word plausible or not plausible with the sentence and manipulated the target word present or not present in preview. EEG results showed that the identity preview benefit can affect not only the early preview positivity, reflecting the early orthographic processing of words, but also the N400 and LPC components, reflecting the late and in-depth semantic processing of words. Conflicting sentential context, in which the target word is implausible and cannot be integrated into the sentence, can interfere temporarily with these processes. These findings suggest that in the process of sentence reading, an identical preview word can promote the subsequent reading process at multiple levels, and its role is modulated by contextual information.

Fixation-related fMRI analysis reveals the neural basis of parafoveal processing in self-paced reading of Chinese words

While parafoveal word processing plays an important role in natural reading, the underlying neural mechanism remains unclear. The present study investigated the neural basis of parafoveal processing during Chinese word reading with the co-registration of eye-tracking and functional magnetic resonance imaging (fMRI) using fixation-related fMRI analysis. In the gaze-contingent boundary paradigm, preview conditions (words that are identical, orthographically similar, and unrelated to target words), pre-target word frequency and target word frequency were manipulated. When fixating the pre-target word, the identical preview condition elicited lower brain activation in the left fusiform gyrus relative to unrelated and orthographically similar preview conditions and there were significant interactions of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus, left fusiform gyrus and supplementary motor area. When fixating the target word, there was a significant main effect of preview condition on brain activation of the right fusiform gyrus and a significant interaction of preview condition and pre-target word frequency on brain activation of the left middle frontal gyrus. These results suggest that fixation-related brain activation provides immediate measures and new perspectives to understand the mechanism of parafoveal processing in self-paced reading.

Detection of deviance in Japanese kanji compound words

Reading fluency is based on the automatic visual recognition of words. As a manifestation of the automatic processing of words, an automatic deviance detection of visual word stimuli can be observed in the early stages of visual recognition. To clarify whether this phenomenon occurs with Japanese kanji compounds—since their lexicality is related to semantic association—we investigated the brain response by utilizing three types of deviants: differences in font type, lexically correct or incorrect Japanese kanji compound words and pseudo-kanji characters modified from correct and incorrect compounds. We employed magnetoencephalography (MEG) to evaluate the spatiotemporal profiles of the related brain regions. The study included 22 adult native Japanese speakers (16 females). The abovementioned three kinds of stimuli containing 20% deviants were presented during the MEG measurement. Activity in the occipital pole region of the brain was observed upon the detection of font-type deviance within 250 ms of stimulus onset. Although no significant activity upon detecting lexically correct/incorrect kanji compounds or pseudo-kanji character deviations was observed, the activity in the posterior transverse region of the collateral sulcus (pCoS)—which is a fusiform neighboring area—was larger when detecting lexically correct kanji compounds than when detecting pseudo-kanji characters. Taken together, these results support the notion that the automatic detection of deviance in kanji compounds may be limited to a low-level feature, such as the stimulus stroke thickness.

Brain Source Correlates of Speech Perception and Reading Processes in Children With and Without Reading Difficulties

Neural correlates in reading and speech processing have been addressed extensively in the literature. While reading skills and speech perception have been shown to be associated with each other, their relationship remains debatable. In this study, we investigated reading skills, speech perception, reading, and their correlates with brain source activity in auditory and visual modalities. We used high-density event-related potentials (ERPs), fixation-related potentials (FRPs), and the source reconstruction method. The analysis was conducted on 12–13-year-old schoolchildren who had different reading levels. Brain ERP source indices were computed from frequently repeated Finnish speech stimuli presented in an auditory oddball paradigm. Brain FRP source indices were also computed for words within sentences presented in a reading task. The results showed significant correlations between speech ERP sources and reading scores at the P100 (P1) time range in the left hemisphere and the N250 time range in both hemispheres, and a weaker correlation for visual word processing N170 FRP source(s) in the posterior occipital areas, in the vicinity of the visual word form areas (VWFA). Furthermore, significant brain-to-brain correlations were found between the two modalities, where the speech brain sources of the P1 and N250 responses correlated with the reading N170 response. The results suggest that speech processes are linked to reading fluency and that brain activations to speech are linked to visual brain processes of reading. These results indicate that a relationship between language and reading systems is present even after several years of exposure to print.

ERPs Reveal How Semantic and Syntactic Processing Unfold across Parafoveal and Foveal Vision during Sentence Comprehension

We examined how readers process content and function words in sentence comprehension with ERPs. Participants read simple declarative sentences using a rapid serial visual presentation (RSVP) with flankers paradigm. Sentences contained either an unexpected semantically anomalous content word, an unexpected syntactically anomalous function word or were well formed with no anomalies. ERPs were examined when target words were in the parafoveal or foveal vision. Unexpected content words elicited a typically distributed N400 when displayed in the parafovea, followed by a longer-lasting, widely distributed positivity starting around 300 ms once foveated. Unexpected function words elicited a left lateralized LAN-like component when presented in the parafovea, followed by a left lateralized, posteriorly distributed P600 when foveated. These results suggested that both semantic and syntactic processing involve two stages-the initial, fast process that can be completed in parafovea, followed by a more in depth attentionally mediated assessment that occurs with direct attention.

Lateralization of early orthographic processing during natural reading is impaired in developmental dyslexia

Skilled reading requires specialized visual cortical processing of orthographic information and its impairment has been proposed as a potential correlate of compromised reading in dyslexia. However, which stage of orthographic information processing during natural reading is disturbed in dyslexics remains unexplored. Here we addressed this question by simultaneously measuring the eye movements and EEG of dyslexic and control young adults during natural reading. Isolated meaningful sentences were presented at five inter-letter spacing levels spanning the range from minimal to extra-large spacing, and participants were instructed to read the text silently at their own pace. Control participants read faster, performed larger saccades and shorter fixations compared to dyslexics. While reading speed peaked around the default letter spacing, saccade amplitude increased and fixation duration decreased with the increase of letter spacing in both groups. Lateralization of occipito-temporal fixation-related EEG activity (FREA) was found in three consecutive time intervals corresponding to early orthographic processing in control readers. Importantly, the lateralization in the time range of the first negative left occipito-temporal FREA peak was specific for first fixations and exhibited an interaction effect between reading ability and letter spacing. The interaction originated in the significant decrease of FREA lateralization at extra-large compared to default letter spacing in control readers and the lack of lateralization in both letter spacing conditions in the case of dyslexics. These findings suggest that expertise-driven hemispheric functional specialization for early orthographic processing thought to be responsible for letter identity extraction during natural reading is compromised in dyslexia.

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Fixation-related saccadic inhibition in free viewing in response to stimulus saliency

Microsaccades that occur during fixation were studied extensively in response to transient stimuli, showing a typical inhibition (Oculomotor Inhibition, OMI), and a later release with a latency that depends on stimulus saliency, attention, and expectations. Here, we investigated the hypothesis that in free viewing every saccade provides a new transient stimulation that should result in a stimulus-dependent OMI like a flashed presentation during fixation. Participants (N = 16) freely inspected static displays of randomly oriented Gabor texture images, with varied contrast and spatial frequency (SF) for periods of 10 s each. Eye tracking recordings were divided into epochs triggered by saccade landing (> 1 dva), and microsaccade latency relative to fixation onset was computed (msRT). We found that the msRT in free viewing was shorter for more salient stimuli (higher contrast or lower SF), as previously found for flashed stimuli. It increased with saccade size and decreased across successive saccades, but only for higher contrast, suggesting contrast-dependent repetition enhancement in free viewing. Our results indicate that visual stimulus-dependent inhibition of microsaccades also applies to free viewing. These findings are in agreement with the similarity found between event-related and fixation-related potentials and open the way for studies combining both approaches to study natural vision.

Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements

Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing.

Neural evidence for lexical parafoveal processing

In spite of the reduced visual acuity, parafoveal information plays an important role in natural reading. However, competing models on reading disagree on whether words are previewed parafoveally at the lexical level. We find neural evidence for lexical parafoveal processing by combining a rapid invisible frequency tagging (RIFT) approach with magnetoencephalography (MEG) and eye-tracking. In a silent reading task, target words are tagged (flickered) subliminally at 60 Hz. The tagging responses measured when fixating on the pre-target word reflect parafoveal processing of the target word. We observe stronger tagging responses during pre-target fixations when followed by low compared with high lexical frequency targets. Moreover, this lexical parafoveal processing is associated with individual reading speed. Our findings suggest that reading unfolds in the fovea and parafovea simultaneously to support fluent reading.

During natural viewing, neural processing of visual targets continues throughout saccades

Relatively little is known about visual processing during free-viewing visual search in realistic dynamic environments. Free-viewing is characterized by frequent saccades. During saccades, visual processing is thought to be suppressed, yet we know that the presaccadic visual content can modulate postsaccadic processing. To better understand these processes in a realistic setting, we study here saccades and neural responses elicited by the appearance of visual targets in a realistic virtual environment. While subjects were being driven through a 3D virtual town, they were asked to discriminate between targets that appear on the road. Using a system identification approach, we separated overlapping and correlated activity evoked by visual targets, saccades, and button presses. We found that the presence of a target enhances early occipital as well as late frontocentral saccade-related responses. The earlier potential, shortly after 125 ms post-saccade onset, was enhanced for targets that appeared in the peripheral vision as compared to the central vision, suggesting that fast peripheral processing initiated before saccade onset. The later potential, at 195 ms post-saccade onset, was strongly modulated by the visibility of the target. Together these results suggest that, during natural viewing, neural processing of the presaccadic visual stimulus continues throughout the saccade, apparently unencumbered by saccadic suppression.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

Cross-linguistic semantic preview benefit in Basque-Spanish bilingual readers: Evidence from fixation-related potentials

During reading, we can process and integrate information from words allocated in the parafoveal region. However, whether we extract and process the meaning of parafoveal words is still under debate. Here, we obtained Fixation-Related Potentials in a Basque-Spanish bilingual sample during a Spanish reading task. By using the boundary paradigm, we presented different parafoveal previews that could be either Basque non-cognate translations or unrelated Basque words. We prove for the first time cross-linguistic semantic preview benefit effects in alphabetic languages, providing novel evidence of modulations in the N400 component. Our findings suggest that the meaning of parafoveal words is processed and integrated during reading and that such meaning is activated and shared across languages in bilingual readers.

Are emojis processed like words?: Eye movements reveal the time course of semantic processing for emojified text

Emojis have many functions that support reading. Most obviously, they convey semantic information and support reading comprehension (Lo, CyberPsychology & Behavior, 11[5], 595-597, 2008; Riordan, Computers in Human Behavior, 76, 75-86, 2017b). However, it is undetermined whether emojis recruit the same perceptual and cognitive processes for identification and integration during reading as do words. To investigate whether emojis are processed like words, we used eye tracking to examine the time course of semantic processing of emojis during reading. Materials consisted of sentences containing a target word (e.g., coffee in the sentence "My tall coffee is just the right temperature") when there was no emoji present and when there was a semantically congruent (i.e., synonymous) emoji (e.g., the cup of coffee emoji, ) or an incongruent emoji (e.g., the beer mug emoji, ) present at the end of the sentence. Similar to congruency effects with words, congruent emojis were fixated for shorter periods and were less likely to be refixated than were incongruent emojis. In addition, congruent emojis were more frequently skipped than incongruent emojis, which suggests that semantic aspects of emoji processing begin in the parafovea. Finally, the presence of an emoji, relative to its absence increased target-word skipping rates and reduced total time on target words. We discuss the implications of our findings for models of eye-movement control during reading.

The behavioural preview effect with faces is susceptible to statistical regularities: Evidence for predictive processing across the saccade

The world around us appears stable and continuous despite saccadic eye movements. This apparent visual stability is achieved by trans-saccadic perception leading at the behavioural level to preview effects: performance in processing a foveal stimulus is better if the stimulus remained unchanged (valid) compared to when it changed (invalid) during the saccade that brought it into focus. Trans-saccadic perception is known to predictively adapt to the statistics of the environment. Here, we asked whether the behavioural preview effect shows the same characteristics, employing a between-participants training design. Participants made saccades to faces which could change their orientation (upright/inverted) during the saccade. In addition, the post-saccadic face was slightly tilted and participants reported this tilt upon fixation. In a training phase, one group of participants conducted only invalid trials whereas another group conducted only valid trials. In a subsequent test phase with 50% valid and 50% invalid trials, we measured the preview effect. Invalid training reduced the preview effect. With a mixed-model analysis, we could show how this training effect gradually declines in the course of the test phase. These results show that the behavioural preview effect adapts to the statistics of the environment suggesting that it results from predictive processes.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Neurophysiological, Oculomotor, and Computational Modeling of Impaired Reading Ability in Schizophrenia

Schizophrenia (Sz) is associated with deficits in fluent reading ability that compromise functional outcomes. Here, we utilize a combined eye-tracking, neurophysiological, and computational modeling approach to analyze underlying visual and oculomotor processes. Subjects included 26 Sz patients (SzP) and 26 healthy controls. Eye-tracking and electroencephalography data were acquired continuously during the reading of passages from the Gray Oral Reading Tests reading battery, permitting between-group evaluation of both oculomotor activity and fixation-related potentials (FRP). Schizophrenia patients showed a marked increase in time required per word (d = 1.3, P < .0001), reflecting both a moderate increase in fixation duration (d = .7, P = .026) and a large increase in the total saccade number (d = 1.6, P < .0001). Simulation models that incorporated alterations in both lower-level visual and oculomotor function as well as higher-level lexical processing performed better than models that assumed either deficit-type alone. In neurophysiological analyses, amplitude of the fixation-related P1 potential (P1f) was significantly reduced in SzP (d = .66, P = .013), reflecting reduced phase reset of ongoing theta-alpha band activity (d = .74, P = .019). In turn, P1f deficits significantly predicted increased saccade number both across groups (P = .017) and within SzP alone (P = .042). Computational and neurophysiological methods provide increasingly important approaches for investigating sensory contributions to impaired cognition during naturalistic processing in Sz. Here, we demonstrate deficits in reading rate that reflect both sensory/oculomotor- and semantic-level impairments and that manifest, respectively, as alterations in saccade number and fixation duration. Impaired P1f generation reflects impaired fixation-related reset of ongoing brain rhythms and suggests inefficient information processing within the early visual system as a basis for oculomotor dyscontrol during fluent reading in Sz.

Oculomotor inhibition precedes temporally expected auditory targets

Eye movements are inhibited prior to the onset of temporally-predictable visual targets. This oculomotor inhibition effect could be considered a marker for the formation of temporal expectations and the allocation of temporal attention in the visual domain. Here we show that eye movements are also inhibited before predictable auditory targets. In two experiments, we manipulate the period between a cue and an auditory target to be either predictable or unpredictable. The findings show that although there is no perceptual gain from avoiding gaze-shifts in this procedure, saccades and blinks are inhibited prior to predictable relative to unpredictable auditory targets. These findings show that oculomotor inhibition occurs prior to auditory targets. This link between auditory expectation and oculomotor behavior reveals a multimodal perception action coupling, which has a central role in temporal expectations.

Eye contact in active and passive viewing: Event-related brain potential evidence from a combined eye tracking and EEG study

Eye contact is a salient social cue, which is assumed to influence already early neural correlates of face perception. Specifically, the N170 component of the event-related potential (ERP) has often been found to be larger for faces with an averted gaze as compared to faces that directly look at the observer. In most existing ERP studies, effects of eye contact were investigated under comparatively artificial conditions where participants were instructed to maintain a steady fixation while they passively observed gaze changes in the stimulus face. It is therefore unclear to what extent neural correlates of eye contact generalize to more naturalistic situations that involve a continuous interplay between directed and averted gaze between the communication partners. To start bridging this gap, the present study compared the passive viewing of gaze changes to an active condition in which the participant's own gaze (measured online with an eye tracker) interacted with the gaze position of a continuously presented stimulus face. We also investigated whether eye contact effects were modulated by the face's emotional expression. In both the passive and the active viewing condition, N170 amplitudes were larger when the gaze of the stimulus faces was averted rather than directed towards the participant. Furthermore, eye contact decreased P300 amplitudes in both conditions. The emotional expression of the face also modulated the N170, but this effect did not interact with that of gaze direction. We conclude that the neural correlates of gaze perception during active gaze interactions are comparable to those found during passive viewing, encouraging the further study of eye contact effects in more naturalistic settings.

Fixation-related Brain Potentials during Semantic Integration of Object–Scene Information

In vision science, a particularly controversial topic is whether and how quickly the semantic information about objects is available outside foveal vision. Here, we aimed at contributing to this debate by coregistering eye movements and EEG while participants viewed photographs of indoor scenes that contained a semantically consistent or inconsistent target object. Linear deconvolution modeling was used to analyze the ERPs evoked by scene onset as well as the fixation-related potentials (FRPs) elicited by the fixation on the target object (t) and by the preceding fixation (t − 1). Object–scene consistency did not influence the probability of immediate target fixation or the ERP evoked by scene onset, which suggests that object–scene semantics was not accessed immediately. However, during the subsequent scene exploration, inconsistent objects were prioritized over consistent objects in extrafoveal vision (i.e., looked at earlier) and were more effortful to process in foveal vision (i.e., looked at longer). In FRPs, we demonstrate a fixation-related N300/N400 effect, whereby inconsistent objects elicit a larger frontocentral negativity than consistent objects. In line with the behavioral findings, this effect was already seen in FRPs aligned to the pretarget fixation t − 1 and persisted throughout fixation t, indicating that the extraction of object semantics can already begin in extrafoveal vision. Taken together, the results emphasize the usefulness of combined EEG/eye movement recordings for understanding the mechanisms of object–scene integration during natural viewing.

Face Selective Neural Activity: Comparisons Between Fixed and Free Viewing

Event Related Potentials (ERPs) are widely used to study category-selective EEG responses to visual stimuli, such as the face-selective N170 component. Typically, this is done by flashing stimuli at the point of static gaze fixation. While allowing for good experimental control, these paradigms ignore the dynamic role of eye-movements in natural vision. Fixation-related potentials (FRPs), obtained using simultaneous EEG and eye-tracking, overcome this limitation. Various studies have used FRPs to study processes such as lexical processing, target detection and attention allocation. The goal of this study was to carefully compare face-sensitive activity time-locked to an abrupt stimulus onset at fixation, with that time-locked to a self-generated fixation on a stimulus. Twelve participants participated in three experimental conditions: Free-viewing (FRPs), Cued-viewing (FRPs) and Control (ERPs). We used a multiple regression approach to disentangle overlapping activity components. Our results show that the N170 face-effect is evident for the first fixation on a stimulus, whether it follows a self-generated saccade or stimulus appearance at fixation point. The N170 face-effect has similar topography across viewing conditions, but there were major differences within each stimulus category. We ascribe these differences to an overlap of the fixation-related lambda response and the N170. We tested the plausibility of this account using dipole simulations. Finally, the N170 exhibits category-specific adaptation in free viewing. This study establishes the comparability of the free-viewing N170 face-effect with the classic event-related effect, while highlighting the importance of accounting for eye-movement related effects.

Parafoveal-on-foveal repetition effects in sentence reading: A co-registered eye-tracking and electroencephalogram study

When reading, can the next word in the sentence (word n + 1) influence how you read the word you are currently looking at (word n)? Serial models of sentence reading state that this generally should not be the case, whereas parallel models predict that this should be the case. Here we focus on perhaps the simplest and the strongest Parafoveal‐on‐Foveal (PoF) manipulation: word n + 1 is either the same as word n or a different word. Participants read sentences for comprehension and when their eyes left word n, the repeated or unrelated word at position n + 1 was swapped for a word that provided a syntactically correct continuation of the sentence. We recorded electroencephalogram and eye‐movements, and time‐locked the analysis of fixation‐related potentials (FRPs) to fixation of word n. We found robust PoF repetition effects on gaze durations on word n, and also on the initial landing position on word n. Most important is that we also observed significant effects in FRPs, reaching significance at 260 ms post‐fixation of word n. Repetition of the target word n at position n + 1 caused a widely distributed reduced negativity in the FRPs. Given the timing of this effect, we argue that it is driven by orthographic processing of word n + 1, while readers were still looking at word n, plus the spatial integration of orthographic information extracted from these two words in parallel.

A hotly debated issue in reading research concerns the serial versus parallel nature of word identification processes. Evidence in favor of parallel processing has been criticized because it has often been obtained with artificial reading paradigms. Here we show that fixation‐related potentials elicited by a target word embedded in a normal sentence are impacted by the nature of the word immediately to the right of the fixated target word (n), and that this impact became significant 260 ms after fixation of word n. This is in line with parallel orthographic processing across adjacent words during sentence reading.

Interaction of top-down category-level expectation and bottom-up sensory input in early stages of visual-orthographic processing

How and when top-down information modulates visual-orthographic processing is an essential question in reading research. In a previous study, we showed that task modulation of print-tuning started at around 170 ms after stimulus presentation in the N1 offset of the ERP, while the N1 onset was yet unaffected. Here we test how prior category-level expectation affects visual-orthographic processing. Familiar, left/right-structured Chinese characters and stroke number matched, unfamiliar Korean characters were presented, while expectation about the upcoming stimuli was manipulated with green and blue colored frames (high Chinese vs. high Korean expectation). EEG data of 18 native Chinese speakers were recorded while participants performed an expectation judgment task. Results from occipito-temporal and whole map analyses revealed that effects of prior expectation changed throughout the N1. Accordingly in the N1 onset, a print tuning main effect was found, with a stronger N1 to Chinese characters than Korean characters, irrespective of expectation. In the N1 offset, an expectation-by-character interaction was observed at the whole map level, with a more negative N1 to Korean characters than Chinese characters when expecting a Chinese character, but no such difference when expecting a Korean character. Moreover, the expectation-by-character interaction continued to the N250, with similar responses to Chinese and Korean characters under the Chinese expectation condition, while less negative N250 to Korean than Chinese under the Korean expectation condition. Taken together, the current study provides evidence that prior category-level expectation starts to take effect at an early stage even within 200 ms by facilitating the processing of expected stimuli, suggesting that category-level expectation can influence early visual-orthographic processing during word recognition.

Eye Movements and Fixation-Related Potentials in Reading: A Review

The present review is addressed to researchers in the field of reading and psycholinguistics who are both familiar with and new to co-registration research of eye movements (EMs) and fixation related-potentials (FRPs) in reading. At the outset, we consider a conundrum relating to timing discrepancies between EM and event related potential (ERP) effects. We then consider the extent to which the co-registration approach might allow us to overcome this and thereby discriminate between formal theoretical and computational accounts of reading. We then describe three phases of co-registration research before evaluating the existing body of such research in reading. The current, ongoing phase of co-registration research is presented in comprehensive tables which provide a detailed summary of the existing findings. The thorough appraisal of the published studies allows us to engage with issues such as the reliability of FRP components as correlates of cognitive processing in reading and the advantages of analysing both data streams (i.e., EMs and FRPs) simultaneously relative to each alone, as well as the current, and limited, understanding of the relationship between EM and FRP measures. Finally, we consider future directions and in particular the potential of analytical methods involving deconvolution and the potential of measurement of brain oscillatory activity.

Post-Saccadic Face Processing Is Modulated by Pre-Saccadic Preview: Evidence from Fixation-Related Potentials

Humans actively sample their environment with saccadic eye movements to bring relevant information into high-acuity foveal vision. Despite being lower in resolution, peripheral information is also available before each saccade. How the pre-saccadic extrafoveal preview of a visual object influences its post-saccadic processing is still an unanswered question. The current study investigated this question by simultaneously recording behavior and fixation-related brain potentials while human subjects made saccades to face stimuli. We manipulated the relationship between pre-saccadic "previews" and post-saccadic images to explicitly isolate the influences of the former. Subjects performed a gender discrimination task on a newly foveated face under three preview conditions: scrambled face, incongruent face (different identity from the foveated face), and congruent face (same identity). As expected, reaction times were faster after a congruent-face preview compared with a scrambled-face preview. Importantly, intact face previews (either incongruent or congruent) resulted in a massive reduction of post-saccadic neural responses. Specifically, we analyzed the classic face-selective N170 component at occipitotemporal electroencephalogram electrodes, which was still present in our experiments with active looking. However, the post-saccadic N170 was strongly attenuated following intact-face previews compared with the scrambled condition. This large and long-lasting decrease in evoked activity is consistent with a trans-saccadic mechanism of prediction that influences category-specific neural processing at the start of a new fixation. These findings constrain theories of visual stability and show that the extrafoveal preview methodology can be a useful tool to investigate its underlying mechanisms.SIGNIFICANCE STATEMENT Neural correlates of object recognition have traditionally been studied by flashing stimuli to the central visual field. This procedure differs in fundamental ways from natural vision, where viewers actively sample the environment with eye movements and also obtain a low-resolution preview of soon-to-be-fixated objects. Here we show that the N170, a classic electrophysiological marker of the structural encoding of faces, also occurs during a more natural viewing condition but is strongly reduced due to extrafoveal preprocessing (preview benefit). Our results therefore highlight the importance of peripheral vision during trans-saccadic processing in building a coherent and stable representation of the world around us.

A co-registration investigation of inter-word spacing and parafoveal preview: Eye movements and fixation-related potentials

Participants' eye movements (EMs) and EEG signal were simultaneously recorded to examine foveal and parafoveal processing during sentence reading. All the words in the sentence were manipulated for inter-word spacing (intact spaces vs. spaces replaced by a random letter) and parafoveal preview (identical preview vs. random letter string preview). We observed disruption for unspaced text and invalid preview conditions in both EMs and fixation-related potentials (FRPs). Unspaced and invalid preview conditions received longer reading times than spaced and valid preview conditions. In addition, the FRP data showed that unspaced previews disrupted reading in earlier time windows of analysis, compared to string preview conditions. Moreover, the effect of parafoveal preview was greater for spaced relative to unspaced conditions, in both EMs and FRPs. These findings replicate well-established preview effects, provide novel insight into the neural correlates of reading with and without inter-word spacing and suggest that spatial selection precedes lexical processing.

Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments

Combining EEG with eye-tracking is a promising approach to study neural correlates of natural vision, but the resulting recordings are also heavily contaminated by activity of the eye balls, eye lids, and extraocular muscles. While Independent Component Analysis (ICA) is commonly used to suppress these ocular artifacts, its performance under free viewing conditions has not been systematically evaluated and many published reports contain residual artifacts. Here I evaluated and optimized ICA-based correction for two tasks with unconstrained eye movements: visual search in images and sentence reading. In a first step, four parameters of the ICA pipeline were varied orthogonally: the (1) high-pass and (2) low-pass filter applied to the training data, (3) the proportion of training data containing myogenic saccadic spike potentials (SP), and (4) the threshold for eye tracker-based component rejection. In a second step, the eye-tracker was used to objectively quantify the correction quality of each ICA solution, both in terms of undercorrection (residual artifacts) and overcorrection (removal of neurogenic activity). As a benchmark, results were compared to those obtained with an alternative spatial filter, Multiple Source Eye Correction (MSEC). With commonly used settings, Infomax ICA not only left artifacts in the data, but also distorted neurogenic activity during eye movement-free intervals. However, correction results could be strongly improved by training the ICA on optimally filtered data in which SPs were massively overweighted. With optimized procedures, ICA removed virtually all artifacts, including the SP and its associated spectral broadband artifact from both viewing paradigms, with little distortion of neural activity. It also outperformed MSEC in terms of SP correction. Matlab code is provided.

Alternating-color words influence Chinese sentence reading: Evidence from neural connectivity

In order to investigate how language and attention systems are affected by word boundary information during reading, we conducted a functional magnetic resonance imaging (fMRI) experiment in which text-color in naturally unspaced Chinese sentences were systematically manipulated in three experimental conditions, that is, text-color alternation consistent or inconsistent with word boundary (i.e., alternating-color word and non-word conditions), as well as a mono-color baseline condition. Twenty college students (14 females; 23.1 years old) were required to silently read 72 sentences during fMRI scanning. We found that the conditions of word boundary modulated the brain connections between the visual word form area (VWFA) and dorsal attention regions, and between the VWFA and language-related regions. These results suggest that the coordination between the VWFA and dorsal attention regions plays an important role in grouping characters and guiding the saccade according to perceptual grouping based on color, and that the connection between VWFA and MTG could be the neural mechanism of lexical access during Chinese text reading.

Event-related brain potentials reveal how multiple aspects of semantic processing unfold across parafoveal and foveal vision during sentence reading

Recent event-related brain potential (ERP) experiments have demonstrated parafoveal N400 expectancy and congruity effects, showing that semantic information can be accessed from words in parafoveal vision (a conclusion also supported by some eye-tracking work). At the same time, it is unclear how higher-order integrative aspects of language comprehension unfold across the visual field during reading. In the current study, we recorded ERPs in a parafoveal flanker paradigm, while readers were instructed to read passively for comprehension or to judge the plausibility of sentences in which target words varied in their semantic expectancy and congruity. We directly replicated prior work showing graded N400 effects for parafoveal viewing, which are then not duplicated when the target words are processed foveally. Critically, although N400 effects were not modulated by task goals, a posteriorly distributed late positive component thought to reflect semantic integration processes was observed to semantic incongruities only in the plausibility judgment task. However, this effect was observed at a considerable delay, appearing only after words had moved into foveal vision. Our findings thus suggest that semantic access can be initiated in parafoveal vision, whereas central foveal vision may be necessary to enact higher-order (and task-dependent) integrative processing.

The peripheral preview effect with faces: Combined EEG and eye-tracking suggests multiple stages of trans-saccadic predictive and non-predictive processing

The world appears stable despite saccadic eye-movements. One possible explanation for this phenomenon is that the visual system predicts upcoming input across saccadic eye-movements based on peripheral preview of the saccadic target. We tested this idea using concurrent electroencephalography (EEG) and eye-tracking. Participants made cued saccades to peripheral upright or inverted face stimuli that changed orientation (invalid preview) or maintained orientation (valid preview) while the saccade was completed. Experiment 1 demonstrated better discrimination performance and a reduced fixation-locked N170 component (fN170) with valid than with invalid preview, demonstrating integration of pre- and post-saccadic information. Moreover, the early fixation-related potentials (FRP) showed a preview face inversion effect suggesting that some pre-saccadic input was represented in the brain until around 170 ms post fixation-onset. Experiment 2 replicated Experiment 1 and manipulated the proportion of valid and invalid trials to test whether the preview effect reflects context-based prediction across trials. A whole-scalp Bayes factor analysis showed that this manipulation did not alter the fN170 preview effect but did influence the face inversion effect before the saccade. The pre-saccadic inversion effect declined earlier in the mostly invalid block than in the mostly valid block, which is consistent with the notion of pre-saccadic expectations. In addition, in both studies, we found strong evidence for an interaction between the pre-saccadic preview stimulus and the post-saccadic target as early as 50 ms (Experiment 2) or 90 ms (Experiment 1) into the new fixation. These findings suggest that visual stability may involve three temporal stages: prediction about the saccadic target, integration of pre-saccadic and post-saccadic information at around 50-90 ms post fixation onset, and post-saccadic facilitation of rapid categorization.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

Parafoveal previews and lexical frequency in natural reading: Evidence from eye movements and fixation-related potentials

Participants' eye movements and electroencephalogram (EEG) signal were recorded as they read sentences displayed according to the gaze-contingent boundary paradigm. Two target words in each sentence were manipulated for lexical frequency (high vs. low frequency) and parafoveal preview of each target word (identical vs. string of random letters vs. string of Xs). Eye movement data revealed visual parafoveal-on-foveal (PoF) effects, as well as foveal visual and orthographic preview effects and word frequency effects. Fixation-related potentials (FRPs) showed visual and orthographic PoF effects as well as foveal visual and orthographic preview effects. Our results replicated the early preview positivity effect (Dimigen, Kliegl, & Sommer, 2012) in the X-string preview condition, and revealed different neural correlates associated with a preview comprised of a string of random letters relative to a string of Xs. The former effects seem likely to reflect difficulty associated with the integration of parafoveal and foveal information, as well as feature overlap, while the latter reflect inhibition, and potentially disruption, to processing underlying reading. Interestingly, and consistent with Kretzschmar, Schlesewsky, and Staub (2015), no frequency effect was reflected in the FRP measures. The findings provide insight into the neural correlates of parafoveal processing and written word recognition in reading and demonstrate the value of utilizing ecologically valid paradigms to study well established phenomena that occur as text is read naturally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Semantic anomaly detection in school-aged children during natural sentence reading - A study of fixation-related brain potentials

In this study, we investigated the effects of context-related semantic anomalies on the fixation-related brain potentials of 12-13-year-old Finnish children in grade 6 during sentence reading. The detection of such anomalies is typically reflected in the N400 event-related potential. We also examined whether the representation invoked by the sentence context extends to the orthographic representation level by replacing the final words of the sentence with an anomalous word neighbour of a plausible word. The eye-movement results show that the anomalous word neighbours of plausible words cause similar first-fixation and gaze duration reactions, as do other anomalous words. Similarly, we observed frontal negativity in the fixation-related potential of the unrelated anomalous words and in the anomalous word neighbours. This frontal negativity was larger in both anomalous conditions than in the response elicited by the plausible condition. We thus show that the brain successfully uses context to separate anomalous words from plausible words on a single letter level during free reading. From the P600 response of the scalp waveform, we observed that the P600 was delayed in the anomalous word neighbour condition. We performed group-level decomposition on the data with ICA (independent component analysis) and analysed the time course and source structure of the decomposed data. This analysis of decomposed brain signals not only confirmed the delay of the P600 response but also revealed that the frontal negativity concealed s more typical and separate N400 response, which was similarly delayed in the anomalous word neighbour condition, as was the P600 response. Source analysis of these independent components implicated the right frontal eye field as the cortical source for the frontal negativity and the middle temporal and parietal regions as cortical sources for the components resembling the N400 and P600 responses. We interpret the delays present in N400 and P600 responses to anomalous word neighbours to reflect competition with the representation of the plausible word just one letter different.

Processing of parafoveally presented words. An fMRI study

The present fMRI study investigated neural correlates of parafoveal preprocessing during reading and the type of information that is accessible from the upcoming - not yet fixated - word. Participants performed a lexical decision flanker task while the constraints imposed by the first three letters (the initial trigram) of parafoveally presented words were controlled. Behavioral results evidenced that the amount of information extracted from parafoveal stimuli, was affected by the difficulty of the foveal stimulus. Easy to process foveal stimuli (i.e., high frequency nouns) allowed parafoveal information to be extracted up to the lexical level. Conversely, when foveal stimuli were difficult to process (orthographically legal nonwords) only constraining trigrams modulated the task performance. Neuroimaging findings showed no effects of lexicality (i.e., difference between words and pseudowords) in the parafovea independently from the difficulty of the foveal stimulus. The constraints imposed by the initial trigrams, however, modulated the hemodynamic response in the left supramarginal gyrus. We interpreted the supramarginal activation as reflecting sublexical (phonological) processes. The missing parafoveal lexicality effect was discussed in relation to findings of experiments which observed effects of parafoveal semantic congruency on electrophysiological correlates.

Getting ahead of yourself: Parafoveal word expectancy modulates the N400 during sentence reading

An important question in the reading literature regards the nature of the semantic information readers can extract from the parafovea (i.e., the next word in a sentence). Recent eye-tracking findings have found a semantic parafoveal preview benefit under many circumstances, and findings from event-related brain potentials (ERPs) also suggest that readers can at least detect semantic anomalies parafoveally (Barber, Van der Meij, & Kutas, Psychophysiology, 50(1), 48-59, 2013). We use ERPs to ask whether fine-grained aspects of semantic expectancy can affect the N400 elicited by a word appearing in the parafovea. In an RSVP-with-flankers paradigm, sentences were presented word by word, flanked 2° bilaterally by the previous and upcoming words. Stimuli consisted of high constraint sentences that were identical up to the target word, which could be expected, unexpected but plausible, or anomalous, as well as low constraint sentences that were always completed with the most expected ending. Findings revealed an N400 effect to the target word when it appeared in the parafovea, which was graded with respect to the target's expectancy and congruency within the sentence context. Furthermore, when targets appeared at central fixation, this graded congruency effect was mitigated, suggesting that the semantic information gleaned from parafoveal vision functionally changes the semantic processing of those words when foveated.