Out of the corner of my eye: Foveal semantic load modulates parafoveal processing in reading

Parafoveal Word Frequency Does Not Modulate the Effect of Foveal Load on Preview in Chinese Reading: Evidence from Eye Movements

The foveal load effect is one of the most fundamental effects in reading psychology, and also one of the most controversial issues in recent years. The foveal load effect refers to the phenomenon that the difficulty of foveal processing affects parafoveal preview. In Chinese reading, whether the foveal load effect exists, as well as whether this effect is modulated by parafoveal word frequency, remains unclear. In this study, the eye-tracking technique was used to track the eye movements of 48 subjects. Utilized the boundary paradigm with single-character words as parafoveal words, the present study manipulated foveal word frequency (high and low), parafoveal word frequency (high and low), and two types of preview (identical preview and pseudocharacter preview) to investigate these questions. The results revealed that the foveal word frequency does not influence preview, suggesting the absence of the foveal load effect when using single-character words as parafoveal words. Furthermore, parafoveal word frequency does not modulate the effect of the foveal load on the preview. This empirical evidence contributes to refining the understanding of the Chinese reading model.

Patterns of hemispheric asymmetry provide evidence dissociating the semantic and syntactic P600

To understand how neural networks in the left (LH) and right (RH) cerebral hemispheres contribute to different aspects of language comprehension, in two experiments we measured event-related potentials (ERPs) as right-handed participants read sentences, some of which contained morphosyntactic and thematic role violations. Replicating prior work (Kuperberg et al., 2006), in Experiment 1 thematic role violations elicited both an N400 and a (semantic) P600 effect. Morphosyntactic violations elicited effects that differed as a function of participants' familial sinistrality (the presence [FS+] or absence [FS-] of a left-handed biological relative): FS+ participants showed a (syntactic) P600 effect whereas FS- participants showed a biphasic N400 and P600 response. To assess whether this difference reflects different underlying patterns of lateralization, in Experiment 2 target words were presented using visual half-field (VF) presentation. Indeed, for morphosyntactic violations, the FS- group elicited an asymmetric pattern, showing a P600 effect only with LH-biased presentation and an N400 effect in both VFs (cf. Lee and Federmeier, 2015). In contrast, FS+ participants showed a bilateral (N400-only) response pattern. This provides further evidence of FS-based differences in hemispheric contributions to syntactic processing. Strikingly, we found that, when lateralized, thematic role violations did not elicit a P600 effect, suggesting that this effect requires contributions from both hemispheres. The different response patterns for morphosyntactic and thematic role animacy violations across FS and VF also point to a processing difference in the comprehension mechanisms underlying the semantic and syntactic P600, which had heretofore been assumed to be variants of the same component.

Connecting and considering: Electrophysiology provides insights into comprehension

The ability to rapidly and systematically access knowledge stored in long-term memory in response to incoming sensory information-that is, to derive meaning from the world-lies at the core of human cognition. Research using methods that can precisely track brain activity over time has begun to reveal the multiple cognitive and neural mechanisms that make this possible. In this article, I delineate how a process of connecting affords an effortless, continuous infusion of meaning into human perception. In a relatively invariant time window, uncovered through studies using the N400 component of the event-related potential, incoming sensory information naturally induces a graded landscape of activation across long-term semantic memory, creating what might be called "proto-concepts". Connecting can be (but is not always) followed by a process of further considering those activations, wherein a set of more attentionally demanding "active comprehension" mechanisms mediate the selection, augmentation, and transformation of the initial semantic representations. The result is a limited set of more stable bindings that can be arranged in time or space, revised as needed, and brought to awareness. With this research, we are coming closer to understanding how the human brain is able to fluidly link sensation to experience, to appreciate language sequences and event structures, and, sometimes, to even predict what might be coming up next.

Effects of word predictability on eye movements during Arabic reading

Contextual predictability influences both the probability and duration of eye fixations on words when reading Latinate alphabetic scripts like English and German. However, it is unknown whether word predictability influences eye movements in reading similarly for Semitic languages like Arabic, which are alphabetic languages with very different visual and linguistic characteristics. Such knowledge is nevertheless important for establishing the generality of mechanisms of eye-movement control across different alphabetic writing systems. Accordingly, we investigated word predictability effects in Arabic in two eye-movement experiments. Both produced shorter fixation times for words with high compared to low predictability, consistent with previous findings. Predictability did not influence skipping probabilities for (four- to eight-letter) words of varying length and morphological complexity (Experiment 1). However, it did for short (three- to four-letter) words with simpler structures (Experiment 2). We suggest that word-skipping is reduced, and affected less by contextual predictability, in Arabic compared to Latinate alphabetic reading, because of specific orthographic and morphological characteristics of the Arabic script.

Semantic parafoveal processing in natural reading: Insight from fixation-related potentials & eye movements

Prior research suggests that we may access the meaning of parafoveal words during reading. We explored how semantic-plausibility parafoveal processing takes place in natural reading through the co-registration of eye movements (EM) and fixation-related potentials (FRPs), using the boundary paradigm. We replicated previous evidence of semantic parafoveal processing from highly controlled reading situations, extending their findings to more ecologically valid reading scenarios. Additionally, and exploring the time-course of plausibility preview effects, we found distinct but complementary evidence from EM and FRPs measures. FRPs measures, showing a different trend than EM evidence, revealed that plausibility preview effects may be long-lasting. We highlight the importance of a co-registration set-up in ecologically valid scenarios to disentangle the mechanisms related to semantic-plausibility parafoveal processing.

Text Captioning Buffers Against the Effects of Background Noise and Hearing Loss on Memory for Speech

OBJECTIVE

Everyday speech understanding frequently occurs in perceptually demanding environments, for example, due to background noise and normal age-related hearing loss. The resulting degraded speech signals increase listening effort, which gives rise to negative downstream effects on subsequent memory and comprehension, even when speech is intelligible. In two experiments, we explored whether the presentation of realistic assistive text captioned speech offsets the negative effects of background noise and hearing impairment on multiple measures of speech memory.

DESIGN

In Experiment 1, young normal-hearing adults (N = 48) listened to sentences for immediate recall and delayed recognition memory. Speech was presented in quiet or in two levels of background noise. Sentences were either presented as speech only or as text captioned speech. Thus, the experiment followed a 2 (caption vs no caption) × 3 (no noise, +7 dB signal-to-noise ratio, +3 dB signal-to-noise ratio) within-subjects design. In Experiment 2, a group of older adults (age range: 61 to 80, N = 31), with varying levels of hearing acuity completed the same experimental task as in Experiment 1. For both experiments, immediate recall, recognition memory accuracy, and recognition memory confidence were analyzed via general(ized) linear mixed-effects models. In addition, we examined individual differences as a function of hearing acuity in Experiment 2.

RESULTS

In Experiment 1, we found that the presentation of realistic text-captioned speech in young normal-hearing listeners showed improved immediate recall and delayed recognition memory accuracy and confidence compared with speech alone. Moreover, text captions attenuated the negative effects of background noise on all speech memory outcomes. In Experiment 2, we replicated the same pattern of results in a sample of older adults with varying levels of hearing acuity. Moreover, we showed that the negative effects of hearing loss on speech memory in older adulthood were attenuated by the presentation of text captions.

CONCLUSIONS

Collectively, these findings strongly suggest that the simultaneous presentation of text can offset the negative effects of effortful listening on speech memory. Critically, captioning benefits extended from immediate word recall to long-term sentence recognition memory, a benefit that was observed not only for older adults with hearing loss but also young normal-hearing listeners. These findings suggest that the text captioning benefit to memory is robust and has potentially wide applications for supporting speech listening in acoustically challenging environments.

Event-related brain potentials reveal how multiple aspects of semantic processing unfold across parafoveal and foveal vision during sentence reading

Recent event-related brain potential (ERP) experiments have demonstrated parafoveal N400 expectancy and congruity effects, showing that semantic information can be accessed from words in parafoveal vision (a conclusion also supported by some eye-tracking work). At the same time, it is unclear how higher-order integrative aspects of language comprehension unfold across the visual field during reading. In the current study, we recorded ERPs in a parafoveal flanker paradigm, while readers were instructed to read passively for comprehension or to judge the plausibility of sentences in which target words varied in their semantic expectancy and congruity. We directly replicated prior work showing graded N400 effects for parafoveal viewing, which are then not duplicated when the target words are processed foveally. Critically, although N400 effects were not modulated by task goals, a posteriorly distributed late positive component thought to reflect semantic integration processes was observed to semantic incongruities only in the plausibility judgment task. However, this effect was observed at a considerable delay, appearing only after words had moved into foveal vision. Our findings thus suggest that semantic access can be initiated in parafoveal vision, whereas central foveal vision may be necessary to enact higher-order (and task-dependent) integrative processing.

Modulation of parafoveal word processing by cognitive load during modified visual search tasks

During visual search for simple items, the amount of information that can be processed in parafoveal vision depends on the cognitive resources that are available. However, whether this applies to the semantic processing of words remains controversial. This work was designed to manipulate simultaneously two sources of cognitive load to study their impact on the depth of parafoveal word processing during a modified visual search task. The participants had to search for target words among parafoveally presented semantic, orthographic or target-unrelated distractor words while their electroencephalogram was recorded. The task-related load was manipulated by either giving target words in advance (literal task) or giving only a semantic clue to define them (categorical task). The foveal load was manipulated by displaying either a word or hash symbols at the centre of the screen. Parafoveal orthographic and semantic distractors had an impact on the early event-related potential component P2a only in the literal task and when hash symbols were displayed at the fovea, i.e., when both the task-related and foveal loads were low. The data show that all sources of cognitive load must be considered to understand how parafoveal words are processed in visual search contexts.

Spill the load: Mixed evidence for a foveal load effect, reliable evidence for a spillover effect in eye-movement control during reading

It has been hypothesized that the processing difficulty of the fixated word (i.e., "foveal load") modulates the amount of parafoveal preprocessing of the next word. Evidence for the hypothesis has been provided by the application of parafoveal masks within the boundary paradigm. Other studies that applied alternative means of manipulating the parafoveal preview (i.e., visual degradation) could not replicate the effect of foveal load. The present study examined the effect of foveal load by directly comparing the application of parafoveal masks (Exp. 1) with the alternative manipulation of visually degrading the parafoveal preview (Exp. 2) in adult readers. Contrary to expectation, we did not find the foveal-load interaction in the first experiment with traditional letter masks. We did, however, find the expected interaction in the second experiment with visually degraded previews. Both experiments revealed a spillover effect indicating that the processing of a word is not (always) fully completed when the reader already fixates the next word (i.e., processing "spills over" to the next word). The implications for models of eye-movement control in reading are discussed.

The P3b and P600(s): Positive contributions to language comprehension

Since its discovery in the 1960s, the P300 has been contributing both directly and indirectly to language research. Perhaps most notably, it has been suggested that the P600, an ERP component that was first characterized in the context of syntactic processing, could be a variant of the P3b subcomponent of the P300. Here, we review studies on both sides of the debate. We also review the "semantic P600," a positivity with a similar time course and distribution to the P600 seen for syntactic manipulations but that is obtained in response to some types of semantic anomalies. Because most current theories of the P600 try to account for both the syntactic and the semantic variant, linking the syntactic P600 to the P3b might also imply a similar link for the semantic P600. However, we describe emerging research in our lab that casts doubt on the idea that the syntactic P600 and the semantic P600 are the same effect. We argue that grouping ERP responses primarily by domain (language vs. nonlanguage) is likely to be misleading and suggest alternative ways of determining whether ERP effects reflect similar or different processing mechanisms.

Processing of parafoveally presented words. An fMRI study

The present fMRI study investigated neural correlates of parafoveal preprocessing during reading and the type of information that is accessible from the upcoming - not yet fixated - word. Participants performed a lexical decision flanker task while the constraints imposed by the first three letters (the initial trigram) of parafoveally presented words were controlled. Behavioral results evidenced that the amount of information extracted from parafoveal stimuli, was affected by the difficulty of the foveal stimulus. Easy to process foveal stimuli (i.e., high frequency nouns) allowed parafoveal information to be extracted up to the lexical level. Conversely, when foveal stimuli were difficult to process (orthographically legal nonwords) only constraining trigrams modulated the task performance. Neuroimaging findings showed no effects of lexicality (i.e., difference between words and pseudowords) in the parafovea independently from the difficulty of the foveal stimulus. The constraints imposed by the initial trigrams, however, modulated the hemodynamic response in the left supramarginal gyrus. We interpreted the supramarginal activation as reflecting sublexical (phonological) processes. The missing parafoveal lexicality effect was discussed in relation to findings of experiments which observed effects of parafoveal semantic congruency on electrophysiological correlates.

Influences on and consequences of parafoveal preview in reading

During reading, information is extracted from upcoming words to the right of the currently fixated word, which facilitates recognition of those words when they are later fixated. According to the foveal load hypothesis (Henderson & Ferreira, 1990), this parafoveal preview benefit depends on how difficult the currently fixated word is to recognize. Furthermore, there is evidence that the influence of lexical variables (frequency and predictability) on word processing changes when no preview of that word is available. The present study reports two moving-window experiments in which the upcoming word to the right of fixation was either included in or excluded from the window. Through this manipulation, accurate parafoveal information was either available or not for each word in the paragraph. Two critical interactions between preview condition and lexical variables were observed. First, the word frequency at word N was found to be the primary influence on the amount of preview benefit obtained at word N+1, consistent with the foveal load hypothesis. Second, denial of preview eliminated the word predictability effect. These findings have implications for models of eye movement control in reading.

Getting ahead of yourself: Parafoveal word expectancy modulates the N400 during sentence reading

An important question in the reading literature regards the nature of the semantic information readers can extract from the parafovea (i.e., the next word in a sentence). Recent eye-tracking findings have found a semantic parafoveal preview benefit under many circumstances, and findings from event-related brain potentials (ERPs) also suggest that readers can at least detect semantic anomalies parafoveally (Barber, Van der Meij, & Kutas, Psychophysiology, 50(1), 48-59, 2013). We use ERPs to ask whether fine-grained aspects of semantic expectancy can affect the N400 elicited by a word appearing in the parafovea. In an RSVP-with-flankers paradigm, sentences were presented word by word, flanked 2° bilaterally by the previous and upcoming words. Stimuli consisted of high constraint sentences that were identical up to the target word, which could be expected, unexpected but plausible, or anomalous, as well as low constraint sentences that were always completed with the most expected ending. Findings revealed an N400 effect to the target word when it appeared in the parafovea, which was graded with respect to the target's expectancy and congruency within the sentence context. Furthermore, when targets appeared at central fixation, this graded congruency effect was mitigated, suggesting that the semantic information gleaned from parafoveal vision functionally changes the semantic processing of those words when foveated.

The Effect of Emotional State on the Processing of Morphosyntactic and Semantic Reversal Anomalies in Japanese: Evidence from Event-Related Brain Potentials

The present study examined the locus responsible for the effect of emotional state on sentence processing in healthy native speakers of Japanese, using event-related brain potentials. The participants were induced into a happy, neutral, or sad mood and then subjected to electroencephalogram recording during which emotionally neutral sentences, including grammatical sentences (e.g. window-NOM close vi, 'The window closes.'), morphosyntactically-violated sentences (e.g. window-ACC close vi, Lit. 'Close the window.'), and semantically-reversed sentences (e.g. window-NOM close vt, 'The window closes pro.') were presented. The results of the ERP experiment demonstrated that while the P600 effect elicited by morphosyntactic violation was not modulated by mood, the P600 effect elicited by semantic reversal anomaly was observed only in participants previously induced into a happy mood. The LAN and N400 were not sensitive to the participants' transient emotional state. These results suggest intact memory access and impaired integration of syntactic and semantic information in individuals in a sad mood.

Event-related brain potentials reveal age-related changes in parafoveal-foveal integration during sentence processing

Normative aging is associated with deficits in visual acuity and cognitive control that impact the allocation of visual attention, but little is known about how those changes affect information extraction and integration during visual language comprehension in older adulthood. In the current study, we used a visual hemi-field flanker RSVP paradigm with event-related brain potentials to study how older readers process fine-grained aspects of semantic expectancy in parafoveal and foveal vision. Stimuli consisted of high constraint sentences with expected, unexpected but plausible, or anomalous parafoveal target words, as well as low constraint sentences with neutral but expected target words. Older adults showed graded parafoveal N400 effects that were strikingly similar to younger readers, indicating intact parafoveal semantic processing. However, whereas young adults were able to use this parafoveal pre-processing to facilitate subsequent foveal viewing, resulting in a reduced foveal N400 effect, older adults were not able to. Instead, older adults re-processed the semantics of words in foveal vision, resulting in a larger foveal N400 effect relative to the young. Collectively, our findings suggest that although parafoveal semantic processing per se is preserved in aging, there exists an age-related deficit in the ability to rapidly integrate parafoveal and foveal visual semantic representations.

The language of arithmetic across the hemispheres: An event-related potential investigation

Arithmetic expressions, like verbal sentences, incrementally lead readers to anticipate potential appropriate completions. Existing work in the language domain has helped us understand how the two hemispheres differently participate in and contribute to the cognitive process of sentence reading, but comparatively little work has been done with mathematical equation processing. In this study, we address this gap by examining the ERP response to provided answers to simple multiplication problems, which varied both in levels of correctness (given an equation context) and in visual field of presentation (joint attention in central presentation, or biased processing to the left or right hemisphere through contralateral visual field presentation). When answers were presented to any of the visual fields (hemispheres), there was an effect of correctness prior to the traditional N400 timewindow, which we interpret as a P300 in response to a detected target item (the correct answer). In addition to this response, equation answers also elicited a late positive complex (LPC) for incorrect answers. Notably, this LPC effect was most prominent in the left visual field (right hemisphere), and it was also sensitive to the confusability of the wrong answer - incorrect answers that were closely related to the correct answer elicited a smaller LPC. This suggests a special, prolonged role for the right hemisphere during answer evaluation.

Pace Yourself: Intraindividual Variability in Context Use Revealed by Self-paced Event-related Brain Potentials

Event-related brain potentials (ERPs) have revealed multiple mechanisms by which contextual constraints impact language processing. At the same time, little work has examined the trial-to-trial dynamics of context use in the brain. In the current study, we probed intraindividual variability in behavioral and neural indices of context processing during reading. In a concurrent self-paced reading and ERP paradigm, participants read sentences that were either strongly or weakly constraining completed with an expected or unexpected target word. Our findings revealed substantial within-subject variability in behavioral and neural responses to contextual constraints. First, context-based amplitude reductions of the N400, a component linked to semantic memory access, were largest among trials eliciting the slowest RTs. Second, the RT distribution of unexpected words in strongly constraining contexts was positively skewed, reflecting an increased proportion of very slow RTs to trials that violated semantic predictions. Among those prediction-violating trials eliciting faster RTs, a late sustained anterior positivity was observed. However, among trials producing the differentially slowed RTs to prediction violations, we observed a markedly earlier effect of constraint in the form of an anterior N2, a component linked to conflict resolution and the cognitive control of behavior. The current study provides the first neurophysiological evidence for the direct role of cognitive control functions in the volitional control of reading. Collectively, our findings suggest that context use varies substantially within individual participants and that coregistering behavioral and neural indices of online sentence processing offers a window into these single-item dynamics.