The time course of predictability effects in reading: Evidence from a survival analysis of fixation durations

Prediction in reading: A review of predictability effects, their theoretical implications, and beyond

Historically, prediction during reading has been considered an inefficient and cognitively expensive processing mechanism given the inherently generative nature of language, which allows upcoming text to unfold in an infinite number of possible ways. This article provides an accessible and comprehensive review of the psycholinguistic research that, over the past 40 or so years, has investigated whether readers are capable of generating predictions during reading, typically via experiments on the effects of predictability (i.e., how well a word can be predicted from its prior context). Five theoretically important issues are addressed: What is the best measure of predictability? What is the functional relationship between predictability and processing difficulty? What stage(s) of processing does predictability affect? Are predictability effects ubiquitous? What processes do predictability effects actually reflect? Insights from computational models of reading about how predictability manifests itself to facilitate the reading of text are also discussed. This review concludes by arguing that effects of predictability can, to a certain extent, be taken as demonstrating evidence that prediction is an important but flexible component of real-time language comprehension, in line with broader predictive accounts of cognitive functioning. However, converging evidence, especially from concurrent eye-tracking and brain-imaging methods, is necessary to refine theories of prediction.

Word Frequency and Predictability Dissociate in Naturalistic Reading

Many studies of human language processing have shown that readers slow down at less frequent or less predictable words, but there is debate about whether frequency and predictability effects reflect separable cognitive phenomena: are cognitive operations that retrieve words from the mental lexicon based on sensory cues distinct from those that predict upcoming words based on context? Previous evidence for a frequency-predictability dissociation is mostly based on small samples (both for estimating predictability and frequency and for testing their effects on human behavior), artificial materials (e.g., isolated constructed sentences), and implausible modeling assumptions (discrete-time dynamics, linearity, additivity, constant variance, and invariance over time), which raises the question: do frequency and predictability dissociate in ordinary language comprehension, such as story reading? This study leverages recent progress in open data and computational modeling to address this question at scale. A large collection of naturalistic reading data (six datasets, >2.2 M datapoints) is analyzed using nonlinear continuous-time regression, and frequency and predictability are estimated using statistical language models trained on more data than is currently typical in psycholinguistics. Despite the use of naturalistic data, strong predictability estimates, and flexible regression models, results converge with earlier experimental studies in supporting dissociable and additive frequency and predictability effects.

Fine-grained time course of verb aspect processing

Sentence processing is known to be highly incremental. Speakers make incremental commitments as the sentence unfolds, dynamically updating their representations based on the smallest pieces of information from the incoming speech stream. Less is known about linguistic processing on the sub-word level, especially with regard to abstract grammatical information. This study employs the Visual World Paradigm to investigate the processing of grammatical aspect by Russian-speaking adults (n = 124). Aspectual information is encoded relatively early within the Russian verb which makes this an ideal testing ground to investigate the incrementality of grammatical processing on the sub-word level. Participants showed preference for pictures of ongoing events when they heard sentences involving Imperfective verbs, and for pictures of completed events when they heard sentences involving Perfective verbs. Crucially, the analysis of the participants’ eye-movements showed that they exhibited preference for the target picture already before they heard the end of the verb. Moreover, the latency of this effect depended on where the aspectual information was encoded within the verb. These results indicate that the processing and integration of grammatical aspect information can happen rapidly and incrementally on a fine-grained word-internal level. Methodologically, the study draws together a set of analytical techniques which can be fruitfully applied to the analysis of effect latencies in a wide range of studies within the Visual World eye-tracking paradigm.

Neurophysiological, Oculomotor, and Computational Modeling of Impaired Reading Ability in Schizophrenia

Schizophrenia (Sz) is associated with deficits in fluent reading ability that compromise functional outcomes. Here, we utilize a combined eye-tracking, neurophysiological, and computational modeling approach to analyze underlying visual and oculomotor processes. Subjects included 26 Sz patients (SzP) and 26 healthy controls. Eye-tracking and electroencephalography data were acquired continuously during the reading of passages from the Gray Oral Reading Tests reading battery, permitting between-group evaluation of both oculomotor activity and fixation-related potentials (FRP). Schizophrenia patients showed a marked increase in time required per word (d = 1.3, P < .0001), reflecting both a moderate increase in fixation duration (d = .7, P = .026) and a large increase in the total saccade number (d = 1.6, P < .0001). Simulation models that incorporated alterations in both lower-level visual and oculomotor function as well as higher-level lexical processing performed better than models that assumed either deficit-type alone. In neurophysiological analyses, amplitude of the fixation-related P1 potential (P1f) was significantly reduced in SzP (d = .66, P = .013), reflecting reduced phase reset of ongoing theta-alpha band activity (d = .74, P = .019). In turn, P1f deficits significantly predicted increased saccade number both across groups (P = .017) and within SzP alone (P = .042). Computational and neurophysiological methods provide increasingly important approaches for investigating sensory contributions to impaired cognition during naturalistic processing in Sz. Here, we demonstrate deficits in reading rate that reflect both sensory/oculomotor- and semantic-level impairments and that manifest, respectively, as alterations in saccade number and fixation duration. Impaired P1f generation reflects impaired fixation-related reset of ongoing brain rhythms and suggests inefficient information processing within the early visual system as a basis for oculomotor dyscontrol during fluent reading in Sz.

Fixation-related NIRS indexes retinotopic occipital processing of parafoveal preview during natural reading

While word frequency and predictability effects have been examined extensively, any evidence on interactive effects as well as parafoveal influences during whole sentence reading remains inconsistent and elusive. Novel neuroimaging methods utilize eye movement data to account for the hemodynamic responses of very short events such as fixations during natural reading. In this study, we used the rapid sampling frequency of near-infrared spectroscopy (NIRS) to investigate neural responses in the occipital and orbitofrontal cortex to word frequency and predictability. We observed increased activation in the right ventral occipital cortex when the fixated word N was of low frequency, which we attribute to an enhanced cost during saccade planning. Importantly, unpredictable (in contrast to predictable) low frequency words increased the activity in the left dorsal occipital cortex at the fixation of the preceding word N-1, presumably due to an upcoming breach of top-down modulated expectation. Opposite to studies that utilized a serial presentation of words (e.g. Hofmann et al., 2014), we did not find such an interaction in the orbitofrontal cortex, implying that top-down timing of cognitive subprocesses is not required during natural reading. We discuss the implications of an interactive parafoveal-on-foveal effect for current models of eye movements.

Do Individuals with Autism Change Their Reading Behavior to Adapt to Errors in the Text?

Reading monitoring is poorly explored, but it may have an impact on well-documented reading comprehension difficulties in autism. This study explores reading monitoring through the impact of instructions and different error types on reading behavior. Individuals with autism and matched controls read correct sentences and sentences containing orthographic and semantic errors. Prior to the task, participants were given instructions either to focus on semantic or orthographic errors. Analysis of eye-movements showed that the group with autism, differently from controls, were less influenced by the error's type in the regression-out to-error measure, showing less change in eye-movements behavior between error types. Individuals with autism might find it more difficult to adapt their reading strategies to various reading materials and task demands.

The time course of age-of-acquisition effects on eye movements during reading: Evidence from survival analyses

Adults process words that are rated as being learned earlier in life faster than words that are rated as being acquired later in life. This age-of-acquisition (AoA) effect has been observed in a variety of word-recognition tasks when word frequency is controlled. AoA has also previously been found to influence fixation durations when words are embedded into sentences and eye movements are recorded. However, the time course of AoA effects during reading has been inconsistent across studies. The current study further explored the time course of AoA effects on distributions of first-fixation durations during reading. Early and late acquired words were embedded into matched neutral sentence frames. Participants read the sentences while their eye movements were recorded. AoA effects were observed in both early and late fixation duration measures, suggesting that AoA has an early and long-lasting effect on word-recognition processes during reading. Survival analysis revealed that the earliest discernable effect of AoA on distributions of first-fixation durations emerged beginning at 158 ms. This rapid influence of AoA was confirmed through the use of Vincentile plots, which demonstrated that the effect of AoA occurred early and was relatively consistent across the distribution of fixations. This pattern of results provides support for the direct lexical-control hypothesis, as well as the viewpoint that AoA may exert an influence at multiple loci within the mental lexicon.

On using distributional analysis techniques for determining the onset of the influence of experimental variables

Much of the investigation of eye-movement control in visual cognition has focused on the influence of experimental variables on mean fixation durations. In the present paper we explored the convergence between two distributional analysis techniques that were recently introduced in this domain. First, Staub, White, Drieghe, Hollway and Rayner, (2010) proposed fitting the ex-Gaussian distribution to individual participants' data in order to ascertain whether a variable has a rapid or a slow influence on fixation durations. Second, the Divergence Point Analysis (DPA) procedure was introduced by Reingold, Reichle, Glaholt and Sheridan (2012, Reingold & Sheridan, 2014) in order to determine more precisely the earliest discernible impact of a variable on the distribution of fixation durations by contrasting survival curves across two experimental conditions and determining the point at which the two curves begin to diverge. In the present paper we introduced a new version of the DPA procedure which is based on ex-Gaussian fitting. We evaluated this procedure by re-analysing data obtained in previous empirical investigations as well as by conducting a simulation study. We demonstrated that the new ex-Gaussian DPA technique produced estimates that were consistent with estimates produced by prior versions of DPA procedure, and in the present simulation, the ex-Gaussian DPA procedure produced somewhat more accurate individual participant divergence point estimates. Based on the present findings we also suggest guidelines for best practices in the use of DPA techniques.

The Word Frequency Effect on Saccade Targeting during Chinese Reading: Evidence from a Survival Analysis of Saccade Length

Our study employs distributional analysis (i.e., survival analysis) to examine how the frequency of target words influences saccade lengths into and out of these target words in Chinese reading. The results of survival analysis indicate the survival curves in the high- and low-frequency conditions diverge for a short saccade length, with more than 80% of the lengths of incoming and outgoing saccades being larger than the divergence points. These results as well as simulations using the novel Dynamic-adjustment Model of saccadic targeting (Liu et al., 2016) are consistent with previous mean-based results and provide more precise information to support this novel model. The implications for saccade target selection during the reading of Chinese are discussed.

An Analysis of the Time Course of Lexical Processing During Reading

Reingold, Reichle, Glaholt, and Sheridan (2012) reported a gaze-contingent eye-movement experiment in which survival-curve analyses were used to examine the effects of word frequency, the availability of parafoveal preview, and initial fixation location on the time course of lexical processing. The key results of these analyses suggest that lexical processing begins very rapidly (after approximately 120 ms) and is supported by substantial parafoveal processing (more than 100 ms). Because it is not immediately obvious that these results are congruent with the theoretical assumption that words are processed and identified in a strictly serial manner, we attempted to simulate the experiment using the E-Z Reader model of eye-movement control (Reichle, 2011). These simulations were largely consistent with the empirical results, suggesting that parafoveal processing does play an important functional role by allowing lexical processing to occur rapidly enough to mediate direct control over when the eyes move during reading.

Estimating the divergence point: a novel distributional analysis procedure for determining the onset of the influence of experimental variables

The divergence point analysis procedure is aimed at obtaining an estimate of the onset of the influence of an experimental variable on response latencies (e.g., fixation duration, reaction time). The procedure involves generating survival curves for two conditions, and using a bootstrapping technique to estimate the timing of the earliest discernible divergence between curves. In the present paper, several key extensions for this procedure were proposed and evaluated by conducting simulations and by reanalyzing data from previous studies. Our findings indicate that the modified versions of the procedure performed substantially better than the original procedure under conditions of low experimental power. Furthermore, unlike the original procedure, the modified procedures provided divergence point estimates for individual participants and permitted testing the significance of the difference between estimates across conditions. The advantages of the modified procedures are illustrated, the theoretical and methodological implications are discussed, and promising future directions are outlined.

Expert vs. novice differences in the detection of relevant information during a chess game: evidence from eye movements

The present study explored the ability of expert and novice chess players to rapidly distinguish between regions of a chessboard that were relevant to the best move on the board, and regions of the board that were irrelevant. Accordingly, we monitored the eye movements of expert and novice chess players, while they selected white's best move for a variety of chess problems. To manipulate relevancy, we constructed two different versions of each chess problem in the experiment, and we counterbalanced these versions across participants. These two versions of each problem were identical except that a single piece was changed from a bishop to a knight. This subtle change reversed the relevancy map of the board, such that regions that were relevant in one version of the board were now irrelevant (and vice versa). Using this paradigm, we demonstrated that both the experts and novices spent more time fixating the relevant relative to the irrelevant regions of the board. However, the experts were faster at detecting relevant information than the novices, as shown by the finding that experts (but not novices) were able to distinguish between relevant and irrelevant information during the early part of the trial. These findings further demonstrate the domain-related perceptual processing advantage of chess experts, using an experimental paradigm that allowed us to manipulate relevancy under tightly controlled conditions.

Individual differences in fixation duration distributions in reading

The present study investigated the relationship between the location and skew of an individual reader's fixation duration distribution. The ex-Gaussian distribution was fit to eye fixation data from 153 subjects in five experiments, four previously presented and one new. The τ parameter was entirely uncorrelated with the μ and σ parameters; by contrast, there was a modest positive correlation between these parameters for lexical decision and speeded pronunciation response times. The conclusion that, for fixation durations, the degree of skew is uncorrelated with the location of the distribution's central tendency was also confirmed nonparametrically, by examining vincentile plots for subgroups of subjects. Finally, the stability of distributional parameters for a given subject was demonstrated to be relatively high. Taken together with previous findings of selective influence on the μ parameter of the fixation duration distribution, the present results suggest that in reading, the location and the skew of the fixation duration distribution may reflect functionally distinct processes. The authors speculate that the skew parameter may specifically reflect the frequency of processing disruption.