Using CRISP to model global characteristics of fixation durations in scene viewing and reading with a common mechanism

Eye movements during text reading align with the rate of speech production

Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3 and 5.5 Hz, reflecting the production and processing of linguistic information chunks (syllables and words) every ~200 ms. Interestingly, ~200 ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ~5 Hz. A subsequent meta-analysis of 142 studies from 14 languages replicates this result and shows that sampling frequencies vary across languages between 3.9 Hz and 5.2 Hz. This variation systematically depends on the complexity of the writing systems (character-based versus alphabetic systems and orthographic transparency). Finally, we empirically demonstrate a positive correlation between speech spectrum and eye movement sampling in low-skilled non-native readers, with tentative evidence from post hoc analysis suggesting the same relationship in low-skilled native readers. On the basis of this convergent evidence, we propose that during reading, our brain's linguistic processing systems imprint a preferred processing rate-that is, the rate of spoken language production and perception-onto the oculomotor system.

Fixation-related potentials in naming speed: A combined EEG and eye-tracking study on children with dyslexia

OBJECTIVE

We combined electroencephalography (EEG) and eye-tracking recordings to examine the underlying factors elicited during the serial Rapid-Automatized Naming (RAN) task that may differentiate between children with dyslexia (DYS) and chronological age controls (CAC).

METHODS

Thirty children with DYS and 30 CAC (M_age = 9.79 years; age range 7.6 through 12.1 years) performed a set of serial RAN tasks. We extracted fixation-related potentials (FRPs) under phonologically similar (rime-confound) or visually similar (resembling lowercase letters) and dissimilar (non-confounding and discrete uppercase letters, respectively) control tasks.

RESULTS

Results revealed significant differences in FRP amplitudes between DYS and CAC groups under the phonologically similar and phonologically non-confounding conditions. No differences were observed in the case of the visual conditions. Moreover, regression analysis showed that the average amplitude of the extracted components significantly predicted RAN performance.

CONCLUSION

FRPs capture neural components during the serial RAN task informative of differences between DYS and CAC and establish a relationship between neurocognitive processes during serial RAN and dyslexia.

SIGNIFICANCE

We suggest our approach as a methodological model for the concurrent analysis of neurophysiological and eye-gaze data to decipher the role of RAN in reading.

Fixation durations in natural scene viewing are guided by peripheral scene content

Fixation durations provide insights into processing demands. We investigated factors controlling fixation durations during scene viewing in two experiments. In Experiment 1, we tested the degree to which fixation durations adapt to global scene processing difficulty by manipulating the contrast (from original contrast to isoluminant) and saturation (original vs. grayscale) of the entire scene. We observed longer fixation durations for lower levels of contrast, and longer fixation durations for grayscale than for color scenes. Thus fixation durations were globally slowed as visual information became more and more degraded, making scene processing increasingly more difficult. In Experiment 2, we investigated two possible sources for this slow-down. We used "checkerboard" stimuli in which unmodified patches alternated with patches from which luminance information had been removed (isoluminant patches). Fixation durations showed an inverted immediacy effect (longer, rather than shorter, fixation durations on unmodified patches) along with a parafoveal-on-foveal effect (shorter fixation durations, when an unmodified patch was fixated next). This effect was stronger when the currently fixated patch was isoluminant as opposed to unmodified. Our results suggest that peripheral scene information substantially affects fixation durations and are consistent with the notion of competition among the current and potential future fixation locations.

Literacy skill and intra-individual variability in eye-fixation durations during reading: Evidence from a diverse community-based adult sample

To understand the effects of literacy on fundamental processes involved in reading, we report a secondary data analysis examining individual differences in global eye-movement measures and first-pass eye-movement distributions in a diverse sample of community-dwelling adults aged 16 to 64. Participants (n = 80) completed an assessment battery probing verbal and non-verbal cognitive abilities and read simple two-sentence passages while their eye movements were recorded. Analyses were focused on characterising the effects of literacy skill on both global indices of eye-fixation distributions and distributional differences in the sensitivity to lexical features. Global reading measures showed that lower literate adults read more slowly on average. However, distributional analyses of fixation durations revealed that the first-pass fixation durations of adults with lower literacy skill were not slower in general (i.e., there was no shift in the fixation duration distribution among lower literate adults). Instead, lower literacy was associated with greater intra-individual variability in first-pass fixation durations, including an increased proportion of extremely long fixations, differentially skewing the distribution of both first-fixation and gaze durations. Exploratory repeated-measures quantile regression analyses of gaze duration revealed differentially greater influences of word length among lower literate readers and greater activation of phonological and orthographic neighbours among higher literate readers, particularly in the tail of the distribution. Collectively, these findings suggest that literacy skill in adulthood is associated with systematic differences in both global and lexically driven eye-movement control during reading.

Examining the neural and cognitive processes that underlie reading through naming speed tasks

We combined fMRI with eye tracking and speech recording to examine the neural and cognitive mechanisms that underlie reading. To simplify the study of the complex processes involved during reading, we used naming speed (NS) tasks (also known as rapid automatized naming or RAN) as a focus for this study, in which average reading right-handed adults named sets of stimuli (letters or objects) as quickly and accurately as possible. Due to the possibility of spoken output during fMRI studies creating motion artifacts, we employed both an overt session and a covert session. When comparing the two sessions, there were no significant differences in behavioral performance, sensorimotor activation (except for regions involved in the motor aspects of speech production) or activation in regions within the left-hemisphere-dominant neural reading network. This established that differences found between the tasks within the reading network were not attributed to speech production motion artifacts or sensorimotor processes. Both behavioral and neuroimaging measures showed that letter naming was a more automatic and efficient task than object naming. Furthermore, specific manipulations to the NS tasks to make the stimuli more visually and/or phonologically similar differentially activated the reading network in the left hemisphere associated with phonological, orthographic and orthographic-to-phonological processing, but not articulatory/motor processing related to speech production. These findings further our understanding of the underlying neural processes that support reading by examining how activation within the reading network differs with both task performance and task characteristics.

Extrafoveal attentional capture by object semantics

There is ongoing debate on whether object meaning can be processed outside foveal vision, making semantics available for attentional guidance. Much of the debate has centred on whether objects that do not fit within an overall scene draw attention, in complex displays that are often difficult to control. Here, we revisited the question by reanalysing data from three experiments that used displays consisting of standalone objects from a carefully controlled stimulus set. Observers searched for a target object, as per auditory instruction. On the critical trials, the displays contained no target but objects that were semantically related to the target, visually related, or unrelated. Analyses using (generalized) linear mixed-effects models showed that, although visually related objects attracted most attention, semantically related objects were also fixated earlier in time than unrelated objects. Moreover, semantic matches affected the very first saccade in the display. The amplitudes of saccades that first entered semantically related objects were larger than 5° on average, confirming that object semantics is available outside foveal vision. Finally, there was no semantic capture of attention for the same objects when observers did not actively look for the target, confirming that it was not stimulus-driven. We discuss the implications for existing models of visual cognition.

Task-Related Differences in Eye Movements in Individuals With Aphasia

Background: Neurotypical young adults show task-based modulation and stability of their eye movements across tasks. This study aimed to determine whether persons with aphasia (PWA) modulate their eye movements and show stability across tasks similarly to control participants.

Methods: Forty-eight PWA and age-matched control participants completed four eye-tracking tasks: scene search, scene memorization, text-reading, and pseudo-reading.

Results: Main effects of task emerged for mean fixation duration, saccade amplitude, and standard deviations of each, demonstrating task-based modulation of eye movements. Group by task interactions indicated that PWA produced shorter fixations relative to controls. This effect was most pronounced for scene memorization and for individuals who recently suffered a stroke. PWA produced longer fixations, shorter saccades, and less variable eye movements in reading tasks compared to controls. Three-way interactions of group, aphasia subtype, and task also emerged. Text-reading and scene memorization were particularly effective at distinguishing aphasia subtype. Persons with anomic aphasia showed a reduction in reading saccade amplitudes relative to their respective control group and other PWA. Persons with conduction/Wernicke’s aphasia produced shorter scene memorization fixations relative to controls or PWA of other subtypes, suggesting a memorization specific effect. Positive correlations across most tasks emerged for fixation duration and did not significantly differ between controls and PWA.

Conclusion: PWA generally produced shorter fixations and smaller saccades relative to controls particularly in scene memorization and text-reading, respectively. The effect was most pronounced recently after a stroke. Selectively in reading tasks, PWA produced longer fixations and shorter saccades relative to controls, consistent with reading difficulty. PWA showed task-based modulation of eye movements, though the pattern of results was somewhat abnormal relative to controls. All subtypes of PWA also demonstrated task-based modulation of eye movements. However, persons with anomic aphasia showed reduced modulation of saccade amplitude and smaller reading saccades, possibly to improve reading comprehension. Controls and PWA generally produced stabile fixation durations across tasks and did not differ in their relationship across tasks. Overall, these results suggest there is potential to differentiate among PWA with varying subtypes and from controls using eye movement measures of task-based modulation, especially reading and scene memorization tasks.

Neural correlates of individual differences in fixation duration during natural reading

Reading requires integration of language and cognitive processes with attention and eye movement control. Individuals differ in their reading ability, but little is known about the neurocognitive processes associated with these individual differences. To investigate this issue, we combined eyetracking and fMRI, simultaneously recording eye movements and BOLD activity while subjects read text passages. We found that the variability and skew of fixation duration distributions across individuals, as assessed by ex-Gaussian analyses, decreased with increasing neural activity in regions associated with the cortical eye movement control network (Left FEF, Left IPS, Left IFG, and Right IFG). The results suggest that individual differences in fixation duration during reading are related to underlying neurocognitive processes associated with the eye movement control system and its relationship to language processing. The results also show that eye movements and fMRI can be combined to investigate the neural correlates of individual differences in natural reading.

Fixation durations in scene viewing: Modeling the effects of local image features, oculomotor parameters, and task

Scene perception requires the orchestration of image- and task-related processes with oculomotor constraints. The present study was designed to investigate how these factors influence how long the eyes remain fixated on a given location. Linear mixed models (LMMs) were used to test whether local image statistics (including luminance, luminance contrast, edge density, visual clutter, and the number of homogeneous segments), calculated for 1° circular regions around fixation locations, modulate fixation durations, and how these effects depend on task-related control. Fixation durations and locations were recorded from 72 participants, each viewing 135 scenes under three different viewing instructions (memorization, preference judgment, and search). Along with the image-related predictors, the LMMs simultaneously considered a number of oculomotor and spatiotemporal covariates, including the amplitudes of the previous and next saccades, and viewing time. As a key finding, the local image features around the current fixation predicted this fixation’s duration. For instance, greater luminance was associated with shorter fixation durations. Such immediacy effects were found for all three viewing tasks. Moreover, in the memorization and preference tasks, some evidence for successor effects emerged, such that some image characteristics of the upcoming location influenced how long the eyes stayed at the current location. In contrast, in the search task, scene processing was not distributed across fixation durations within the visual span. The LMM-based framework of analysis, applied to the control of fixation durations in scenes, suggests important constraints for models of scene perception and search, and for visual attention in general.

Disentangling the mechanisms underlying infant fixation durations in scene perception: A computational account

The goal of this article is to investigate the unexplored mechanisms underlying the development of saccadic control in infancy by determining the generalizability and potential limitations of extending the CRISP theoretical framework and computational model of fixation durations (FDs) in adult scene-viewing to infants. The CRISP model was used to investigate the underlying mechanisms modulating FDs in 6-month-olds by applying the model to empirical eye-movement data gathered from groups of infants and adults during free-viewing of naturalistic and semi-naturalistic videos. Participants also performed a gap-overlap task to measure their disengagement abilities. Results confirmed the CRISP model's applicability to infant data. Specifically, model simulations support the view that infant saccade programming is completed in two stages: an initial labile stage, followed by a non-labile stage. Moreover, results from the empirical data and simulation studies highlighted the influence of the material viewed on the FD distributions in infants and adults, as well as the impact that the developmental state of the oculomotor system can have on saccade programming and execution at 6months. The present work suggests that infant FDs reflect on-line perceptual and cognitive activity in a similar way to adults, but that the individual developmental state of the oculomotor system affects this relationship at 6months. Furthermore, computational modeling filled the gaps of psychophysical studies and allowed the effects of these two factors on FDs to be simulated in infant data providing greater insights into the development of oculomotor and attentional control than can be gained from behavioral results alone.

Perceptual enhancement as a result of a top-down attentional influence in a scene viewing task: Evidence from saccadic inhibition

Prior research has shown that task instructions influence the locations and durations of eye fixations during scene viewing. These task-related changes in gaze patterns are likely to be associated with a top-down influence of attention. Presently we applied a saccadic-inhibition manipulation in order to detect another expected manifestation of top-down attention: perceptual enhancement. Participants viewed eight-item arrays containing photographs from two categories of scenes. Four of the photos depicted natural landscapes ("nature") and the other four depicted urban environments ("buildings"). Participants were instructed to memorize scenes from one of the categories in preparation for a later recognition memory test. During eye fixations the border around the fixated scene flickered briefly from black to white with a random interval between flickers ranging from 400 to 600 ms. We computed the likelihood of a saccade being initiated in the period following the flicker. Consistent with prior research, we found a strong saccadic inhibition effect with maximum saccadic inhibition occurring approximately 97 ms following the flicker. Importantly, the saccadic inhibition effect was stronger and longer lasting when the subject's eyes were fixated on a relevant scene compared to an irrelevant scene. These findings are consistent with perceptual enhancement as a result of top-down attention.

The Influence of Content Meaningfulness on Eye Movements across Tasks: Evidence from Scene Viewing and Reading

The present study investigated the influence of content meaningfulness on eye-movement control in reading and scene viewing. Texts and scenes were manipulated to make them uninterpretable, and then eye-movements in reading and scene-viewing were compared to those in pseudo-reading and pseudo-scene viewing. Fixation durations and saccade amplitudes were greater for pseudo-stimuli. The effect of the removal of meaning was seen exclusively in the tail of the fixation duration distribution in both tasks, and the size of this effect was the same across tasks. These findings suggest that eye movements are controlled by a common mechanism in reading and scene viewing. They also indicate that not all eye movements are responsive to the meaningfulness of stimulus content. Implications for models of eye movement control are discussed.

Oculomotor and cognitive control of eye movements in reading: evidence from mindless reading

In the present study, we investigated the influence of cognitive factors on eye-movement behaviors in reading. Participants performed two tasks: a normal-reading task, as well as a mindless-reading task in which letters were replaced with unreadable block shapes. This mindless-reading task served as an oculomotor control condition, simulating the visual aspects of reading but removing higher-level linguistic processing. Fixation durations, word skipping, and some regressions were influenced by cognitive factors, whereas eye movements within words appeared to be less open to cognitive control. Implications for models of eye-movement control in reading are discussed.

Examining Eye Movements in Visual Search through Clusters of Objects in a Circular Array

Participants were asked to search for a complete O in an array consisting of eight clusters of four Landolt Cs (i.e., Os with a gap) arranged in a ring. The size of the gap in the Cs varied from cluster to cluster but was held constant within a cluster. The manual response time data were consistent with a serial self-terminating search. More importantly, eye movement data supported a serial processing model as (a) clusters were fixated serially (either clockwise or counterclockwise) on most trials and (b) fixation times on a cluster reflected processing time on that cluster and were unaffected by the gap size of either the prior or succeeding cluster. Furthermore, the pattern of fixation times on a cluster was similar to the pattern of response times in a secondary task where a single cluster was presented at fixation. These data extend the findings of Williams and Pollatsek (2007) in which search was through a linear sequence of clusters, and indicate that a serial search pattern through clusters of these kinds of objects is not confined to reading-like linear arrays.

Predicting cognitive state from eye movements

In human vision, acuity and color sensitivity are greatest at the center of fixation and fall off rapidly as visual eccentricity increases. Humans exploit the high resolution of central vision by actively moving their eyes three to four times each second. Here we demonstrate that it is possible to classify the task that a person is engaged in from their eye movements using multivariate pattern classification. The results have important theoretical implications for computational and neural models of eye movement control. They also have important practical implications for using passively recorded eye movements to infer the cognitive state of a viewer, information that can be used as input for intelligent human-computer interfaces and related applications.

The utility of modeling word identification from visual input within models of eye movements in reading

Decades of empirical work have shown that a range of eye movement phenomena in reading are sensitive to the details of the process of word identification. Despite this, major models of eye movement control in reading do not explicitly model word identification from visual input. This paper presents a argument for developing models of eye movements that do include detailed models of word identification. Specifically, we argue that insights into eye movement behavior can be gained by understanding which phenomena naturally arise from an account in which the eyes move for efficient word identification, and that one important use of such models is to test which eye movement phenomena can be understood this way. As an extended case study, we present evidence from an extension of a previous model of eye movement control in reading that does explicitly model word identification from visual input, Mr. Chips (Legge, Klitz, & Tjan, 1997), to test two proposals for the effect of using linguistic context on reading efficiency.