Using support vector machines to identify literacy skills: Evidence from eye movements

Deep learning-driven dyslexia detection model using multi-modality data

Background

Dyslexia is a neurological disorder that affects an individual's language processing abilities. Early care and intervention can help dyslexic individuals succeed academically and socially. Recent developments in deep learning (DL) approaches motivate researchers to build dyslexia detection models (DDMs). DL approaches facilitate the integration of multi-modality data. However, there are few multi-modality-based DDMs.

Methods

In this study, the authors built a DL-based DDM using multi-modality data. A squeeze and excitation (SE) integrated MobileNet V3 model, self-attention mechanisms (SA) based EfficientNet B7 model, and early stopping and SA-based Bi-directional long short-term memory (Bi-LSTM) models were developed to extract features from magnetic resonance imaging (MRI), functional MRI, and electroencephalography (EEG) data. In addition, the authors fine-tuned the LightGBM model using the Hyperband optimization technique to detect dyslexia using the extracted features. Three datasets containing FMRI, MRI, and EEG data were used to evaluate the performance of the proposed DDM.

Results

The findings supported the significance of the proposed DDM in detecting dyslexia with limited computational resources. The proposed model outperformed the existing DDMs by producing an optimal accuracy of 98.9%, 98.6%, and 98.8% for the FMRI, MRI, and EEG datasets, respectively. Healthcare centers and educational institutions can benefit from the proposed model to identify dyslexia in the initial stages. The interpretability of the proposed model can be improved by integrating vision transformers-based feature extraction.

Improved Graph Convolutional Neural Network for Dance Tracking and Pose Estimation

Movement recognition technology is widely used in various practical application scenarios, but there are few researches on dance movement recognition at present. Aiming at the problem of low accuracy of dance movement recognition due to complex pose changes in dance movements, this paper designed an improved graph convolutional neural network algorithm for dance tracking and pose estimation. In this method, the spatial and temporal characteristics of motion are extracted from the skeleton joint diagram of human body. Then, GCN (graph convolutional neural) is used to extract potential spatial information between skeleton nodes. Finally, LSTM (long short-term memory) extracts the time series features before and after human actions as a supplement and performs late fusion of the prediction outputs of the two networks, respectively, to improve the problem of insufficient generalization ability of single network. The experimental results show that this method can effectively improve the accuracy of dance movement recognition in general movement recognition data set and dance pose data set. It has certain application value in dance self-help teaching, professional dancer movement correction, and other application scenarios.

Predicting the Reader's English Level from Reading Fixation Patterns Using the Siamese Convolutional Neural Network

Abstract-Among numerous functions performed by the human eye, reading is a common task that best reflects an individual's understanding and cognitive patterns. Previous studies showed that text comprehension may be determined by comprehension monitoring, a metacognitive process that evaluates and regulates the pattern of comprehension. Herein, we propose a hypothesis: an individual's cognitive pattern during reading is predictive of the level of reading comprehension. According to the criteria of the College English Test Band Six (CET-6), 80 participants (sophomore and junior) were divided into a pass group (n = 40) and a non-pass group (n = 40). Heatmaps of eye fixation counts were collected by an eye-tracker while each participant executed four reading comprehension tests. Using these heatmaps as inputs, we proposed the Siamese convolutional neural network models to predict the English level of participants. Both strategies of "Trained from scratch" and "Pretrained with fine-tuning" were employed. "Soft Voting" was applied to integrate the predictions from four tests. Results showed that the Siamese network model trained by the datasets with the cluster radius of fixation equal to 25 pixels and connection layer by L1 norm distance has a satisfactory or superior performance to other comparative experiments. The AUC values of Siamese networks trained by the two strategies reach 0.941 and 0.956, respectively. This indicates that the individual reading cognitive pattern captured by the eye-tracker can predict the level of reading comprehension through advanced deep learning models.

Eye movements during text reading align with the rate of speech production

Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3 and 5.5 Hz, reflecting the production and processing of linguistic information chunks (syllables and words) every ~200 ms. Interestingly, ~200 ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ~5 Hz. A subsequent meta-analysis of 142 studies from 14 languages replicates this result and shows that sampling frequencies vary across languages between 3.9 Hz and 5.2 Hz. This variation systematically depends on the complexity of the writing systems (character-based versus alphabetic systems and orthographic transparency). Finally, we empirically demonstrate a positive correlation between speech spectrum and eye movement sampling in low-skilled non-native readers, with tentative evidence from post hoc analysis suggesting the same relationship in low-skilled native readers. On the basis of this convergent evidence, we propose that during reading, our brain's linguistic processing systems imprint a preferred processing rate-that is, the rate of spoken language production and perception-onto the oculomotor system.

What Eye Movements Reveal About Later Comprehension of Long Connected Texts

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level cross-validation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns = 104, 130, 147) answered multiple-choice comprehension questions ~30 min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372, ps < .001), in line with the strong association hypothesis. We found that making more, but shorter fixations, consistently predicted comprehension across all studies. Furthermore, models trained on one study's data could successfully predict comprehension on the others, suggesting generalizability across studies. Collectively, these findings suggest that there is a robust link between eye movements and subsequent comprehension of a long connected text, thereby connecting theories of low-level eye movements with those of higher order text processing during reading.

Individual differences in first-pass fixation duration in reading are related to resting-state functional connectivity

Although there are considerable individual differences in eye movements during text reading, their neural correlates remain unclear. In this study, we investigated the relationship between the first-pass fixation duration (FPFD) in natural reading and resting-state functional connectivity (RSFC) in the brain. We defined the brain regions associated with early visual processing, word identification, attention shifts, and oculomotor control as seed regions. The results showed that individual FPFDs were positively correlated with individual RSFCs between the early visual network, visual word form area, and eye movement control/dorsal attention network. Our findings provide new evidence on the neural correlates of eye movements in text reading and indicate that individual differences in fixation time may shape the RSFC differences in the brain through the time-on-task effect and the mechanism of Hebbian learning.

What Can Eye Movements Tell us about Visual Perception Processes in Classroom Contexts? Commentary on a Special Issue

In this commentary to the Special Issue of Educational Psychology Review on visual perceptual processes, I tie the empirical studies reported in the issue with previous research in other domains to offer some points to be considered in future studies. First, I will point out to issues related to the operationalization of the theoretical constructs. The empirical papers in this Special Issue use eye tracking to study students’ engagement, teachers’ expertise, and student-teacher interaction. However, it is not always clear how the observed eye movement patterns reflect these theoretical concepts and the underlying psychological processes. Second, I will reflect on the analyses of the eye movement data presented in the papers. The main advantage of the methodology is that it can provide detailed information about the time-course of processing, and to fully engage its potential, it should be complemented with adequate statistical methods. In my view, the papers in this Special Issue provide valuable novel information about the complex processes underlying learning in variable contexts, and offer an excellent starting point for future research.

Hippocampal connectivity in the aftermath of acute social stress

The hippocampus is a core brain region that responds to stress. Previous studies have found a dysconnectivity between hippocampus and other brain regions under acute and chronic stress. However, whether and how acute social stress influences the directed connectivity patterns from and to the hippocampus remains unclear. In this study, using a within-subject design and Granger causal analysis (GCA), we investigated the alterations of resting state effective connectivity from and to hippocampal subregions after an acute social stressor (the Trier Social Stress Test). Participants were engaged in stress and control conditions spaced approximately one month apart. Our findings showed that stress altered the information flows in the thalamus-hippocampus-insula/midbrain circuit. The changes in this circuit could also predict with high accuracy the stress and control conditions at the subject level. These hippocampus-related brain networks have been documented to be involved in emotional information processing and storage, as well as habitual responses. We speculate that alterations of the effective connectivity between these brain regions may be associated with the registering and encoding of threatening stimuli under stress. Our investigation of hippocampal functional connectivity at a subregional level may help elucidate the functional neurobiology of stress-related psychiatric disorders.

Reading Shakespeare Sonnets: Combining Quantitative Narrative Analysis and Predictive Modeling -an Eye Tracking Study

As a part of a larger interdisciplinary project on Shakespeare sonnets' reception (1, 2), the present study analyzed the eye movement behavior of participants reading three of the 154 sonnets as a function of seven lexical features extracted via Quantitative Narrative Analysis (QNA). Using a machine learning-based predictive modeling approach five 'surface' features (word length, orthographic neighborhood density, word frequency, orthographic dissimilarity and sonority score) were detected as important predictors of total reading time and fixation probability in poetry reading. The fact that one phonological feature, i.e., sonority score, also played a role is in line with current theorizing on poetry reading. Our approach opens new ways for future eye movement research on reading poetic texts and other complex literary materials(3).

One page of text: Eye movements during regular and thorough reading, skimming, and spell checking

Eye movements during regular reading, thorough reading, skimming, and spell checking of single pages of text were measured, to investigate how high-level reading tasks elicited by instructions affect reading behavior. Word frequency and word length effects were found. All results were compared to regular reading. Thorough reading involved longer total reading times and more rereading, and resulted in higher comprehension scores. Skimming involved longer saccades, shorter average fixation durations, more word skipping, shorter total reading times evenly distributed across the page, and resulted in lower comprehension scores. Spell checking involved shorter saccades, longer average fixation durations, less word skipping, longer total reading times evenly distributed across the entire page, and resulted in lower comprehension scores. Replicating local effects shows that paragraphs maintain sufficient experimental rigor, while also enabling reading analyses from a global perspective. Compared to regular reading, thorough reading was more elaborate and less uniform, skimming was faster and more uniform, and spell checking was slower and more uniform.