Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Language Experience Predicts Eye Movements During Online Auditory Comprehension

Experience-based theories of language processing suggest that listeners use the properties of their previous linguistic input to constrain comprehension in real time (e.g. MacDonald & Christiansen, 2002; Smith & Levy, 2013; Stanovich & West, 1989; Mishra, Pandey, Singh, & Huettig, 2012). This project investigates the prediction that individual differences in experience will predict differences in sentence comprehension. Participants completed a visual world eye-tracking task following Altmann and Kamide (1999) which manipulates whether the verb licenses the anticipation of a specific referent in the scene (e.g. The boy will eat/move the cake). Within this paradigm, we ask (1) are there reliable individual differences in language-mediated eye movements during this task? If so, (2) do individual differences in language experience correlate with these differences, and (3) can this relationship be explained by other, more general cognitive abilities? Study 1 finds evidence that language experience predicts an overall facilitation in fixating the target, and Study 2 replicates this effect and finds that it remains when controlling for working memory, inhibitory control, phonological ability, and perceptual speed.

Canalization of Language Structure From Environmental Constraints: A Computational Model of Word Learning From Multiple Cues

There is substantial variation in language experience, yet there is surprising similarity in the language structure acquired. Constraints on language structure may be external modulators that result in this canalization of language structure, or else they may derive from the broader, communicative environment in which language is acquired. In this paper, the latter perspective is tested for its adequacy in explaining robustness of language learning to environmental variation. A computational model of word learning from cross‐situational, multimodal information was constructed and tested. Key to the model's robustness was the presence of multiple, individually unreliable information sources to support learning. This “degeneracy” in the language system has a detrimental effect on learning, compared to a noise‐free environment, but has a critically important effect on acquisition of a canalized system that is resistant to environmental noise in communication.

Using eye tracking and gaze pattern analysis to test a "dirty bomb" decision aid in a pilot RCT in urban adults with limited literacy

Background

Eye tracking is commonly used in marketing to understand complex responses to materials, but has not been used to understand how low-literacy adults access health information or its relationship to decision making.

Methods

This study assessed how participants use a literacy appropriate “dirty bomb” decision aid. Participants were randomized to receive a CDC “factsheet” (n = 21) or literacy-appropriate aid (n = 29) shown on a computer screen. Using 7 content similar slides, gaze patterns, mean pupil fixation time and mean overall time reading and looking at slides were compared. Groups were also compared by literacy level and effect on 'confidence of knowledge' and intended behavior.

Results

Results revealed differing abilities to read densely written material. Intervention participants more precisely followed text on 4 of 7 content-similar slides compared to control participants whose gaze patterns indicated unread text, or repeated attempts at reading the same text, suggesting difficulty in understanding key preparedness messages. Controls had significantly longer pupil fixations on 5 of 7 slides and spent more overall time on every slide. In those with very low literacy, intervention participants were more likely than controls to say they understood what a “dirty bomb” is and how to respond if one should occur.

Conclusions

Results indicate limited- literacy adults, especially those with very low literacy, may not be able to understand how to respond during a “dirty bomb” using available materials, making them vulnerable to negative health events. This study provides insights into how individuals perceive and process risk communication messages, illustrating a rich and nuanced understanding of the qualitative experience of a limited literacy population with written materials. It also demonstrates the feasibility of using these methods on a wider scale to develop more effective health and risk communication messages designed to increase knowledge of and compliance with general health guidelines, and enhance decision making. This has application for those with learning disabilities, those with limited media-literacy skills, and those needing to access the diverse array of assistive technologies now available. Eye tracking is thus a practical approach to understanding these diverse needs to ensure the development of cogent and salient communication.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0304-5) contains supplementary material, which is available to authorized users.

How Word Meaning Influences Word Reading

Understanding how we read is a fundamental question for psychology, with critical implications for education. Studies of word reading tend to focus on the mappings between the written and spoken forms of words. In this article, we review evidence from developmental, neuroimaging, neuropsychological, and computational studies that show that knowledge of word meanings is inextricably involved in word reading. Consequently, models of reading must better specify the role of meaning in skilled reading and its acquisition. Further, our review paves the way for educationally realistic research to confirm whether explicit teaching of oral vocabulary improves word reading.

Delayed Anticipatory Spoken Language Processing in Adults with Dyslexia—Evidence from Eye-tracking

It is now well established that anticipation of upcoming input is a key characteristic of spoken language comprehension. It has also frequently been observed that literacy influences spoken language processing. Here, we investigated whether anticipatory spoken language processing is related to individuals' word reading abilities. Dutch adults with dyslexia and a control group participated in two eye-tracking experiments. Experiment 1 was conducted to assess whether adults with dyslexia show the typical language-mediated eye gaze patterns. Eye movements of both adults with and without dyslexia closely replicated earlier research: spoken language is used to direct attention to relevant objects in the environment in a closely time-locked manner. In Experiment 2, participants received instructions (e.g., 'Kijk naar de(COM) afgebeelde piano(COM)', look at the displayed piano) while viewing four objects. Articles (Dutch 'het' or 'de') were gender marked such that the article agreed in gender only with the target, and thus, participants could use gender information from the article to predict the target object. The adults with dyslexia anticipated the target objects but much later than the controls. Moreover, participants' word reading scores correlated positively with their anticipatory eye movements. We conclude by discussing the mechanisms by which reading abilities may influence predictive language processing.