Real-time interpretation of novel events across childhood

Children’s and adults’ use of fictional discourse and semantic knowledge for prediction in language processing

Using real-time eye-movement measures, we asked how a fantastical discourse context competes with stored representations of real-world events to influence the moment-by-moment interpretation of a story by 7-year-old children and adults. Seven-year-olds were less effective at bypassing stored real-world knowledge during real-time interpretation than adults. Our results suggest that children privilege stored semantic knowledge over situation-specific information presented in a fictional story context. We suggest that 7-year-olds’ canonical semantic and conceptual relations are sufficiently strongly rooted in statistical patterns in language that have consolidated over time that they overwhelm new and unexpected information even when the latter is fantastical and highly salient.

Do Children Use Multi-Word Information in Real-Time Sentence Comprehension?

Meaning in language emerges from multiple words, and children are sensitive to multi-word frequency from infancy. While children successfully use cues from single words to generate linguistic predictions, it is less clear whether and how they use multi-word sequences to guide real-time language processing and whether they form predictions on the basis of multi-word information or pairwise associations. We address these questions in two visual-world eye-tracking experiments with 5- to 8-year-old children. In Experiment 1, we asked whether children generate more robust predictions for the sentence-final object of highly frequent sequences (e.g., "Throw the ball"), compared to less frequent sequences (e.g., "Throw the book"). We further examined if gaze patterns reflect event knowledge or phrasal frequency by comparing the processing of phrases that have the same event structure but differ in multi-word content (e.g., "Brush your teeth" vs. "Brush her teeth"). In the second study, we employed a training paradigm to ask if children are capable of generating predictio.ns from novel multi-word associations while controlling for the overall frequency of the sequences. While the results of Experiment 1 suggested that children primarily relied on event associations to generate real-time predictions, those of Experiment 2 showed that the same children were able to use recurring novel multi-word sequences to generate real-time linguistic predictions. Together, these findings suggest that children can draw on multi-word information to generate linguistic predictions, in a context-dependent fashion, and highlight the need to account for the influence of multi-word sequences in models of language processing.

The Neural Underpinnings of Processing Newly Taught Semantic Information: The Role of Retrieval Practice

Purpose Recent behavioral studies have demonstrated the effectiveness of implementing retrieval practice into learning tasks for children. Such approaches have revealed that repeated spaced retrieval (RSR) is particularly effective in promoting children's learning of word form and meaning information. This study further examines how retrieval practice enhances learning of word meaning information at the behavioral and neural levels. Method Twenty typically developing preschool children were taught novel words using an RSR learning schedule for some words and an immediate retrieval (IR) learning schedule for other words. In addition to the label, children were taught two arbitrary semantic features for each item. Following the teaching phase, children's learning was tested using recall tests. In addition, during the 1-week follow-up, children were presented with pictures and an auditory sentence that correctly labeled the item but stated correct or incorrect semantic information. Event-related brain potentials (ERPs) were time locked to the onset of the words noting the semantic feature. Children provided verbal judgments of whether the semantic feature was correctly paired with the item. Results Children recalled more labels and semantic features for items that had been taught in the RSR learning schedule relative to the IR learning schedule. ERPs also differentiated the learning schedules. Mismatching label-meaning pairings elicited an N400 and late positive component (LPC) for both learning conditions; however, mismatching RSR pairs elicited an N400 with an earlier onset and an LPC with a longer duration, relative to IR mismatching label-meaning pairings. These ERP timing differences indicated that the children were more efficient in processing words that were taught in the RSR schedule relative to the IR learning schedule. Conclusions Spaced retrieval practice promotes learning of both word form and meaning information. The findings lay the necessary groundwork for better understanding of processing newly learned semantic information in preschool children. Supplemental Material https://doi.org/10.23641/asha.15063060.

Eye Tracking Measures for Studying Language Comprehension Deficits in Aphasia: A Systematic Search and Scoping Review

Aim The aim of this scoping review is to identify the eye tracking paradigms and eye movement measures used to investigate auditory and reading comprehension deficits in persons with aphasia (PWA). Method MEDLINE via PubMed, Cochrane, CINAHL, Embase, PsycINFO, OTseeker, Scopus, Google Scholar, Grey Literature Database, and ProQuest Search (Dissertations & Theses) were searched for relevant studies. The Covidence software was used to manage the initial and full-text screening process for the search. Results and Discussion From a total of 1,803 studies, 68 studies were included for full-text screening. In addition, 418 records from gray literature were also screened. After full-text screening, 16 studies were included for this review-12 studies for auditory comprehension in PWA and four studies for reading comprehension in PWA. The review highlights the use of common eye tracking paradigms used to study language comprehension in PWA. We also discusse eye movement measures and how they help in assessing auditory and reading comprehension. Methodological challenges of using eye tracking are discussed. Conclusion The studies summarized in this scoping review provide evidence that the eye tracking methods are beneficial for studying auditory and reading comprehension in PWA.

Semantic processing of adjectives and nouns in American Sign Language: effects of reference ambiguity and word order across development

When processing spoken language sentences, listeners continuously make and revise predictions about the upcoming linguistic signal. In contrast, during comprehension of American Sign Language (ASL), signers must simultaneously attend to the unfolding linguistic signal and the surrounding scene via the visual modality. This may affect how signers activate potential lexical candidates and allocate visual attention as a sentence unfolds. To determine how signers resolve referential ambiguity during real-time comprehension of ASL adjectives and nouns, we presented deaf adults (n = 18, 19-61 years) and deaf children (n = 20, 4-8 years) with videos of ASL sentences in a visual world paradigm. Sentences had either an adjective-noun ("SEE YELLOW WHAT? FLOWER") or a noun-adjective ("SEE FLOWER WHICH? YELLOW") structure. The degree of ambiguity in the visual scene was manipulated at the adjective and noun levels (i.e., including one or more yellow items and one or more flowers in the visual array). We investigated effects of ambiguity and word order on target looking at early and late points in the sentence. Analysis revealed that adults and children made anticipatory looks to a target when it could be identified early in the sentence. Further, signers looked more to potential lexical candidates than to unrelated competitors in the early window, and more to matched than unrelated competitors in the late window. Children's gaze patterns largely aligned with those of adults with some divergence. Together, these findings suggest that signers allocate referential attention strategically based on the amount and type of ambiguity at different points in the sentence when processing adjectives and nouns in ASL.

Developmental Timescale of Rapid Adaptation to Conflicting Cues in Real-Time Sentence Processing

Children and adults use established global knowledge to generate real-time linguistic predictions, but less is known about how listeners generate predictions in circumstances that semantically conflict with long-standing event knowledge. We explore these issues in adults and 5- to 10-year-old children using an eye-tracked sentence comprehension task that tests real-time activation of unexpected events that had been previously encountered in brief stories. Adults generated predictions for these previously unexpected events based on these discourse cues alone, whereas children overall did not override their established global knowledge to generate expectations for semantically conflicting material; however, they do show an increased ability to integrate discourse cues to generate appropriate predictions for sentential endings. These results indicate that the ability to rapidly integrate and deploy semantically conflicting knowledge has a long developmental trajectory, with adult-like patterns not emerging until later in childhood.

Vocabulary size and Native Speaker self-identification influence flexibility in linguistic prediction among adult bilinguals

When language users predict upcoming speech, they generate pluralistic expectations, weighted by likelihood (Kuperberg & Jaeger, 2016). Many variables influence the prediction of highly-likely sentential outcomes, but less is known regarding variables affecting the prediction of less-likely outcomes. Here we explore how English vocabulary size and self-identification as a Native Speaker (NS) of English modulate adult bi-/multilinguals' pre-activation of less-likely sentential outcomes in two visual-world experiments. Participants heard transitive sentences containing an agent, action and theme (The pirate chases the ship) while viewing four referents varying in expectancy by relation to the agent and action. In experiment 1 (N=70), spoken themes referred to highly-expected items (e.g., ship). Results indicate lower-skill (smaller vocabulary size) and less confident (not identifying as NS) bi-/multilinguals activate less-likely action-related referents more than their higher-skill/confidence peers. In experiment 2 (N=65), themes were one of two less-likely items (The pirate chases the bone/cat). Results approaching significance indicate an opposite but similar size effect: higher-skill/confidence listeners activate less-likely action-related (e.g., bone) referents slightly more than lower-skill/confidence listeners. Results across experiments suggest higher-skill/confidence participants more flexibly modulate their linguistic predictions per the demands of the task, with similar but not identical patterns emerging when bi-/multilinguals are grouped by self-ascribed NS-status versus vocabulary size.

Distinct effects of different visual cues on sentence comprehension and later recall: The case of speaker gaze versus depicted actions

Language-processing accounts are beginning to accommodate different visual context effects, but they remain underspecified regarding differences between cues, both during sentence comprehension and subsequent recall. We monitored participants' eye movements to mentioned characters while they listened to transitive sentences. We varied whether speaker gaze, a depicted action, neither, or both of these visual cues were available, as well as whether both cues were deictic (Experiment 1) or only speaker gaze (Experiment 2). Speaker gaze affected eye movements during comprehension similarly early to a single deictic action depiction, but significantly earlier than non-deictic action depictions; conversely, depicted actions but not speaker gaze positively affected later recall of sentence content. Thus, cue type and cue-language relations must be accommodated in characterising real-time situated language comprehension and subsequent recall of sentence content.

Maternal Socioeconomic Status Influences the Range of Expectations During Language Comprehension in Adulthood

In infancy, maternal socioeconomic status (SES) is associated with real-time language processing skills, but whether or not (and if so, how) this relationship carries into adulthood is unknown. We explored the effects of maternal SES in college-aged adults on eye-tracked, spoken sentence comprehension tasks using the visual world paradigm. When sentences ended in highly plausible, expected target nouns (Exp. 1), higher SES was associated with a greater likelihood of considering alternative endings related to the action of the sentence. Moreover, for unexpected sentence endings (Exp. 2), individuals from higher SES backgrounds were sensitive to whether the ending was action-related (plausible) or unrelated (implausible), showing a benefit for plausible endings. Individuals from lower SES backgrounds did not show this advantage. This suggests maternal SES can influence the dynamics of sentence processing even in adulthood, with consequences for processing unexpected content. These findings highlight the importance of early lexical experience for adult language skills.

Lexical leverage: category knowledge boosts real-time novel word recognition in 2-year-olds

Recent research suggests that infants tend to add words to their vocabulary that are semantically related to other known words, though it is not clear why this pattern emerges. In this paper, we explore whether infants leverage their existing vocabulary and semantic knowledge when interpreting novel label-object mappings in real time. We initially identified categorical domains for which individual 24-month-old infants have relatively higher and lower levels of knowledge, irrespective of overall vocabulary size. Next, we taught infants novel words in these higher and lower knowledge domains and then asked if their subsequent real-time recognition of these items varied as a function of their category knowledge. While our participants successfully acquired the novel label-object mappings in our task, there were important differences in the way infants recognized these words in real time. Namely, infants showed more robust recognition of high (vs. low) domain knowledge words. These findings suggest that dense semantic structure facilitates early word learning and real-time novel word recognition.

Tracking sentence comprehension: Test-retest reliability in people with aphasia and unimpaired adults

PURPOSE

Visual-world eyetracking is increasingly used to investigate online language processing in normal and language impaired listeners. Tracking changes in eye movements over time also may be useful for indexing language recovery in those with language impairments. Therefore, it is critical to determine the test-retest reliability of results obtained using this method.

METHODS

Unimpaired young adults and people with aphasia took part in two eyetracking sessions spaced about one week apart. In each session, participants completed a sentence-picture matching task in which they listened to active and passive sentences (e.g., The [N1+Auxwoman was] [Vvisiting/visited] [NP/PP2(by) the man]) and selected between two pictures with reversed thematic roles. We used intraclass correlations (ICCs) to examine the test-retest reliability of response measures (accuracy, reaction time (RT)) and online eye movements (i.e., the likelihood of fixating the target picture in each region of the sentence) in each participant group.

RESULTS

In the unimpaired adults, accuracy was at ceiling (thus ICCs were not computed), with moderate ICCs for RT (i.e., 0.4 - 0.58) for passive sentences and low (<0.4) for actives. In individuals with aphasia, test-retest reliability was strong (0.590.75) for RT for both sentence types. Similarly, for the unimpaired listeners, reliability of eye movements was moderate for passive sentences (NP/PP2 region) and low in all regions for active sentences. But, for the aphasic participant group, eye movement reliability was excellent for passive sentences (in the first second after sentence end) and strong for active sentences (V and NP/PP2 regions).

CONCLUSION

Results indicated moderate-to-low reliability for unimpaired listeners; however, reliable eye movement patterns were detected for processes specific to passive sentences (e.g., thematic reanalysis). In contrast, individuals with aphasia exhibited strong and stable performance across sentence types in response measures and online eye movements. These findings indicate that visual-world eyetracking provides a reliable measure of online sentence comprehension, and thus may be useful for investigating sentence processing changes over time.

The amount and structure of prior event experience affects anticipatory sentence interpretation

Listeners easily interpret speech about novel events in everyday conversation; however, much of research on mechanisms of spoken language comprehension, by design, capitalises on event knowledge that is familiar to most listeners. This paper explores how listeners generalise from previous experience during incremental processing of novel spoken sentences. In two studies, participants initially heard stories that conveyed novel event mappings between agents, actions and objects, and their ability to interpret a novel, related event in real-time was measured via eye-tracking. A single exposure to a novel event was not sufficient to support generalisation in real-time sentence processing. When each story event was repeated with either the same agent or a different, related agent, listeners generalised in the repetition condition, but not in the multiple agent condition. These findings shed light on the conditions under which listeners leverage prior event experience while interpreting novel linguistic signals in everyday speech.

Roses Are Red, Socks Are Blue: Switching Dimensions Disrupts Young Children's Language Comprehension

Language is used to identify objects in many different ways. An apple can be identified using its name, color, and other attributes. Skilled language comprehension requires listeners to flexibly shift between different dimensions. We asked whether this shifting would be difficult for 3-year-olds, who have relatively immature executive function skills and struggle to switch between dimensions in card sorting tasks. In the current experiment, children first heard a series of sentences identifying objects using a single dimension (either names or colors). In the second half of the experiment, the labeling dimension was switched. Children were significantly less accurate in fixating the correct object following the dimensional switch. This disruption, however, was temporary; recognition accuracy recovered with increased exposure to the new labeling dimension. These findings provide the first evidence that children's difficulty in shifting between dimensions impacts their ability to comprehend speech. This limitation may affect children's ability to form rich, multi-dimensional representations when learning new words.

Semantic Structure in Vocabulary Knowledge Interacts With Lexical and Sentence Processing in Infancy

Although the size of a child's vocabulary associates with language-processing skills, little is understood regarding how this relation emerges. This investigation asks whether and how the structure of vocabulary knowledge affects language processing in English-learning 24-month-old children (N = 32; 18 F, 14 M). Parental vocabulary report was used to calculate semantic density in several early-acquired semantic categories. Performance on two language-processing tasks (lexical recognition and sentence processing) was compared as a function of semantic density. In both tasks, real-time comprehension was facilitated for higher density items, whereas lower density items experienced more interference. The findings indicate that language-processing skills develop heterogeneously and are influenced by the semantic network surrounding a known word.

Where are the cookies? Two- and three-year-olds use number-marked verbs to anticipate upcoming nouns

We tested toddlers' and adults' predictive use of English subject-verb agreement. Participants saw pairs of pictures differing in number and kind (e.g., one apple, two cookies), and heard sentences with a target noun naming one of the pictures. The target noun was the subject of a preceding agreeing verb in informative trials (e.g., Wherearethe good cookies?), but not in uninformative trials (Do you see the good cookies?). In Experiment 1, 3-year-olds and adults were faster and more likely to shift their gaze from distractor to target upon hearing an informative agreeing verb. In Experiment 2, 2.5-year-olds were faster to shift their gaze from distractor to target in response to the noun in informative trials, and were more likely to be fixating the target already at noun onset. Thus, toddlers used agreeing verbs to predict number features of an upcoming noun. These data provide strong new evidence for the broad scope of predictive processing in online language comprehension.