Sample size for measuring grammaticality in preschool children from picture-elicited language samples

Remote collection of language samples from three-year-olds

We characterised language samples collected remotely from typically developing three-year-olds by comparing them against independent language samples collected in person from age-matched peers with and without language delays. Forty-eight typically developing, English-learning three-year-olds were administered a picture description task via Zoom. The in-person comparison groups were two sets of independent language samples from age-matched typically developing as well as language-delayed children available on the Child Language Data Exchange System. The findings show that although language samples collected remotely from three-year-olds yield numerically dissimilar lexical and grammatical measures compared to samples collected in person, they still consistently distinguish toddlers with and without language delays.

Predicting language outcomes at 3 years using individual differences in morphological segmentation in infancy

In previous research, infants' performance on speech perception tasks has been shown to predict later language outcomes, typically vocabulary size. We used Bayesian analyses to model trial-level looking time behavior of individual infants on morphological segmentation experiments. We compared the usefulness of Bayesian estimates and the raw looking time difference measures used in previous studies to predict (a) vocabulary size at 30 months and (b) outcome measures obtained from language samples elicited via a picture description task at 36 months. We found that both estimates of morphological segmentation reliably predicted expressive vocabulary at 30 months. The Bayesian estimate also credibly predicted the correct use of verb tense morphemes obtained from the language sample. We therefore conclude that the Bayesian estimate is better for indexing individual differences in segmentation tasks and more useful for predicting clinically relevant language outcomes.

Investigating morphosyntactic and semantic measures in bilingual Azeri-Persian speaking children aged 5.5 to 6.5 years with and without language impairment

The present study aims to investigate morphosyntactic and semantic measures in bilingual Azeri-Persian-speaking children aged 5.5-6.5 years with and without language impairment. In this cross-sectional study, the bilingual participants were thirty children with language impairment (LI) and fifteen typically developing children (TD) who were selected from nurseries and Speech therapy clinics. The language samples were collected through story-telling in Azeri and Persian languages, separately. The linguistic analysis was done based on morphosyntactic and semantic parameters. The Alberta Language and Development Questionnaire (ALDeQ) parent report questionnaire was completed via interviewing with the parents to differentiate language impairment from language differences. Study findings revealed a significant difference between the morphosyntactic and semantic scores in two groups of bilingual Azeri-Persian speaking LI and TD children (p ˂ 0.05). Also, the results demonstrated no significant relationship between the scores of linguistic scores and age in LI and TD children (p < 0.05). According to the result of the study, morphosyntactic and semantic parameters of language samples in bilingual Azeri-Persian-speaking children could be utilized to provide diagnostic information for speech and language pathologists in LI children among bilingual Azeri-Persian communities.

The Reliability of Short Conversational Language Sample Measures in Children With and Without Developmental Language Disorder

PURPOSE

Language sample analysis (LSA) represents an ecologically valid method for diagnosing, identifying goals, and measuring progress in children with developmental language disorder (DLD). LSA is, however, time consuming. The purpose of this study was to determine the length of sample needed to obtain reliable LSA measures for children in kindergarten and first grade with typical language (TL) and DLD using automated analyses from the Systematic Analysis of Language Transcripts software.

METHOD

Play-based conversational language samples collected on kindergarten to first-grade children with TL (n = 21) and DLD (n = 21) from a community-based sample were analyzed. Eight LSA measures were calculated from 1-, 3-, 5-, 7-, and 10-min sample cuts and compared to 20-min samples for reliability.

RESULTS

Reliability estimates were similar for the TL and DLD groups except for errors and omissions, which showed overall higher levels of reliability in the DLD group and reached acceptable levels at 3 min. Percent grammatical utterances were reliable at 7 min in the DLD group and not reliable in shorter samples in the TL group. The subordination index was reliable at 10 min for both groups. Number of different words reached acceptable reliability at the 3-min length for the DLD group and at the 10-min length for the TL group. Utterances and words per minute were reliable at 3 min and mean length of utterance at 7 min in both groups.

CONCLUSIONS

Speech-language pathologists can obtain reliable LSA measures from shorter, 7-min conversational language samples from kindergarten to first-grade children with DLD. Shorter language samples may encourage increased use of LSA.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19529287.

Using Animated Action Scenes to Remotely Assess Sentence Diversity in Toddlers

Although language samples are child-friendly and well-suited for obtaining global measures of language production, structured protocols have the potential to elicit many different exemplars of language structures in a shorter amount of time. We created a structured elicitation protocol, the Sentence Diversity Priming Task (SDPT), to efficiently assess sentence diversity in toddlers via video chat platforms. Sentence diversity is operationalized as the number of different subject–verb combinations in active declarative sentences. The task is presented as an animated picture book, with parents serving as the child's primary partner during administration. In this article, we provide the rationale for assessing sentence diversity, describe the task, and present preliminary analyses of compliance and developmental associations for 32 typically developing toddlers, 30–35 months old, with average language abilities. The preliminary findings suggest that the SDPT is an engaging task that holds toddlers' attention, reveals robust individual differences in their ability to produce sentences, is positively correlated with parent-reported language measures, and has the potential for assessing children's language growth over time. Finally, recommendations and tips for developing and remotely administering the protocol are provided, with an emphasis on encouraging parent involvement and increasing toddler compliance.

Predicting Language Performance From Narrative Language Samples

PURPOSE

Analysis of narrative language samples is a recommended clinical practice in the assessment of children's language skills, but we know little about how results from such analyses relate to overall oral language ability across the early school years. We examined the relations between language sample metrics from a short narrative retell, collected in kindergarten, and an oral language factor in grades kindergarten through 3. Our specific questions were to determine the extent to which metrics from narrative language sample analysis are concurrently related to language in kindergarten and predict language through Grade 3.

METHOD

Participants were a sample of 284 children who were administered a narrative retell task in kindergarten and a battery of vocabulary and grammar measures in kindergarten through Grade 3. Language samples were analyzed for number of different words, mean length of utterance, and a relatively new metric, percent grammatical utterances (PGUs). Structural equation models were used to estimate the concurrent and longitudinal relationships.

RESULTS

The narrative language sample metrics were consistently correlated with the individual vocabulary and grammar measures as well as the language factor in each grade, and also consistently and uniquely predicted the language factor in each grade. Standardized path estimates in the structural equation models ranged from 0.20 to 0.39.

CONCLUSIONS

This study found narrative language sample metrics to be predictive, concurrently and longitudinally, of a latent factor of language from kindergarten through Grade 3. These results further validate the importance of collecting and analyzing narrative language samples, to include PGU along with more traditional metrics, and point to directions for future research.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.17700980.

Using Free Computer-Assisted Language Sample Analysis to Evaluate and Set Treatment Goals for Children Who Speak African American English

Purpose Spoken language sample analysis (LSA) is widely considered to be a critical component of assessment for child language disorders. It is our best window into a preschool child's everyday expressive communicative skills. However, historically, the process can be cumbersome, and reference values against which LSA findings can be "benchmarked" are based on surprisingly little data. Moreover, current LSA protocols potentially disadvantage speakers of nonmainstream English varieties, such as African American English (AAE), blurring the line between language difference and disorder. Method We provide a tutorial on the use of free software (Computerized Language Analysis [CLAN]) enabled by the ongoing National Institute on Deafness and Other Communication Disorders-funded "Child Language Assessment Project." CLAN harnesses the advanced computational power of the Child Language Data Exchange System archive (www.childes.talkbank.org), with an aim to develop and test fine-grained and potentially language variety-sensitive benchmarks for a range of LSA measures. Using retrospective analysis of data from AAE-speaking children, we demonstrate how CLAN LSA can facilitate dialect-fair assessment and therapy goal setting. Results Using data originally collected to norm the Diagnostic Evaluation of Language Variation, we suggest that Developmental Sentence Scoring does not appear to bias against children who speak AAE but does identify children who have language impairment (LI). Other LSA measure scores were depressed in the group of AAE-speaking children with LI but did not consistently differentiate individual children as LI. Furthermore, CLAN software permits rapid, in-depth analysis using Developmental Sentence Scoring and the Index of Productive Syntax that can identify potential intervention targets for children with developmental language disorder.

Mean Length of Utterance and Other Quantitative Measures of Spontaneous Speech in Russian-Speaking Children

Purpose This study investigated methodological and theoretical aspects of using mean length of utterance (MLU) and its alternatives in cross-linguistic research and, in particular, its applicability to Russian-a language with a rich system of grammatical and derivational morphemes. Method We collected audio recordings of spontaneous speech samples from 27 Russian-speaking children aged between 2;9 and 5;7 (years;months) over individual play sessions. For each participant, we transcribed the first 100 complete utterances and coded them for several types of utterance length measurements, including length in morphemes (grammatical and derivational), words, and syllables. At a sample level, we calculated the average number of produced unique grammatical forms, getting an alternative quantitative estimate of children's morphosyntactic abilities. Results A combination of Pearson correlation analysis and Bland-Altman difference plots established that MLU can be reliably used in Russian-speaking children aged around 3;0. The average number of unique grammatical forms remains a sensitive measurement of language capabilities even in older children aged over 3;6. Two quantitative measurements, MLU in syllables and morphemes, show good agreement, suggesting that these measurements can be used interchangeably across studies. Sample size analysis revealed that samples under 75 utterances do not provide sufficient reliability for estimating a child's MLU. Conclusions This article demonstrated that MLU can be used in young Russian-speaking children under 3;0-3;6. Also, we showed that the classical morpheme calculation approach can be substituted with counting syllables, which is more time efficient in the absence of automated parsers and is potentially more appropriate for some (e.g., polysynthetic) languages. Our proposed alternative to MLU-the average number of grammatical forms in a sample-appears to be a more sensitive measurement of language capabilities even in older children.

Percent Grammatical Utterances Between 4 and 9 Years of Age for the Edmonton Narrative Norms Instrument: Reference Data and Psychometric Properties

Purpose The purpose of this article was to provide the reference data and evaluate psychometric properties for the percent grammatical utterances (PGU; Eisenberg & Guo, 2013) in children between 4 and 9 years of age from the database of the Edmonton Narrative Norms Instrument (ENNI; Schneider, Dubé, & Hayward, 2005). Method Participants were 377 children who were between 4 and 9 years of age, including 300 children with typical language (TL) and 77 children with language impairment (LI). Narrative samples were collected using the ENNI protocol (i.e., a story generation task). PGU was computed from the samples. Split-half reliability, concurrent criterion validity, and diagnostic accuracy for PGU were further evaluated. Results PGU increased significantly in children between 4 and 9 years of age in both the TL and LI groups. In addition, the correlation coefficients for the split-half reliability and concurrent criterion validity of PGU were all large (rs ≥ .557, ps < .001). The diagnostic accuracy of PGU was also good or acceptable from ages 4 to 9 years. Conclusions With the attested psychometric properties, PGU computed from the ENNI could be used as an assessment tool for identifying children with LI between 4 and 9 years of age. The reference data of PGU could also be used for monitoring treatment progress. Supplemental Material https://doi.org/10.23641/asha.9630590.

Percent Grammatical Responses as a General Outcome Measure: Initial Validity

Purpose

This report investigated the validity of using percent grammatical responses (PGR) as a measure for assessing grammaticality. To establish construct validity, we computed the correlation of PGR with another measure of grammar skills and with an unrelated skill area. To establish concurrent validity for PGR, we computed the correlation of PGR with a previously validated measure of grammaticality, percent grammatical utterances (PGU), and examined the extent to which PGR and PGU agreed upon pass/fail decisions for children.

Method

Participants included 79 3-year-olds from mostly middle socioeconomic status homes. Language samples were elicited by asking children to describe 15 pictures in response to 4 questions per picture. To calculate PGU, children's responses to all 4 questions were segmented into communication units, and each communication unit was evaluated for grammatical errors. To calculate PGR, the entire response to just the first question was evaluated for grammatical errors.

Results

PGR scores significantly correlated with a standardized test of grammar (r = .70), but not with a measure of vocabulary (i.e., type-token ratio; r = .11). In addition, PGR scores were significantly correlated with PGU scores (r = .88). Agreement between PGR and PGU was 92% for pass decisions and 94% for fail decisions.

Conclusions

The current study establishes the construct validity of PGR as a measure of grammar and supports the use of PGR as a measure to assess grammaticality.

The Persian developmental sentence scoring as a clinical measure of morphosyntax in children

Background: Developmental Sentence Scoring (DSS) was developed as a numerical measurement and a clinical method based on the morphosyntactic acquisition in the English language. The aim of this study was to develop a new numerical tool similar to DSS to assess the morphosyntactic abilities in Persian-speaking children. Methods: In this cross-sectional and comparative study, the language samples of 115 typically developing Persian-speaking children aged 30 - 65 months were audio recorded during the free play and picture description sessions. The Persian Developmental Sentence Score (PDSS) and the Mean Length of Utterance (MLU) were calculated. Pearson correlation and one - way Analysis of variance (ANOVA) were used for data analysis. Results: The correlation between PDSS and MLU in morphemes (convergent validity) was significant with a correlation coefficient of 0.97 (p< 0.001). The value Cronbach's Alpha (α= 0.79) in the grammatical categories and the split-half coefficient (0.86) indicated acceptable internal consistency reliability. Conclusion: The PDSS could be used as a reliable numerical measurement to estimate the syntactic development in Persian-speaking children.