Revisiting Reliability: Using Sampling Utterances and Grammatical Analysis Revised (SUGAR) to Compare 25- and 50-Utterance Language Samples

Transcription Decisions of Conjoined Independent Clauses Are Equitable Across Dialects but Impact Measurement Outcomes

PURPOSE

Transcription of conjoined independent clauses within language samples varies across professionals. Some transcribe these clauses as two separate utterances, whereas others conjoin them within a single utterance. As an inquiry into equitable practice, we examined rates of conjoined independent clauses produced by children and the impact of separating these clauses within utterances on measures of mean length of utterance (MLU) by a child's English dialect, clinical status, and age.

METHOD

The data were archival and included 246 language samples from children classified by their dialect (African American English or Southern White English) and clinical status (developmental language disorder [DLD] or typically developing [TD]), with those in the TD group further classified by their age (4 years [TD4] or 6 years [TD6]).

RESULTS

Rates of conjoined independent clauses and the impact of these clauses on MLU varied by clinical status (DLD < TD) and age (TD4 < TD6), but not by dialect. Correlations between the rate of conjoined clauses, MLU, and language test scores were also similar across the two dialects.

CONCLUSIONS

Transcription decisions regarding conjoined independent clauses within language samples lead to equitable measurement outcomes across dialects of English. Nevertheless, transcribing conjoined independent clauses as two separate utterances reduces one's ability to detect syntactic differences between children with and without DLD and document syntactic growth as children age.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25822675.

Consistency and reliability of automated language measures across expressive language samples in autism

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder with substantial clinical heterogeneity, especially in language and communication ability. There is a need for validated language outcome measures that show sensitivity to true change for this population. We used Natural Language Processing to analyze expressive language transcripts of 64 highly-verbal children and young adults (age: 6-23 years, mean 12.8 years; 78.1% male) with ASD to examine the validity across language sampling context and test-retest reliability of six previously validated Automated Language Measures (ALMs), including Mean Length of Utterance in Morphemes, Number of Distinct Word Roots, C-units per minute, unintelligible proportion, um rate, and repetition proportion. Three expressive language samples were collected at baseline and again 4 weeks later. These samples comprised interview tasks from the Autism Diagnostic Observation Schedule (ADOS-2) Modules 3 and 4, a conversation task, and a narration task. The influence of language sampling context on each ALM was estimated using either generalized linear mixed-effects models or generalized linear models, adjusted for age, sex, and IQ. The 4 weeks test-retest reliability was evaluated using Lin's Concordance Correlation Coefficient (CCC). The three different sampling contexts were associated with significantly (P < 0.001) different distributions for each ALM. With one exception (repetition proportion), ALMs also showed good test-retest reliability (median CCC: 0.73-0.88) when measured within the same context. Taken in conjunction with our previous work establishing their construct validity, this study demonstrates further critical psychometric properties of ALMs and their promising potential as language outcome measures for ASD research.

Disentangling the Grammar of 3- to 6-Year-Old Dutch Children With a Developmental Language Disorder

PURPOSE

Children with a developmental language disorder (DLD) are often delayed in their grammatical development. This is suggested to be the most important characteristic and clinical marker of DLD. However, it is unknown if this assumption is valid for young children, in the earliest stages of grammatical development. For this reason, this study investigates the complexity, diversity, and accuracy of the grammatical repertoires of 3- to 6-year-old Dutch children with DLD, in comparison to that of typically developing (TD) children matched on grammatical level.

METHOD

Language samples of 59 children (29 children with DLD and 30 TD children) were analyzed using multiple measures of grammatical complexity, diversity, and accuracy. The TD children and children with DLD were language-matched on their grammatical development using the levels of the Dutch version of the Language Assessment, Remediation, and Screening Procedure, the Taal Analyse Remediëring en Screening Procedure (TARSP; Schlichting, 2017). Thus, the children with DLD were significantly older than the TD children (respectively DLD age range: 2;7-5;4 [years;months], M _age = 4;1; and TD age range: 2;0-3;9, M _age = 2;9).

RESULTS

The results show that children with DLD are comparable to language-matched TD children in their grammatical accuracy and diversity, but that they produce less complex utterances.

CONCLUSIONS

The results indicate that children with DLD lag behind in their grammatical complexity as compared to language-matched TD children. The results also suggest that grammatical TARSP level is not sufficiently informative for selecting treatment goals. Instead, the results underline the importance of conducting language sample analyses, with special reference to the complexity of the utterances of a child with DLD.

The Reliability of Short Conversational Language Sample Measures in Children With and Without Developmental Language Disorder

PURPOSE

Language sample analysis (LSA) represents an ecologically valid method for diagnosing, identifying goals, and measuring progress in children with developmental language disorder (DLD). LSA is, however, time consuming. The purpose of this study was to determine the length of sample needed to obtain reliable LSA measures for children in kindergarten and first grade with typical language (TL) and DLD using automated analyses from the Systematic Analysis of Language Transcripts software.

METHOD

Play-based conversational language samples collected on kindergarten to first-grade children with TL (n = 21) and DLD (n = 21) from a community-based sample were analyzed. Eight LSA measures were calculated from 1-, 3-, 5-, 7-, and 10-min sample cuts and compared to 20-min samples for reliability.

RESULTS

Reliability estimates were similar for the TL and DLD groups except for errors and omissions, which showed overall higher levels of reliability in the DLD group and reached acceptable levels at 3 min. Percent grammatical utterances were reliable at 7 min in the DLD group and not reliable in shorter samples in the TL group. The subordination index was reliable at 10 min for both groups. Number of different words reached acceptable reliability at the 3-min length for the DLD group and at the 10-min length for the TL group. Utterances and words per minute were reliable at 3 min and mean length of utterance at 7 min in both groups.

CONCLUSIONS

Speech-language pathologists can obtain reliable LSA measures from shorter, 7-min conversational language samples from kindergarten to first-grade children with DLD. Shorter language samples may encourage increased use of LSA.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.19529287.

Predicting Language Performance From Narrative Language Samples

PURPOSE

Analysis of narrative language samples is a recommended clinical practice in the assessment of children's language skills, but we know little about how results from such analyses relate to overall oral language ability across the early school years. We examined the relations between language sample metrics from a short narrative retell, collected in kindergarten, and an oral language factor in grades kindergarten through 3. Our specific questions were to determine the extent to which metrics from narrative language sample analysis are concurrently related to language in kindergarten and predict language through Grade 3.

METHOD

Participants were a sample of 284 children who were administered a narrative retell task in kindergarten and a battery of vocabulary and grammar measures in kindergarten through Grade 3. Language samples were analyzed for number of different words, mean length of utterance, and a relatively new metric, percent grammatical utterances (PGUs). Structural equation models were used to estimate the concurrent and longitudinal relationships.

RESULTS

The narrative language sample metrics were consistently correlated with the individual vocabulary and grammar measures as well as the language factor in each grade, and also consistently and uniquely predicted the language factor in each grade. Standardized path estimates in the structural equation models ranged from 0.20 to 0.39.

CONCLUSIONS

This study found narrative language sample metrics to be predictive, concurrently and longitudinally, of a latent factor of language from kindergarten through Grade 3. These results further validate the importance of collecting and analyzing narrative language samples, to include PGU along with more traditional metrics, and point to directions for future research.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.17700980.

Language Sample Analysis in Clinical Practice: Speech-Language Pathologists' Barriers, Facilitators, and Needs

Purpose Most speech-language pathologists (SLPs) working with children with developmental language disorder (DLD) do not perform language sample analysis (LSA) on a regular basis, although they do regard LSA as highly informative for goal setting and evaluating grammatical therapy. The primary aim of this study was to identify facilitators, barriers, and needs related to performing LSA by Dutch SLPs working with children with DLD. The secondary aim was to investigate whether a training would change the actual performance of LSA. Method A focus group with 11 SLPs working in Dutch speech-language pathology practices was conducted. Barriers, facilitators, and needs were identified using thematic analysis and categorized using the theoretical domain framework. To address the barriers, a training was developed using software program CLAN. Changes in barriers and use of LSA were evaluated with a survey sent to participants before, directly after, and 3 months posttraining. Results The barriers reported in the focus group were SLPs' lack of knowledge and skills, time investment, negative beliefs about their capabilities, differences in beliefs about their professional role, and no reimbursement from health insurance companies. Posttraining survey results revealed that LSA was not performed more often in daily practice. Using CLAN was not the solution according to participating SLPs. Time investment remained a huge barrier. Conclusions A training in performing LSA did not resolve the time investment barrier experienced by SLPs. User-friendly software, developed in codesign with SLPs might provide a solution. For the short-term, shorter samples, preferably from narrative tasks, should be considered.