Using Free Computer-Assisted Language Sample Analysis to Evaluate and Set Treatment Goals for Children Who Speak African American English

Early Identification of Language Disorders Using Natural Language Processing and Machine Learning: Challenges and Emerging Approaches

PURPOSE

Recent advances in artificial intelligence provide opportunities to capture and represent complex features of human language in a more automated manner, offering potential means of improving the efficiency of language assessment. This review article presents computerized approaches for the analysis of narrative language and identification of language disorders in children.

METHOD

We first describe the current barriers to clinicians' use of language sample analysis, narrative language sampling approaches, and the data processing stages that precede analysis. We then present recent studies demonstrating the automated extraction of linguistic features and identification of developmental language disorder using natural language processing and machine learning. We explain how these tools operate and emphasize how the decisions made in construction impact their performance in important ways, especially in the analysis of child language samples. We conclude with a discussion of major challenges in the field with respect to bias, access, and generalizability across settings and applications.

CONCLUSION

Given the progress that has occurred over the last decade, computer-automated approaches offer a promising opportunity to improve the efficiency and accessibility of language sample analysis and expedite the diagnosis and treatment of language disorders in children.

Using PAALSS for the manual analysis of language samples of individuals who use aided AAC in Spanish: A pilot study

This pilot study describes the application of the Protocol for the Analysis of Aided Language Samples in Spanish (PAALSS), specifically designed for the manual analysis of language samples from individuals in the early stages of Spanish aided language development. Data were collected from 22 language samples from 16 individuals who use aided AAC and are at the earlier stages of Spanish language development. The primary objective of this study was to explore the feasibility of using PAALSS as an analytical tool to describe various aspects of the language samples, including lexical productivity, lexical diversity, morphology, grammatical complexity, and syntax. Results are presented according to four different groupings, based on the language samples' grammatical complexity scores. The study provides preliminary evidence of the potential of PAALSS as a useful tool for the manual analysis of language samples from users of AAC in Spanish. However, future studies are needed to establish its formal psychometric and measurement properties.

Using language sample analyses across English dialects: A case-based approach for preschoolers

This study compared language samples from typically developing 4-year-olds who spoke African American English (AAE), Jamaican English (JE), or Mainstream American English (MAE) to assess the value of using language sample analysis (LSA) measures for characterising language use across dialects of English. Specific LSA metrics included mean length of utterance (MLU) in morphemes and in words, the Index of Productive Syntax (IPSyn), Developmental Sentence Scoring (DSS) and measures of lexical diversity. Children demonstrated diverse linguistic patterns across dialects, but a Kruskal-Wallis H test did not reveal significant differences in scores obtained through LSA measures. Notably, the IPSyn captured morphosyntactic structures in each category across dialects where prior research has highlighted limitations. This preliminary study uses a case-based approach to illustrate the applicability of LSAs in describing linguistic variations across children who speak different dialects of English. Moreover, the findings from this study underscore the potential use of LSAs in describing linguistic patterns to support the characterisation of communication profiles for culturally and linguistically diverse children.

Should We Stop Using Lexical Diversity Measures in Children's Language Sample Analysis?

PURPOSE

Prior work has identified weaknesses in commonly used indices of lexical diversity in spoken language samples, such as type-token ratio (TTR) due to sample size and elicitation variation, we explored whether TTR and other diversity measures, such as number of different words/100 (NDW), vocabulary diversity (VocD), and the moving average TTR would be more sensitive to child age and clinical status (typically developing [TD] or developmental language disorder [DLD]) if samples were obtained from standardized prompts.

METHOD

We utilized archival data from the norming samples of the Test of Narrative Language and the Edmonton Narrative Norms Instrument. We examined lexical diversity and other linguistic properties of the samples, from a total of 1,048 children, ages 4-11 years; 798 of these were considered TD, whereas 250 were categorized as having a language learning disorder.

RESULTS

TTR was the least sensitive to child age or diagnostic group, with good potential to misidentify children with DLD as TD and TD children as having DLD. Growth slopes of NDW were shallow and not very sensitive to diagnostic grouping. The strongest performing measure was VocD. Mean length of utterance, TNW, and verbs/utterance did show both good growth trajectories and ability to distinguish between clinical and typical samples.

CONCLUSIONS

This study, the largest and best controlled to date, re-affirms that TTR should not be used in clinical decision making with children. A second popular measure, NDW, is not measurably stronger in terms of its psychometric properties. Because the most sensitive measure of lexical diversity, VocD, is unlikely to gain popularity because of reliance on computer-assisted analysis, we suggest alternatives for the appraisal of children's expressive vocabulary skill.

Transcription Decisions of Conjoined Independent Clauses Are Equitable Across Dialects but Impact Measurement Outcomes

PURPOSE

Transcription of conjoined independent clauses within language samples varies across professionals. Some transcribe these clauses as two separate utterances, whereas others conjoin them within a single utterance. As an inquiry into equitable practice, we examined rates of conjoined independent clauses produced by children and the impact of separating these clauses within utterances on measures of mean length of utterance (MLU) by a child's English dialect, clinical status, and age.

METHOD

The data were archival and included 246 language samples from children classified by their dialect (African American English or Southern White English) and clinical status (developmental language disorder [DLD] or typically developing [TD]), with those in the TD group further classified by their age (4 years [TD4] or 6 years [TD6]).

RESULTS

Rates of conjoined independent clauses and the impact of these clauses on MLU varied by clinical status (DLD < TD) and age (TD4 < TD6), but not by dialect. Correlations between the rate of conjoined clauses, MLU, and language test scores were also similar across the two dialects.

CONCLUSIONS

Transcription decisions regarding conjoined independent clauses within language samples lead to equitable measurement outcomes across dialects of English. Nevertheless, transcribing conjoined independent clauses as two separate utterances reduces one's ability to detect syntactic differences between children with and without DLD and document syntactic growth as children age.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25822675.

Grammaticality Judgments of Tense and Agreement by Children With and Without Developmental Language Disorder Across Dialects of English

PURPOSE

Within General American English (GAE), the grammar weaknesses of children with developmental language disorder (DLD) have been documented with many tasks, including grammaticality judgments. Recently, Vaughn et al. replicated this finding with a judgment task targeting tense and agreement (T/A) structures for children who spoke African American English (AAE), a dialect that contains a greater variety of T/A forms than GAE. In the current study, we further tested this finding for children who spoke Southern White English (SWE), another dialect that contains a greater variety of T/A forms than GAE but less variety than AAE. Then, combining the SWE and AAE data, we explored the effects of a child's dialect, clinical group, and production of T/A forms on the children's judgments.

METHOD

The data were from 88 SWE-speaking children (DLD, n = 18; typically developing [TD], n = 70) and 91 AAE-speaking children (DLD, n = 34; TD, n = 57) previously studied. As in the AAE study, the SWE judgment data were examined both with A' scores and percentages of acceptability, with comparisons between dialects made on percentages of acceptability.

RESULTS

As in AAE, the SWE DLD group had significantly different A' scores and percentages of acceptability than the SWE TD group for all sentence types, including those with T/A structures. Additional analyses indicated that the judgments of the TD but not the DLD groups showed dialect effects. Except for verbal -s, overt production and grammaticality judgments were correlated for the TD but not for the DLD groups.

CONCLUSIONS

Children with DLD across dialects of English present grammar difficulties that affect their ability to make judgments about sentences. More cross-dialectal research is needed to better understand the grammatical weaknesses of childhood DLD, especially for structures such as verbal -s that are expressed differently across dialects of English.

Assessment and Therapy Goal Planning Using Free Computerized Language Analysis Software

Background

We discuss a free software system (Computerized Language Analysis [CLAN]) that can enable fast, thorough, and informative language sample analysis (LSA).

Method

We describe methods for eliciting, transcribing, analyzing, and interpreting language samples. Using a hypothetical child speaker, we illustrate use KidEval to generate a diagnostic report.

Results

Given LSA results suggestive of expressive language delay, we analyze further using CLAN's Developmental Sentence Score and Index of Productive Syntax routines, and outline the child's use of Brown's morphemes.

Discussion

This tutorial provides an introduction to the use of free CLAN software. We discuss how LSA results can be used to structure therapy goals that address specific aspects of grammatical structure that the child may not yet demonstrate in their spoken language. Finally, we provide answers to common questions, including user support.

Sharpening Our Tools: A Systematic Review to Identify Diagnostically Accurate Language Sample Measures

PURPOSE

This systematic review provides a comprehensive summary of the diagnostic accuracy of English language sample analysis (LSA) measures for the identification of developmental language disorder.

METHOD

An electronic database search was conducted to identify English publications reporting empirical data on the diagnostic accuracy of English LSA measures for children aged 3 years or older.

RESULTS

Twenty-eight studies were reviewed. Studies included between 18 and 676 participants ranging in age from 3;0 to 13;6 (years;months). Analyzed measures targeted multiple linguistic domains, and diagnostic accuracy ranged from less than 25% to greater than 90%. Morphosyntax measures achieved the highest accuracy, especially in combination with length measures, and at least one acceptable measure was identified for each 1-year age band up to 10 years old.

CONCLUSION

Several LSA measures or combinations of measures are clinically useful for the identification of developmental language disorder, although more research is needed to replicate findings using rigorous methods and to explore measures that are informative for adolescents and across diverse varieties of English.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21183247.

Measurement of Lexical Diversity in Children's Spoken Language: Computational and Conceptual Considerations

Background

Type-Token Ratio (TTR), given its relatively simple hand computation, is one of the few LSA measures calculated by clinicians in everyday practice. However, it has significant well-documented shortcomings; these include instability as a function of sample size, and absence of clear developmental profiles over early childhood. A variety of alternative measures of lexical diversity have been proposed; some, such as Number of Different Words/100 (NDW) can also be computed by hand. However, others, such as Vocabulary Diversity (VocD) and the Moving Average Type Token Ratio (MATTR) rely on complex resampling algorithms that cannot be conducted by hand. To date, no large-scale study of all four measures has evaluated how well any capture typical developmental trends over early childhood, or whether any reliably distinguish typical from atypical profiles of expressive child language ability.

Materials and Methods

We conducted linear and non-linear regression analyses for TTR, NDW, VocD, and MATTR scores for samples taken from 946 corpora from typically developing preschool children (ages 2-6 years), engaged in adult-child toy play, from the Child Language Data Exchange System (CHILDES). These were contrasted with 504 samples from children known to have delayed expressive language skills (total n = 1,454 samples). We also conducted a separate sub-analysis which examined possible contextual effects of sampling environment on lexical diversity.

Results

Only VocD showed significantly different mean scores between the typically -developing children and delayed developing children group. Using TTR would actually misdiagnose typical children and miss children with known language impairment. However, computation of VocD as a function of toy interactions was significant and emerges as a further caution in use of lexical diversity as a valid proxy index of children's expressive vocabulary skill.

Discussion

This large scale statistical comparison of computer-implemented algorithms for expressive lexical profiles in young children with traditional, hand-calculated measures showed that only VocD met criteria for evidence-based use in LSA. However, VocD was impacted by sample elicitation context, suggesting that non-linguistic factors, such as engagement with elicitation props, contaminate estimates of spoken lexical skill in young children. Implications and suggested directions are discussed.

Dynamic Norming and Open Science

In a recent issue of JSLHR, Tucci et al. (2022) presented a method for assigning SEM scores to a language sample. However, this method is based on data that are not publicly available and uses a commercial analysis program that is not open source. The TalkBank system and the Child Language Data Exchange System database provides free analysis software based on openly accessible data, thereby adhering to Open Science standards, which represent an important next step for the fields of speech and hearing.

The Index of Productive Syntax: Psychometric Properties and Suggested Modifications

PURPOSE

The Index of Productive Syntax (IPSyn) is a well-known language sample analysis tool. However, its psychometric properties have not been assessed across a wide sample of typically developing preschool-age children and children with language disorders. We sought to determine the profile of IPSyn scores by age over early childhood. We additionally explored whether the IPSyn could be shortened to fewer items without loss of information and whether the required language sample could be shortened from a current required number of 100 utterances to 50.

METHOD

We used transcripts from the Child Language Data Exchange System, including 1,051 samples of adult-child conversational play with toys within the theoretical framework of item response theory. Samples included those from typically developing children as well as children with hearing loss, Down syndrome, and late language emergence.

RESULTS

The Verb Phrase and Sentence Structure subscales showed more stable developmental trajectories over the preschool years and greater differentiation between typical and atypical cohorts than did the Noun Phrase and Question/Negation subscales. A number of current IPSyn scoring items can be dropped without loss of information, and 50-utterance samples demonstrate most of the same psychometric properties of longer samples.

DISCUSSION

Our findings suggest ways in which the IPSyn can be automated and streamlined (proposed IPSyn-C) so as to provide useful clinical guidance with fewer items and a shorter required language sample. Reference values for the IPSyn-C are provided. Trajectories for one subscale (Question/Negation) appear inherently unstable and may require structured elicitation. Potential limitations, ramifications, and future directions are discussed.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.16915690.

An Evaluation of Expedited Transcription Methods for School-Age Children's Narrative Language: Automatic Speech Recognition and Real-Time Transcription

Purpose This study examined the accuracy and potential clinical utility of two expedited transcription methods for narrative language samples elicited from school-age children (7;5-11;10 [years;months]) with developmental language disorder. Transcription methods included real-time transcription produced by speech-language pathologists (SLPs) and trained transcribers (TTs) as well as Google Cloud Speech automatic speech recognition. Method The accuracy of each transcription method was evaluated against a gold-standard reference corpus. Clinical utility was examined by determining the reliability of scores calculated from the transcripts produced by each method on several language sample analysis (LSA) measures. Participants included seven certified SLPs and seven TTs. Each participant was asked to produce a set of six transcripts in real time, out of a total 42 language samples. The same 42 samples were transcribed using Google Cloud Speech. Transcription accuracy was evaluated through word error rate. Reliability of LSA scores was determined using correlation analysis. Results Results indicated that Google Cloud Speech was significantly more accurate than real-time transcription in transcribing narrative samples and was not impacted by speech rate of the narrator. In contrast, SLP and TT transcription accuracy decreased as a function of increasing speech rate. LSA metrics generated from Google Cloud Speech transcripts were also more reliably calculated. Conclusions Automatic speech recognition showed greater accuracy and clinical utility as an expedited transcription method than real-time transcription. Though there is room for improvement in the accuracy of speech recognition for the purpose of clinical transcription, it produced highly reliable scores on several commonly used LSA metrics. Supplemental Material https://doi.org/10.23641/asha.15167355.

Forum: Serving African American English Speakers in Schools Through Interprofessional Education & Practice

Purpose African American English (AAE) speakers often face mismatches between home language and school language, coupled with negative attitudes toward AAE in the classroom. This forum, Serving African American English Speakers in Schools Through Interprofessional Education & Practice, will help researchers, parents, and school-based practitioners communicate in ways that are synergistic, collaborative, and transparent to improve educational outcomes of AAE speakers. Method The forum includes a tutorial offering readers instructions on how to engage in community-based participatory research (Holt, 2021). Through two clinical focus articles, readers will recognize how AAE develops during the preschool years and is expressed across various linguistic contexts and elicitation tasks (Newkirk-Turner & Green, 2021) and identify markers of developmental language disorder within AAE from language samples analyzed in Computerized Language Analysis (Overton et al., 2021). Seven empirical articles employ such designs as quantitative (Byrd & Brown, 2021; Diehm & Hendricks, 2021; Hendricks & Jimenez, 2021; Maher et al., 2021; Mahurin-Smith et al., 2021), qualitative (Hamilton & DeThorne, 2021), and mixed methods (Mills et al., 2021). These articles will help readers identify ways in which AAE affects how teachers view its speakers' language skills and communicative practices and relates to its speakers' literacy outcomes. Conclusion The goal of the forum is to make a lasting contribution to the discipline with a concentrated focus on how to assess and address communicative variation in the U.S. classroom.