Can You See My Screen? Virtual Assessment in Speech and Language

Amyotrophic Lateral Sclerosis-Bulbar Dysfunction Index-Remote: Test-Retest and Interrater Reliability of Candidate Items

PURPOSE

The primary aim of this study was to establish the reliability of candidate items as a step in the development of the Amyotrophic Lateral Sclerosis-Bulbar Dysfunction Index-Remote (ALS-BDI-Remote), a novel tool being developed for the detection and monitoring of bulbar signs and symptoms in remote settings.

METHOD

The set of candidate items included 40 items covering three domains: cranial nerve examination, auditory-perceptual evaluation, and functional assessment. Forty-eight participants diagnosed with ALS and exhibiting a range of bulbar disease severity were included. Data collection for each participant took place on Zoom over three sessions. During Session 1, the participants were instructed to adjust their Zoom settings and to optimize their recording environment (e.g., lighting, background noise). Their cognition and eating were screened to determine their ability to follow instructions and their eligibility to perform the swallowing and chewing tasks. During Session 2, two speech-language pathologists (SLPs) administered the tool consecutively to determine the items' interrater reliability. During Session 3, one of the SLPs readministered the tool within 2 weeks of Session 1 to assess test-retest reliability. The reliability of each item was estimated using weighted kappa and the percentage of agreement. To be considered reliable, the items had to reach a threshold of 0.5 weighted kappa or 80% percentage agreement (if skewed distribution of the scores) for both interrater and test-retest reliability.

RESULTS

In total, 33 of the 40 candidate items reached the reliability cutoff for both reliability analyses. All assessment domains included reliable items. Items requiring very good visualization of structures or movements were generally less reliable.

CONCLUSIONS

This study resulted in the selection of reliable items to be included in the next version of the ALS-BDI-Remote, which will undergo psychometric evaluation (reliability, validity, and responsiveness analyses). Additionally, the results contributed to our understanding of the remote administration of SLP assessments for telehealth applications.

Remote First-Language Assessment: Feasibility Study With Vietnamese Bilingual Children and Their Caregivers

PURPOSE

There is a shortage of bilingual speech-language pathologists (SLPs) in the United States. For Vietnamese, less than 1% of SLPs speak the language compared with a Vietnamese American population of > 2.1 million. This study examines the feasibility and social validity of remote child language assessment with the help of a caregiver to address the need for first language assessments among Vietnamese-speaking children.

METHOD

Twenty-one dyads of caregivers and typically developing children (aged 3-6 years) completed two assessment sessions in their first language, Vietnamese, using Zoom videoconferencing. Sessions were counterbalanced between two conditions in which either the clinician or the caregiver was the task administrator. Children's language samples were elicited using narrative tasks. Social validity was also assessed through caregiver and child questionnaires at the end of each session.

RESULTS

There were no significant differences between conditions on language sample measures nor the measures of social validity. Both caregivers and their children felt positively about the sessions. The caregivers' feelings were related to their perception of children's feelings about the sessions. Children's feelings were related to their Vietnamese language proficiency, caregiver-reported language ability, and whether they were born outside of the United States.

CONCLUSIONS

Findings build the evidence base for telepractice as an effective and socially valid service delivery model for bilingual children in the United States. This study supports the potential for caregivers as task administrators in a telepractice setting, making assessment in a child's first language more feasible and accessible. Future investigation is needed to extend results to bilingual populations with disorders.

Online assessment of narrative macrostructure in adult Irish-English multilinguals

Background

Online assessment of narrative production and comprehension became an important component of language assessment during the COVID-19 pandemic. This study aimed to establish quantitative measures of narrative macrostructure in the production and comprehension of adult Irish-English bilinguals in an online assessment.

Methods

A total of 30 Irish-English bilingual adults participated in an online assessment of oral narrative production and comprehension. Narratives were elicited using LITMUS-MAIN for Irish and English. Story-tell elicitation method was used for all stories. Twenty participants produced Baby Birds and Baby Goats story pairs while 10 participants produced Cat and Dog story pairs. Quantitative measures of story structure, comprehension score, and the overall number of Internal State Terms (ISTs) in production and comprehension were compared across the story pairs, languages, and the output type (production vs. comprehension).

Results

A general linear model indicated no differences in either story structure or story comprehension scores across languages for both sets of stories. Combined analysis for all participants and stories indicated no difference in the story structure scores or comprehension scores across the languages or the story pairs. While the overall number of ISTs was the same across languages, a higher number of ISTs was observed in comprehension relative to production in both languages for Cat and Dog story pair only, but not for Baby birds and Baby goats' stories. The major benefit of using online assessment was the accessibility of participants. The major drawback was the inability to control the environment and the quality of the internet connection.

Conclusion and implications

While online assessment increased the availability of participants, which is a significant factor in rural Ireland characterized by low population density and the high percentage of Irish speakers, the availability of stable internet connection limited the applicability of online assessment. Measures of narrative macrostructure were stable across the languages and the story pairs. This is important because of high variability in exposure to Irish, frequent code-switching, and a high number of morphosyntactic errors due to rapid language change that characterizes Irish-English bilinguals. Identifying reliable measures of language performance for Irish-English adult speakers is an important step toward establishing developmental norms for Irish-English bilinguals.