Clinical measures of communication limitations in dysarthria assessed through crowdsourcing: specificity, sensitivity, and retest-reliability

Comparing online versus laboratory measures of speech perception in older children and adolescents

Given the increasing prevalence of online data collection, it is important to know how behavioral data obtained online compare to samples collected in the laboratory. This study compares online and in-person measurement of speech perception in older children and adolescents. Speech perception is important for assessment and treatment planning in speech-language pathology; we focus on the American English /ɹ/ sound because of its frequency as a clinical target. Two speech perception tasks were adapted for web presentation using Gorilla: identification of items along a synthetic continuum from rake to wake, and category goodness judgment of English /ɹ/ sounds in words produced by various talkers with and without speech sound disorder. Fifty typical children aged 9-15 completed these tasks online using a standard headset. These data were compared to a previous sample of 98 typical children aged 9-15 who completed the same tasks in the lab setting. For the identification task, participants exhibited smaller boundary widths (suggestive of more acute perception) in the in-person setting relative to the online setting. For the category goodness judgment task, there was no statistically significant effect of modality. The correlation between scores on the two tasks was significant in the online setting but not in the in-person setting, but the difference in correlation strength was not statistically significant. Overall, our findings agree with previous research in suggesting that online and in-person data collection do not yield identical results, but the two contexts tend to support the same broad conclusions. In addition, these results suggest that online data collection can make it easier for researchers connect with a more representative sample of participants.

Clinical Practice in Childhood Dysarthria: An Online Survey of German-Speaking Speech-Language Pathologists

PURPOSE

This survey study aimed to establish current clinical practices of German-speaking speech-language pathologists (SLPs) regarding their assessment and treatment of communication disorders in children with neurological conditions, with a particular focus on the management of childhood dysarthria.

METHOD

A 23-question cross-sectional online survey was disseminated to practicing SLPs in Germany, Austria, and Switzerland via relevant professional bodies. SLPs were invited to provide information on their current assessment and treatment practices. Demographic data including case load and clinical settings were also gathered to contextualize practices.

RESULTS

One hundred two SLPs responded to the survey, of which 68 valid responses were analyzed. German-speaking SLPs comprehensively assess and treat various aspects of overall communication, language, and swallowing functions in children with neurological conditions. Speech motor aspects did not represent a main intervention focus. In cases where the dysarthric component was targeted, specific approaches for childhood dysarthria were rarely used. Instead, SLPs reported using approaches developed for speech disorders other than dysarthria.

CONCLUSIONS

German-speaking SLPs working with children with neurological conditions use various assessment and treatment methods to support children's communication. However, dysarthria-specific approaches were not an established part of clinical practice. Results of the survey highlight the need for access to relevant developments in German and for evaluation of current curricula for speech-language pathology students and continuing education opportunities for practicing clinicians.

Clinical Assessment of Communication-Related Speech Parameters in Dysarthria: The Impact of Perceptual Adaptation

PURPOSE

In current clinical practice, intelligibility of dysarthric speech is commonly assessed by speech-language therapists (SLTs), in most cases by the therapist caring for the patient being diagnosed. Since SLTs are familiar with dysarthria in general and with the speech of the individual patient to be assessed in particular, they have an adaptation advantage in understanding the patient's utterances. We examined whether and how listeners' assessments of communication-related speech parameters vary as a function of their familiarity with dysarthria in general and with the diagnosed patients in particular.

METHOD

Intelligibility, speech naturalness, and perceived listener effort were assessed in 20 persons with dysarthria (PWD). Patients' speech samples were judged by the individual treating therapists, five dysarthria experts who were unfamiliar with the patients, and crowdsourced naïve listeners. Adaptation effects were analyzed using (a) linear mixed models of overall scoring levels, (b) regression models of severity dependence, (c) network analyses of between-listener and between-parameter relationships, and (d) measures of intra- and interobserver consistency.

RESULTS

Significant advantages of dysarthria experts over laypeople were found in all parameters. An overall advantage of the treating therapists over nonfamiliar experts was only seen in listening effort. Severity-dependent adaptation effects occurred in all parameters. The therapists' responses were heterogeneous and inconsistent with those of the unfamiliar experts and the naïve listeners.

CONCLUSIONS

The way SLTs evaluate communication-relevant speech parameters of the PWD whom they care for is influenced not only by adaptation benefits but also by therapeutic biases. This finding weakens the validity of assessments of communication-relevant speech parameters by the treating therapists themselves and encourages the development and use of alternative methods.

Speech Naturalness in the Assessment of Childhood Dysarthria

PURPOSE

This study investigated perceived speech naturalness estimated by adult listeners in typically developing children and children with dysarthria. We aimed to identify predictors of naturalness among auditory-perceptual parameters and to evaluate the concept of naturalness as a clinical marker of childhood dysarthria.

METHOD

In a listening experiment, naive adult listeners rated speech naturalness of 144 typically developing children (3-9 years old) and 28 children with neurological conditions (5-9 years old) on a visual analog scale. Speech samples were recorded using the materials of the Bogenhausen Dysarthria Scales-Childhood Dysarthria, which also provides for auditory-perceptual judgments covering all speech subsystems.

RESULTS

Children with dysarthria obtained significantly lower naturalness ratings compared to typically developing children. However, there was a substantial age effect observable in the typically developing children; that is, younger typically developing children were also perceived as somewhat unnatural. The ratings of the typically developing children were influenced by the occurrence of developmental speech features; for the children with neurological conditions, specific symptoms of dysarthria had an additional effect. In both groups, the perception of naturalness was predominantly determined by the children's articulation and intelligibility.

CONCLUSIONS

Both symptoms of childhood dysarthria and developmental speech features (e.g., regarding articulation and intelligibility) were associated to some extent with unnatural speech by the listeners. Thus, perceived speech naturalness appears less suitable as a marker of dysarthria in children than in adults.

Crowdsourced Perceptual Ratings of Voice Quality in People With Parkinson's Disease Before and After Intensive Voice and Articulation Therapies: Secondary Outcome of a Randomized Controlled Trial

PURPOSE

Limited research has examined the suitability of crowdsourced ratings to measure treatment effects in speakers with Parkinson's disease (PD), particularly for constructs such as voice quality. This study obtained measures of reliability and validity for crowdsourced listeners' ratings of voice quality in speech samples from a published study. We also investigated whether aggregated listener ratings would replicate the original study's findings of treatment effects based on the Acoustic Voice Quality Index (AVQI) measure.

METHOD

This study reports a secondary outcome measure of a randomized controlled trial with speakers with dysarthria associated with PD, including two active comparators (Lee Silverman Voice Treatment [LSVT LOUD] and LSVT ARTIC), an inactive comparator (untreated PD), and a healthy control group. Speech samples from three time points (pretreatment, posttreatment, and 6-month follow-up) were presented in random order for rating as "typical" or "atypical" with respect to voice quality. Untrained listeners were recruited through the Amazon Mechanical Turk crowdsourcing platform until each sample had at least 25 ratings.

RESULTS

Intrarater reliability for tokens presented repeatedly was substantial (Cohen's κ = .65-.70), and interrater agreement significantly exceeded chance level. There was a significant correlation of moderate magnitude between the AVQI and the proportion of listeners classifying a given sample as "typical." Consistent with the original study, we found a significant interaction between group and time point, with the LSVT LOUD group alone showing significantly higher perceptually rated voice quality at posttreatment and follow-up relative to the pretreatment time point.

CONCLUSIONS

These results suggest that crowdsourcing can be a valid means to evaluate clinical speech samples, even for less familiar constructs such as voice quality. The findings also replicate the results of the study by Moya-Galé et al. (2022) and support their functional relevance by demonstrating that the effects of treatment measured acoustically in that study are perceptually apparent to everyday listeners.

Indicators of Communication Limitation in Dysarthria and Their Relation to Auditory-Perceptual Speech Symptoms: Construct Validity of the KommPaS Web App

PURPOSE

Despite extensive research into communication-related parameters in dysarthria, such as intelligibility, naturalness, and perceived listener effort, the existing evidence has not been translated into a clinically applicable, comprehensive, and valid diagnostic tool so far. This study addresses Communication-Related Parameters in Speech Disorders (KommPaS), a new web-based diagnostic instrument for measuring indices of communication limitation in individuals with dysarthria through online crowdsourcing. More specifically, it answers questions about the construct validity of KommPaS. In the first part, the interrelationship of the KommPaS variables intelligibility, naturalness, perceived listener effort, and speech rate were explored in order to draw a comprehensive picture of a patient's limitations and avoid the collection of redundant information. Second, the influences of motor speech symptoms on the KommPaS variables were studied in order to delineate the structural relationships between two complementary diagnostic perspectives.

METHOD

One hundred persons with dysarthria of different etiologies and varying degrees of severity were examined with KommPaS to obtain layperson-based data on communication-level parameters, and with the Bogenhausen Dysarthria Scale (BoDyS) to obtain expert-based, function-level data on dysarthria symptoms. The internal structure of the KommPaS variables and their dependence on the BoDyS variables were analyzed using structural equation modeling.

RESULTS

Despite a high multicollinearity, all KommPaS variables were shown to provide complementary diagnostic information and their mutual interconnections were delineated in a path graph model. Regarding the influence of the BoDyS scales on the KommPaS variables, separate linear regression models revealed plausible predictor sets. A complete path model of KommPaS and BoDyS variables was developed to map the complex interplay between variables at the functional and the communication levels of dysarthria assessment.

CONCLUSION

In validating a new clinical tool for the diagnostics of communication limitations in dysarthria, this study is the first to draw a comprehensive picture of how auditory-perceptual characteristics of dysarthria interact at the levels of expert-based functional and layperson-based communicative assessments.