Effect of the number of presentations on listener transcriptions and reliability in the assessment of speech intelligibility in children

Comparing the Informativeness of Single-Word Samples and Connected Speech Samples in Assessing Speech Sound Disorders

Purpose Speech-language pathologists (SLPs) are faced with the challenge of quickly and accurately identifying children who present with speech sound disorders (SSD) compared to typically developing (TD) children. The goal of this study was to compare the clinical relevance of two speech sampling methods (single-word vs. connected speech samples) in how sensitive they are in detecting atypical speech sound development in children, and to know whether the information obtained from single-word samples is representative enough of children's overall speech sound performance. Method We compared the speech sound performance of 37 preschool children with SSD (M _age = 4;11 years) and 37 age-sex-matched typically developing children (M _age = 5;0 years) by eliciting their speech in two ways: (a) a picture-naming task to elicit single words, and (b) a story-retelling task to elicit connected speech. Four speech measures were compared across sample type (single words vs. connected speech) and across groups (SSD vs. TD): intelligibility, speech accuracy, phonemic inventory, and phonological patterns. Results Interaction effects were found between sample type and group on several speech sound performance measures. Single-word speech samples were found to differentiate the SSD group from the TD group, and were more sensitive than connected speech samples across various measures. The effect size of single-word samples was consistently higher than connected speech samples for three measures: intelligibility, speech accuracy, and phonemic inventory. The gap in sample type informativeness may be attributed to salience and avoidance effects, given that children tend to avoid producing unfamiliar phonemes in connected speech. The number of phonological patterns produced was the only measure that revealed no gap between two sampling types for both groups. Conclusions On measures of intelligibility, speech accuracy, and phonemic inventory, obtaining a single-word sample proved to be a more informative method of differentiating children with SSD from TD children than connected speech samples. This finding may guide SLPs in their choice of sampling type when they are under time pressure. We discuss how children's performance on the connected speech sample may be biased by salience and avoidance effects and/or task design, and may, therefore, not necessarily reveal a poorer performance than single-word samples, particularly in intelligibility, speech accuracy, and the number of phonological patterns, if these task limitations are circumvented. Our findings show that the performance gap, typically observed between the two sampling types, largely depends on which performance measures are evaluated with the speech sample. Our study is the first to address sampling type differences in SSD versus TD children and has significant clinical implications for SLPs looking for sampling types and measures that reliably identify SSD in preschool-aged children.

Measuring intelligibility in spontaneous speech using syllables perceived as understood

PURPOSE

Intelligibility, the ability to convey a message by speech, is one of the most important variables in speech-language pathology. The assessment of intelligibility is a challenge especially when it comes to spontaneous speech. The aim of the study was to investigate validity and reliability of a method for assessment of intelligibility, syllables perceived as understood (SPU); a method that is more time-efficient than previous methods based on transcription, as it does not require a master transcript for reference.

METHOD

A group of 20 adult listeners transcribed stimuli consisting of spontaneous speech from 16 children (14 with speech sound disorder and two with typical speech and language development, age 4:4 to 8:1, M = 6:0). Intelligibility was calculated based on these orthographic transcripts, as a) proportion of syllables perceived as understood (SPU) and b) proportion of syllables correctly understood (SCU), with reference to a master transcript. Validity was checked through investigation of the correlation and difference between these two measures. Reliability was analysed with inter-listener reliability by intra-class correlation.

RESULTS

The correlation between SPU and SCU (the gold standard intelligibility score) was strong and statistically significant, with SPU being consistently higher than SCU. Inter-listener reliability for single measures of intra-class correlation of the assessment by syllables perceived as understood was moderate to low, whereas the inter-listener reliability for average measures of intra-class correlation was high.

CONCLUSIONS

The method based on SPU might be used for assessment of intelligibility if the median from several listeners is used or when comparing results from the same listener over time. The SPU method might therefore be a valuable tool in a clinical and research context as a more valid option than rating scales and a more time-efficient method than the gold standard SCU method. However, it should be noted that the reliability of the SPU is not as high as for the SCU.

Optimizing linguistic materials for feature-based intelligibility assessment in speech impairments

Assessing the intelligibility of speech-disordered individuals generally involves asking them to read aloud texts such as word lists, a procedure that can be time-consuming if the materials are lengthy. This paper seeks to optimize such elicitation materials by identifying an optimal trade-off between the quantity of material needed for assessment purposes and its capacity to elicit a robust intelligibility metrics. More specifically, it investigates the effect of reducing the number of pseudowords used in a phonetic-acoustic decoding task in a speech-impaired population in terms of the subsequent impact on the intelligibility classifier as quantified by accuracy indexes (AUC of ROC, Balanced Accuracy index and F-scores). A comparison of obtained accuracy indexes shows that when reduction of the amount of elicitation material is based on a phonetic criterion-here, related to phonotactic complexity-the classifier has a higher classifying ability than when the material is arbitrarily reduced. Crucially, downsizing the material to about 30% of the original dataset does not diminish the classifier's performance nor affect its stability. This result is of significant interest to clinicians as well as patients since it validates a tool that is both reliable and efficient.

Intelligibility in Down syndrome: Effect of measurement method and listener experience

BACKGROUND

Speech intelligibility is a global indicator of the severity of a speech problem. It is a measure that has been used frequently in research and clinical assessment of speech. Previous studies have shown that factors, such as measurement method and listener experience, can influence speech intelligibility scores. However, these factors of speech intelligibility assessment have not yet been investigated in people with Down syndrome (DS).

AIMS

To compare the speech intelligibility scores in speakers with DS measured using two methods: orthographic transcription and visual analogue scale (VAS), by two groups of listeners, experienced listeners and naïve listeners. Also, to examine the relationship across the four sets of speech intelligibility scores by means of correlational analysis.

METHODS & PROCEDURES

A total of 30 adolescents and adults with DS read or repeated 12 sentences from a standardized test of intelligibility for adults with dysarthria. Each sentence was saved as a separate sound file and the 360 sentences were divided to form eight sets of stimuli. A total of 32 adults (16 experienced and 16 naïve) served as listeners of speech intelligibility. Each listener heard a single set of sentences and independently estimated the level of intelligibility for each sentence using a VAS in one task and wrote down the words perceived (i.e., orthographic transcription) in another task. The order of the two tasks was counterbalanced across listeners and the tasks were completed at least 1 week apart.

OUTCOMES & RESULTS

Repeated-measures analysis of variance (ANOVA), confirmed by mixed-methods analysis, showed that the scores obtained using orthographic transcription were significantly higher than those obtained using VAS; and the experienced listeners' scores were significantly higher than the naïve listeners' scores. Spearman rank correlation analysis showed that the four sets of scores across all conditions were strongly positively correlated with each other.

CONCLUSIONS & IMPLICATIONS

Listeners, both experienced and naïve, may udge speech in DS differently when using orthographic transcription versus VAS as the method of measurement. In addition, experienced listeners can judge speech intelligibility differently compared with listeners who are less exposed to unclear speech, which may not represent 'real-world' functional communicative ability. Speech and language therapists should be aware of the effect of these factors when measuring intelligibility scores and direct comparison of scores obtained using different procedures and by different groups of listeners is not recommended. What this paper adds What is already known on the subject Previous research on other clinical groups (e.g., Parkinson's disease) has shown that speech intelligibility scores can vary across different measurement methods and when judged by listeners with different experience. However, these factors have not yet been investigated in people with DS. What this paper adds to existing knowledge Similar to the findings reported for other clinical groups, using an impressionistic measurement method, such as VAS, can result in different speech intelligibility scores compared with scores obtained from orthographic transcription in speakers with DS. Furthermore, experienced listeners can perceive intelligibility as better compared with naïve (untrained) listeners for this group. What are the potential or actual clinical implications of this work? When measuring speech intelligibility, speech and language therapists should be aware that scores obtained using orthographic transcription can be higher than those obtained using VAS. They should also be aware that their increased exposure to hearing atypical speech may cause them to judge the speech difficulty as less severe and lead to an inaccurate representation of speech performance. Speech and language therapists should consider these factors when interpreting assessment results and especially when using intelligibility measures for treatment outcomes.

The Effects of Deep Brain Stimulation on Speech Intelligibility in Persons With Essential Tremor

Purpose The aim of this study was to investigate how deep brain stimulation (DBS) of the caudal zona incerta (cZi) affects speech intelligibility in persons with essential tremor. Method Thirty-five participants were evaluated: off stimulation, on chronic stimulation optimized to alleviate tremor, and during unilateral stimulation at increasing amplitude levels. At each stimulation condition, the participants read 10 unique nonsense sentences from the Swedish Test of Intelligibility. Two listeners, blinded to stimulation condition, transcribed all recorded sentences orthographically in a randomized procedure. A mean speech intelligibility score for each patient and stimulation condition was computed, and comparisons were made between scores off and on stimulation. Results Chronic cZi-DBS had no significant effect on speech intelligibility, and there was no difference in outcome between bilateral and unilateral treatments. During unilateral stimulation at increasing amplitudes, nine participants demonstrated deteriorating speech intelligibility. These nine participants were on average older and had more superior contacts activated during the evaluation compared with the participants without deterioration. Conclusions Chronic cZi-DBS, optimized for tremor suppression, does not generally affect speech intelligibility in persons with essential tremor. Furthermore, speech intelligibility may be preserved in many individuals, even when stimulated at high amplitudes. Adverse effects of high-amplitude unilateral stimulation observed in this study were associated with stimulation originating from a more superior location, as well as with the participants' age. These results, highlighting age and stimulation location as contributing to speech intelligibility outcomes, were, however, based on a limited number of individuals experiencing adverse effects with high-amplitude stimulation and should, therefore, be interpreted with caution.

The Effects of Modulating Fundamental Frequency and Speech Rate on the Intelligibility, Communication Efficiency, and Perceived Naturalness of Synthetic Speech

Purpose This study investigated how modulating fundamental frequency (f0) and speech rate differentially impact the naturalness, intelligibility, and communication efficiency of synthetic speech. Method Sixteen sentences of varying prosodic content were developed via a speech synthesizer. The f0 contour and speech rate of these sentences were altered to produce 4 stimulus sets: (a) normal rate with a fixed f0 level, (b) slow rate with a fixed f0 level, (c) normal rate with prosodically natural f0 variation, and (d) normal rate with prosodically unnatural f0 variation. Sixteen listeners provided orthographic transcriptions and judgments of naturalness for these stimuli. Results Sentences with f0 variation were rated as more natural than those with a fixed f0 level. Conversely, sentences with a fixed f0 level demonstrated higher intelligibility than those with f0 variation. Speech rate did not affect the intelligibility of stimuli with a fixed f0 level. Communication efficiency was highest for sentences produced at a normal rate and a fixed f0 level. Conclusions Sentence-level f0 variation increased naturalness ratings of synthesized speech, whether the variation was prosodically natural or not. However, these f0 variations reduced intelligibility. There is evidence of a trade-off in naturalness and intelligibility of synthesized speech, which may impact future speech synthesis designs. Supplemental Material https://doi.org/10.23641/asha.8847833.

An Investigation of the Clinical Use of a Single-Word Procedure to Assess Intelligibility (Swedish Test of Intelligibility for Children) and an Evaluation of the Validity and Reliability of the Intelligibility in Context Scale

Purpose Intelligibility is a core concept of speech-language pathology, central both to the assessment of speech disorders and to intervention in such disorders. One purpose of this study was to investigate the reliability and usability of a single-word assessment procedure, the Swedish Test of Intelligibility for Children (STI-CH), in a clinical setting. Another purpose was to investigate the validity and reliability of an assessment method designed to assess functional intelligibility: the Intelligibility in Context Scale (ICS). Method Thirty children aged 4;1-10;1 years;months were recruited from speech-language pathology clinics in Western Sweden. Each child's treating speech-language pathologist served as listener in the STI-CH assessment while the ICS was completed by each child's parents. External listeners (2 last-year speech-language pathology students) were used to assess the validity of the speech-language pathology. Results The mean duration of the test procedure for the STI-CH was about 19 min, and 57% of the test sessions were reported as difficult. There was a weak but statistically significant correlation between the results from the STI-CH and the ICS ( r = .40, p < .05), and both methods showed high reliability in terms of interlistener reliability (intraclass correlation exceeding .97) and internal consistency, respectively. Conclusions The STI-CH had high reliability and was time efficient but had some procedural problems. The ICS had moderate validity but high reliability. The STI-CH is promising for clinical use but needs to be developed further. The validity of the ICS can be discussed and needs to be further investigated.