Speech-Language Pathologists' Use of Intelligibility Measures in Adults With Dysarthria

A comparative study of automatic vowel articulation index and auditory-perceptual assessments of speech intelligibility in Parkinson's disease

PURPOSE

The purpose of this study was to analyse the relationship between automatic vowel articulation index (aVAI) and direct magnitude estimation (DME) among speakers with Parkinson's disease (PD) and healthy controls. We further analysed the potential of aVAI to serve as an objective measure of speech impairment in the clinical setting.

METHOD

Speech samples from native Finnish speakers were utilised. Expert raters utilised DME to scale the intelligibility of speech samples. aVAI scores for PD speakers and healthy control speakers were analysed in relationship to DME speech intelligibility ratings and, among PD speakers, disease stage utilising nonparametric statistical analysis.

RESULT

Mean DME intelligibility ratings were lower among PD speakers compared to healthy controls. Mean aVAI scores were nearly the same between speaker groups. DME intelligibility ratings and aVAI were strongly correlated within the PD speaker group. aVAI and DME intelligibility ratings were moderately correlated with disease stage as measured by the Hoehn and Yahr scale.

CONCLUSION

aVAI was observed to be a promising tool for analysing vowel articulation in PD speakers. Further research is warranted on the application of aVAI as an objective measure of severity of speech impairment in the clinical setting, with varying patient populations and speech samples.

Clinical Assessment of Communication-Related Speech Parameters in Dysarthria: The Impact of Perceptual Adaptation

PURPOSE

In current clinical practice, intelligibility of dysarthric speech is commonly assessed by speech-language therapists (SLTs), in most cases by the therapist caring for the patient being diagnosed. Since SLTs are familiar with dysarthria in general and with the speech of the individual patient to be assessed in particular, they have an adaptation advantage in understanding the patient's utterances. We examined whether and how listeners' assessments of communication-related speech parameters vary as a function of their familiarity with dysarthria in general and with the diagnosed patients in particular.

METHOD

Intelligibility, speech naturalness, and perceived listener effort were assessed in 20 persons with dysarthria (PWD). Patients' speech samples were judged by the individual treating therapists, five dysarthria experts who were unfamiliar with the patients, and crowdsourced naïve listeners. Adaptation effects were analyzed using (a) linear mixed models of overall scoring levels, (b) regression models of severity dependence, (c) network analyses of between-listener and between-parameter relationships, and (d) measures of intra- and interobserver consistency.

RESULTS

Significant advantages of dysarthria experts over laypeople were found in all parameters. An overall advantage of the treating therapists over nonfamiliar experts was only seen in listening effort. Severity-dependent adaptation effects occurred in all parameters. The therapists' responses were heterogeneous and inconsistent with those of the unfamiliar experts and the naïve listeners.

CONCLUSIONS

The way SLTs evaluate communication-relevant speech parameters of the PWD whom they care for is influenced not only by adaptation benefits but also by therapeutic biases. This finding weakens the validity of assessments of communication-relevant speech parameters by the treating therapists themselves and encourages the development and use of alternative methods.

Oromotor Nonverbal Performance and Speech Motor Control: Theory and Review of Empirical Evidence

This position paper offers a perspective on the long-standing debate concerning the role of oromotor, nonverbal gestures in understanding typical and disordered speech motor control secondary to neurological disease. Oromotor nonverbal tasks are employed routinely in clinical and research settings, but a coherent rationale for their use is needed. The use of oromotor nonverbal performance to diagnose disease or dysarthria type, versus specific aspects of speech production deficits that contribute to loss of speech intelligibility, is argued to be an important part of the debate. Framing these issues are two models of speech motor control, the Integrative Model (IM) and Task-Dependent Model (TDM), which yield contrasting predictions of the relationship between oromotor nonverbal performance and speech motor control. Theoretical and empirical literature on task specificity in limb, hand, and eye motor control is reviewed to demonstrate its relevance to speech motor control. The IM rejects task specificity in speech motor control, whereas the TDM is defined by it. The theoretical claim of the IM proponents that the TDM requires a special, dedicated neural mechanism for speech production is rejected. Based on theoretical and empirical information, the utility of oromotor nonverbal tasks as a window into speech motor control is questionable.

Clinical measures of communication limitations in dysarthria assessed through crowdsourcing: specificity, sensitivity, and retest-reliability

Assessing the impact of dysarthria on a patient's ability to communicate should be an integral part of patient management. However, due to the high demands on reliable quantification of communication limitations, hardly any formal clinical tests with approved psychometric properties have been developed so far. This study investigates a web-based assessment of communication impairment in dysarthria, named KommPaS. The test comprises measures of intelligibility, naturalness, perceived listener effort and communication efficiency, as well as a total score that integrates these parameters. The approach is characterized by a quasi-random access to a large inventory of test materials and to a large group of naïve listeners, recruited via crowdsourcing. As part of a larger research program to establish the clinical applicability of this new approach, the present paper focuses on two psychometric issues, namely specificity and sensitivity (study 1) and retest-reliability (study 2). Study 1: KommPaS was administered to 54 healthy adults and 100 adult persons with dysarthria (PWD). Non-parametric criterion-based norms (specificity: 0.95) were used to derive a standard metric for each of the four component variables, and corresponding sensitivity values for the presence of dysarthria were identified. Overall classification accuracy of the total score was determined using a ROC analysis. The resulting cutscores showed a high accuracy in the separation of PWD from healthy speakers for the naturalness and the total score. Study 2: A sub-group of 20 PWD enrolled in study 1 were administered a second KommPaS examination. ICC analyses revealed good to excellent retest reliabilities for all parameters.

The Reliability and Validity of Speech-Language Pathologists’ Estimations of Intelligibility in Dysarthria

This study examined the reliability and validity of speech-language pathologists’ (SLP) estimations of speech intelligibility in dysarthria, including a visual analog scale (VAS) method and a percent estimation method commonly used in clinical settings. Speech samples from 20 speakers with dysarthria of varying etiologies were used to collect orthographic transcriptions from naïve listeners n=70 and VAS ratings and percent estimations of intelligibility from SLPs n=21. Intra- and interrater reliability for the two SLP intelligibility measures were evaluated, and the relationship between these measures was assessed. Finally, linear regression was used to evaluate the relationship between the naïve listeners’ orthographic transcription scores and the two SLP intelligibility measures. The results indicated that the intrarater reliability for both SLP intelligibility measures was strong, and the interrater reliability between the SLP ratings was moderate to excellent. A moderate positive relationship between SLPs’ VAS ratings and percent estimations was also observed. Finally, both SLPs’ percent estimations and VAS ratings were predictive of naïve listeners’ orthographic transcription scores, with SLPs’ percent estimations being the strongest predictor. In conclusion, the average SLP percent estimations and VAS ratings are valid and reliable intelligibility measures. However, the validity and reliability of these measures vary between SLPs.

Criteria for creating new standard reading passages for the assessment of speech and voice: A Delphi consensus study

Standard reading passages allow for the study of the integrated functions of speech and voice components in contextual, running speech, with target stimuli in a controlled environment. In both clinical practice and research, these texts provide rapid insight into the characteristics of the patient's speech, with fewer hesitations than in conversational speech and better predictability by the evaluator. Although a plethora of texts exist in different languages, they present various limitations. A specifically created standardised text in each language allowing for an ecological assessment of speech and voice functions, meeting most required criteria for standard speech and voice assessment and adapted to the target language's cultural and linguistic specificities, would therefore be an interesting option. However, no guidelines exist for the creation of such a reading passage. This article describes the international Delphi consensus study carried out to identify a minimal set of criteria to take into account when creating standard reading passages for an overall speech and voice assessment in adolescents and adults. This survey was conducted in three consecutive rounds; forty experts participated in the first round, with a total dropout of 17% from round 1 to round 3. It results in a minimal set of ten criteria which were selected by a majority of the experts and were rated as most important. This set contains five phoneme-level, two word-level, two sentence-level criteria and one global-level criterion. It can be used as a general guideline for the creation of standard reading passages in Indo-European Romance and Germanic languages such as English, French and German. The construction of a new reading passage in French following this guideline is briefly described.

An Information Theoretic Approach to Symbolic Learning in Synthetic Languages

An important aspect of using entropy-based models and proposed “synthetic languages”, is the seemingly simple task of knowing how to identify the probabilistic symbols. If the system has discrete features, then this task may be trivial; however, for observed analog behaviors described by continuous values, this raises the question of how we should determine such symbols. This task of symbolization extends the concept of scalar and vector quantization to consider explicit linguistic properties. Unlike previous quantization algorithms where the aim is primarily data compression and fidelity, the goal in this case is to produce a symbolic output sequence which incorporates some linguistic properties and hence is useful in forming language-based models. Hence, in this paper, we present methods for symbolization which take into account such properties in the form of probabilistic constraints. In particular, we propose new symbolization algorithms which constrain the symbols to have a Zipf–Mandelbrot–Li distribution which approximates the behavior of language elements. We introduce a novel constrained EM algorithm which is shown to effectively learn to produce symbols which approximate a Zipfian distribution. We demonstrate the efficacy of the proposed approaches on some examples using real world data in different tasks, including the translation of animal behavior into a possible human language understandable equivalent.

"You Say Severe, I Say Mild": Toward an Empirical Classification of Dysarthria Severity

PURPOSE

The main purpose of this study was to create an empirical classification system for speech severity in patients with dysarthria secondary to amyotrophic lateral sclerosis (ALS) by exploring the reliability and validity of speech-language pathologists' (SLPs') ratings of dysarthric speech.

METHOD

Ten SLPs listened to speech samples from 52 speakers with ALS and 20 healthy control speakers. SLPs were asked to rate the speech severity of the speakers using five response options: normal, mild, moderate, severe, and profound. Four severity-surrogate measures were also calculated: SLPs transcribed the speech samples for the calculation of speech intelligibility and rated the effort it took to understand the speakers on a visual analog scale. In addition, speaking rate and intelligible speaking rate were calculated for each speaker. Intrarater and interrater reliability were calculated for each measure. We explored the validity of clinician-based severity ratings by comparing them to the severity-surrogate measures. Receiver operating characteristic (ROC) curves were conducted to create optimal cutoff points for defining dysarthria severity categories.

RESULTS

Intrarater and interrater reliability for the clinician-based severity ratings were excellent and were comparable to reliability for the severity-surrogate measures explored. Clinician severity ratings were strongly associated with all severity-surrogate measures, suggesting strong construct validity. We also provided a range of values for each severity-surrogate measure within each severity category based on the cutoff points obtained from the ROC analyses.

CONCLUSIONS

Clinician severity ratings of dysarthric speech are reliable and valid. We discuss the underlying challenges that arise when selecting a stratification measure and offer recommendations for a classification scheme when stratifying patients and research participants into speech severity categories.

Measuring intelligibility in spontaneous speech using syllables perceived as understood

PURPOSE

Intelligibility, the ability to convey a message by speech, is one of the most important variables in speech-language pathology. The assessment of intelligibility is a challenge especially when it comes to spontaneous speech. The aim of the study was to investigate validity and reliability of a method for assessment of intelligibility, syllables perceived as understood (SPU); a method that is more time-efficient than previous methods based on transcription, as it does not require a master transcript for reference.

METHOD

A group of 20 adult listeners transcribed stimuli consisting of spontaneous speech from 16 children (14 with speech sound disorder and two with typical speech and language development, age 4:4 to 8:1, M = 6:0). Intelligibility was calculated based on these orthographic transcripts, as a) proportion of syllables perceived as understood (SPU) and b) proportion of syllables correctly understood (SCU), with reference to a master transcript. Validity was checked through investigation of the correlation and difference between these two measures. Reliability was analysed with inter-listener reliability by intra-class correlation.

RESULTS

The correlation between SPU and SCU (the gold standard intelligibility score) was strong and statistically significant, with SPU being consistently higher than SCU. Inter-listener reliability for single measures of intra-class correlation of the assessment by syllables perceived as understood was moderate to low, whereas the inter-listener reliability for average measures of intra-class correlation was high.

CONCLUSIONS

The method based on SPU might be used for assessment of intelligibility if the median from several listeners is used or when comparing results from the same listener over time. The SPU method might therefore be a valuable tool in a clinical and research context as a more valid option than rating scales and a more time-efficient method than the gold standard SCU method. However, it should be noted that the reliability of the SPU is not as high as for the SCU.

Inteligibilidad: diferentes perspectivas de su concepto y evaluación. Una revisión sistemática

Una adecuada inteligibilidad es un indicador del éxito en la comunicación. Su interferencia, a causa de cualquier patología, debe ser abordada en los objetivos primordiales de la terapia. No obstante, los métodos para su evaluación son poco conocidos. Objetivo: desarrollar una revisión sistemática que describa los métodos empleados en la medición de la inteligibilidad y discutir la conceptualización de este constructo. Metodología: se realiza una búsqueda en las bases de datos PubMed, ScieLo y Science Direct para identificar artículos publicados entre 1990 y 2018. Análisis y discusiones: Se seleccionan 50 investigaciones en inglés, español y portugués, que mostraron preferencia por medidas objetivas de evaluación, por sobre las medidas subjetivas de escala. La inteligibilidad se define mejor como un constructo dependiente de la señal acústica, a diferencia del concepto de comprensibilidad. Esta revisión ofrece información actualizada y organizada para optimizar procedimientos clínicos y científicos de medición de la inteligibilidad. 

Residual Neural Network precisely quantifies dysarthria severity-level based on short-duration speech segments

Recently, we have witnessed Deep Learning methodologies gaining significant attention for severity-based classification of dysarthric speech. Detecting dysarthria, quantifying its severity, are of paramount importance in various real-life applications, such as the assessment of patients' progression in treatments, which includes an adequate planning of their therapy and the improvement of speech-based interactive systems in order to handle pathologically-affected voices automatically. Notably, current speech-powered tools often deal with short-duration speech segments and, consequently, are less efficient in dealing with impaired speech, even by using Convolutional Neural Networks (CNNs). Thus, detecting dysarthria severity-level based on short speech segments might help in improving the performance and applicability of those systems. To achieve this goal, we propose a novel Residual Network (ResNet)-based technique which receives short-duration speech segments as input. Statistically meaningful objective analysis of our experiments, reported over standard Universal Access corpus, exhibits average values of 21.35% and 22.48% improvement, compared to the baseline CNN, in terms of classification accuracy and F1-score, respectively. For additional comparisons, tests with Gaussian Mixture Models and Light CNNs were also performed. Overall, the values of 98.90% and 98.00% for classification accuracy and F1-score, respectively, were obtained with the proposed ResNet approach, confirming its efficacy and reassuring its practical applicability.

Assessment of adult speech disorders: current situation and needs in French-speaking clinical practice

INTRODUCTION

Speech assessment methods used in clinical practice are varied and mainly perceptual and motor. Reliable assessment of speech disorders is essential for the tailoring of the patient's treatment plan.

OBJECTIVE

To describe current clinical practices and identify the shortcomings and needs reported by French-speaking clinicians regarding the assessment of speech disorders in adult patients.

METHODS

Data were collected using an online questionnaire for French-speaking speech and language pathologists (SLPs) in Belgium, France, Switzerland, Luxembourg, and Maghreb. Forty-nine questions were grouped into six domains: participant data, educational and occupational background, experience with speech disorders, patient population, tools and tasks for speech assessment, and possible lacks regarding the current assessment of speech disorders.

RESULTS

Responses from 119 clinicians were included in the analyses. SLPs generally use "à la carte" assessment with a large variety of tasks and speech samples. About one quarter of them do not use existing assessment batteries. Those who do mostly use them partially. Pseudo-words are rarely used and are absent from standardized batteries, in contrast to the major use of words and sentences. Perceptual evaluation largely prevails (mainly overall ratings of speech "intelligibility", "severity," and "comprehensibility" and percent-correct phonemes), whereas the recording equipment for acoustic measures is not standardized and only scarcely described by the SLPs. The most commonly used questionnaire to assess the functional impact of the speech disorder is the Voice Handicap Index; one quarter of the SLPs does not use any questionnaire. Overall, the available tools are considered only moderately satisfactory. The main reported shortcomings are a lack of objectivity and reproducibility of speech measures; exhaustiveness and consideration of specific speech parameters (prosody, speech rate, and nasality); practicality of the assessment tools.

CONCLUSION

This study highlights a lack of standardization of the speech assessment in French-speaking adults and the need to offer new reliable tools for an optimized, accurate speech assessment. The automation of these tools would allow for rapid, reproducible, and accurate measures.

Visual Analog Scale Ratings and Orthographic Transcription Measures of Sentence Intelligibility in Parkinson's Disease With Variable Listener Exposure

Purpose While orthographic transcription (OT) is the gold standard for measures of intelligibility, it is relatively inaccessible to clinicians. This study investigates the relationship between visual analog scale (VAS) ratings and OT measures of intelligibility for speakers with Parkinson's disease (PD), with the eventual goal of developing more clinically feasible assessments of intelligibility. Method Twenty speakers with PD and 5 controls read 11 sentences. First, 33 listeners completed an OT task using 1 sentence from each speaker. An additional 33 listeners rated the intelligibility of 1 sentence from each speaker using a VAS, reflecting a minimized exposure VAS (MEV) task. Lastly, 14 additional listeners each rated the intelligibility of all 11 sentences produced by all speakers using a VAS, reflecting an extended exposure VAS (EEV) task. Smaller listener groups were simulated from each VAS task for comparison to scores from the OT task. Results There was a strong relationship between OT and both MEV and EEV. This relationship remained strong (R ² ≥ .82) even when only 1 listener in MEV and 2 listeners in EEV were simulated per sentence. Conclusions VAS ratings may be a suitable alternative to OT measures of sentence intelligibility for PD using listeners with both minimal and extended exposure to the stimuli.

Assessment of intelligibility in dysarthria: development of a Maltese word and phrase list

This paper describes the development of the Maltese Intelligibility Lists (MIL) for the assessment of word and phrase intelligibility in dysarthria. Two main tools were employed: the Frenchay Dysarthria Assessment-2 (FDA), and the Maltese Language Resource Server (MLRS). Three main criteria served as the basis for the construction of the word and phrase lists: frequency of occurrence of Maltese phonemes, word frequency and an analysis of syllable types and structures. The most common 500 words in the MLRS corpus (Korpus Malti v. 3) were broadly transcribed and an analysis of different types of syllables and their frequency of occurrence was carried out. Based on this analysis, the relevant proportion of different syllable types required for the word and phrase lists for Maltese was calculated in line with the number of items present in the FDA-2. With regards to phoneme frequency, the words chosen demonstrate a similar short-vowel and consonant distribution as reported in a previous large-scale study. The MIL consists of 116 words and 50 phrases which are representative of Standard Maltese and can be used in the clinic to assess speech intelligibility in Maltese individuals with dysarthria.