A Comparison of Manual Versus Automated Quantitative Production Analysis of Connected Speech

Differences in Connected Speech Outcomes Across Elicitation Methods

Background

Connected speech is often used to assess many aspects of an individual's language abilities after stroke. However, it is unknown the degree to which elicitation methods differ in generating structural and syntactic aspects of connected speech, two critical components of successful communication. Quantifying the degree to which elicitation methods differ in eliciting structurally, syntactically, and lexically complex connected speech at the earliest stage of stroke before reorganization and rehabilitation of function independent of clinical diagnosis of aphasia has not been examined to date. Addressing this gap has implications for early clinical intervention as well as empirical studies of connected speech production.

Aims

We compared two common elicitation methods, picture description and storytelling on lexical, structural, and syntactic measures of connected speech in speakers during the acute stage of left hemisphere stroke.

Methods & Procedures

We measured connected speech using an automated quantitative production analysis approach (Fromm et al., 2021) in 71 native-English speaking participants (27 female; 59 ± 13 years) within an average 3.9 days from left hemisphere stroke onset. We tested the degree of agreement and consistency between elicitation methods for lexical, structural, and syntactic measures of connected speech, as well as the degree of concordance in classifying deficits across individuals.

Outcomes & Results

Storytelling elicited significantly more words and more structurally complex, lexically diverse, and syntactically accurate speech in comparison to picture description. Elicitation methods differed in measuring outcomes across participants for the lexical and syntactic, but not structural complexity aspects of connected speech where storytelling classified more participants with impairments in comparison to picture description.

Conclusions

These differences suggest storytelling provides assessment of connected speech abilities more reflective of real-world abilities where its use is particularly critical for examining individual differences and providing diagnoses of acute stroke language deficits. As a result, using storytelling as a connected speech elicitation method more effectively captures a patient's language capabilities after stroke, consequently informing clinical diagnosis and treatment.

Automation of Language Sample Analysis

PURPOSE

A major barrier to the wider use of language sample analysis (LSA) is the fact that transcription is very time intensive. Methods that can reduce the required time and effort could help in promoting the use of LSA for clinical practice and research.

METHOD

This article describes an automated pipeline, called Batchalign, that takes raw audio and creates full transcripts in Codes for the Human Analysis of Talk (CHAT) transcription format, complete with utterance- and word-level time alignments and morphosyntactic analysis. The pipeline only requires major human intervention for final checking. It combines a series of existing tools with additional novel reformatting processes. The steps in the pipeline are (a) automatic speech recognition, (b) utterance tokenization, (c) automatic corrections, (d) speaker ID assignment, (e) forced alignment, (f) user adjustments, and (g) automatic morphosyntactic and profiling analyses.

RESULTS

For work with recordings from adults with language disorders, six major results were obtained: (a) The word error rate was between 2.4% for controls and 3.4% for patients, (b) utterance tokenization accuracy was at the level reported for speakers without language disorders, (c) word-level diarization accuracy was at 93% for control participants and 83% for participants with language disorders, (d) utterance-level diarization accuracy based on word-level diarization was high, (e) adherence to CHAT format was fully accurate, and (f) human transcriber time was reduced by up to 75%.

CONCLUSION

The pipeline dramatically shortens the time gap between data collection and data analysis and provides an output superior to that typically generated by human transcribers.

A systematic review of studies on connected speech processing: Trends, key findings, and implications

Connected speech processing (CSP) is of great significance to individuals’ language and cognitive development. It is particularly crucial not only for clinical detection and treatment of developmental disorders, but also for the Foreign/second language teaching instructions. However, given the importance of this field, there is a clear lack of systematic reviews that summarize the key findings of previous studies. To this end, through searching in the scientific databases PsycInfo, Scopus, PubMed, ERIC, Taylor and Francis, and Web of Science, the present study identified 128 core CSP articles with high reference values according to PRISMA guidance and the following results were obtained through quantitative analysis and qualitative comparative synthesis: (1) The number of studies on CSP published per year showed an upward trend; however, most focused on English language, whereas the studies on other languages were comparatively rare; (2) CSP was found to be affected by multiple factors, among which speech speed, semantics, word frequency, and phonological awareness were most frequently investigated; (3) the deficit in CSP capacity was widely recognized as a significant predictor and indicator of developmental disorders; (4) more studies were carried out on connected speech production than on perception; and (5) almost no longitudinal studies have ever been conducted among either native or non-native speakers. Therefore, future research is needed to explore the developmental trajectory of CSP skills of typically developing language learners and speakers with cognitive disorders over different periods of time. It is also necessary to deepen the understanding of the processing mechanism beyond their performance and the role played by phonological awareness and lexical representations in CSP.

Anterior connectivity critical for recovery of connected speech after stroke

Connected speech recovers to different degrees across people after left hemisphere stroke, but white matter predictors of differential recovery from the acute stage of stroke are unknown. We assessed changes in lexical-syntactic aspects of connected speech in a longitudinal analysis of 40 individuals (18 females) from the acute stage of left hemisphere stroke (within an average of 4 days post-stroke) to subacute (within 2 months) and chronic stages (early: 6 months, late: 1 year) while measuring the extent of acute lesions on white matter tracts to identify tracts predictive of recovery. We found that acute damage to the frontal aslant tract led to a decreased recovery of the fluency and structural complexity of connected speech during the year following left hemisphere stroke. The results were independent of baseline performance, overall lesion volume and the proportion of damage to tract-adjacent grey matter. This longitudinal analysis from acute to chronic stroke provides the first evidence that recovery of fluent and structurally complex spontaneous connected speech requires intact left frontal connectivity via the frontal aslant tract. That the frontal aslant tract was critical for recovery at early as well as later stages of stroke demonstrates that anterior connectivity plays a lasting and important role for the reorganization of function related to the successful production of connected speech.

Neural bases of elements of syntax during speech production in patients with aphasia

The ability to string together words into a structured arrangement capable of conveying nuanced information is key to speech production. The assessment of the neural bases for structuring sentences has been challenged by the need of experts to delineate the aberrant morphosyntactic structures in aphasic speech. Most studies have relied on focused tasks with limited ecological validity. We characterized syntactic complexity during connected speech produced by patients with chronic post-stroke aphasia. We automated this process by employing Natural Language Processing (NLP). We conducted voxel-based and connectome-based lesion-symptom mapping to identify brain regions crucially associated with sentence production and syntactic complexity. Posterior-inferior aspects of left frontal and parietal lobes, as well as white matter tracts connecting these areas, were essential for syntactic complexity, particularly the posterior inferior frontal gyrus. These findings suggest that sentence structuring during word production depends on the integrity of Broca's area and the dorsal stream of language processing.