Disordered speech disrupts conversational entrainment: a study of acoustic-prosodic entrainment and communicative success in populations with communication challenges

The Effects of Interactive Context on Acoustic Characteristics of Speech in People With Dysarthria: A Preliminary Study

PURPOSE

The current study compared temporal and spectral acoustic contrast between vowel segments produced by speakers with dysarthria across three speech tasks-interactive, solo habitual, and solo clear.

METHOD

Nine speakers with dysarthria secondary to amyotrophic lateral sclerosis participated in the study. Each speaker was paired with a typical interlocutor over videoconferencing software. The speakers produced the vowels /i, ɪ, ɛ, æ/ in /h/-vowel-/d/ words. For the solo tasks, speakers read the stimuli aloud in both their habitual and clear speaking styles. For the interactive task, speakers produced a target stimulus for their interlocutor to select among the four possibilities. We measured the duration difference between long and short vowels, as well as the F1/F2 Euclidean distance between adjacent vowels, and also determined how well the vowels could be classified based on their acoustic characteristics.

RESULTS

Temporal contrast between long and short vowels was higher in the interactive task than in both solo tasks. Spectral distance between adjacent vowel pairs was also higher for some pairs in the interactive task than the habitual speech task. Finally, vowel classification accuracy was highest in the interactive task.

CONCLUSIONS

Overall, we found evidence that individuals with dysarthria produced vowels with greater acoustic contrast in structured interactions than they did in solo tasks. Furthermore, the speech adjustments they made to the vowel segments differed from those observed in solo speech.

Measuring Prosodic Entrainment in Conversation: A Review and Comparison of Different Methods

PURPOSE

This study aims to further our understanding of prosodic entrainment and its different subtypes by analyzing a single corpus of conversations with 12 different methods and comparing the subsequent results.

METHOD

Entrainment on three fundamental frequency features was analyzed in a subset of recordings from the LUCID corpus (Baker & Hazan, 2011) using the following methods: global proximity, global convergence, local proximity, local convergence, local synchrony (Levitan & Hirschberg, 2011), prediction using linear mixed-effects models (Schweitzer & Lewandowski, 2013), geometric approach (Lehnert-LeHouillier, Terrazas, & Sandoval, 2020), time-aligned moving average (Kousidis et al., 2008), HYBRID method (De Looze et al., 2014), cross-recurrence quantification analysis (e.g., Fusaroli & Tylén, 2016), and windowed, lagged cross-correlation (Boker et al., 2002). We employed entrainment measures on a local timescale (i.e., on adjacent utterances), a global timescale (i.e., over larger time frames), and a time series-based timescale that is larger than adjacent utterances but smaller than entire conversations.

RESULTS

We observed variance in results of different methods.

CONCLUSIONS

Results suggest that each method may measure a slightly different type of entrainment. The complex implications this has for existing and future research are discussed.

Speech Entrainment in Adolescent Conversations: A Developmental Perspective

PURPOSE

Defined as the similarity of speech behaviors between interlocutors, speech entrainment plays an important role in successful adult conversations. According to theoretical models of entrainment and research on motoric, cognitive, and social developmental milestones, the ability to entrain should develop throughout adolescence. However, little is known about the specific developmental trajectory or the role of speech entrainment in conversational outcomes of this age group. The purpose of this study is to characterize speech entrainment patterns in the conversations of neurotypical early adolescents.

METHOD

This study utilized a corpus of 96 task-based conversations between adolescents between the ages of 9 and 14 years and a comparison corpus of 32 task-based conversations between adults. For each conversational turn, two speech entrainment scores were calculated for 429 acoustic features across rhythmic, articulatory, and phonatory dimensions. Predictive modeling was used to evaluate the degree of entrainment and relationship between entrainment and two metrics of conversational success.

RESULTS

Speech entrainment increased throughout early adolescence but did not reach the level exhibited in conversations between adults. Additionally, speech entrainment was predictive of both conversational quality and conversational efficiency. Furthermore, models that included all acoustic features and both entrainment types performed better than models that only included individual acoustic feature sets or one type of entrainment.

CONCLUSIONS

Our findings show that speech entrainment skills are largely developed during early adolescence with continued development possibly occurring across later adolescence. Additionally, results highlight the role of speech entrainment in successful conversation in this population, suggesting the import of continued exploration of this phenomenon in both neurotypical and neurodivergent adolescents. We also provide evidence of the value of using holistic measures that capture the multidimensionality of speech entrainment and provide a validated methodology for investigating entrainment across multiple acoustic features and entrainment types.

Classifying Conversational Entrainment of Speech Behavior: An Expanded Framework and Review

Conversational entrainment, also known as alignment, accommodation, convergence, and coordination, is broadly defined as similarity of communicative behavior between interlocutors. Within current literature, specific terminology, definitions, and measurement approaches are wide-ranging and highly variable. As new ways of measuring and quantifying entrainment are developed and research in this area continues to expand, consistent terminology and a means of organizing entrainment research is critical, affording cohesion and assimilation of knowledge. While systems for categorizing entrainment do exist, these efforts are not entirely comprehensive in that specific measurement approaches often used within entrainment literature cannot be categorized under existing frameworks. The purpose of this review article is twofold: First, we propose an expanded version of an earlier framework which allows for the categorization of all measures of entrainment of speech behaviors and includes refinements, additions, and explanations aimed at improving its clarity and accessibility. Second, we present an extensive literature review, demonstrating how current literature fits into the given framework. We conclude with a discussion of how the proposed entrainment framework presented herein can be used to unify efforts in entrainment research.

Rhythm Perception, Speaking Rate Entrainment, and Conversational Quality: A Mediated Model

PURPOSE

Acoustic-prosodic entrainment, defined as the tendency for individuals to modify their speech behaviors to more closely align with the behaviors of their conversation partner, plays an important role in successful interaction. From a mechanistic perspective, acoustic-prosodic entrainment is, by its very nature, a rhythmic activity. Accordingly, it is highly plausible that an individual's rhythm perception abilities play a role in their ability to successfully entrain. Here, we examine the impact of rhythm perception in speaking rate entrainment and subsequent conversational quality.

METHOD

A round-robin paradigm was used to collect 90 dialogues from neurotypical adults. Additional assessments determined participants' rhythm perception abilities, social competence, and partner familiarity (i.e., whether the conversation partners knew each other prior to the interaction. Mediation analysis was used to examine the relationships between rhythm perception scores, speaking rate entrainment (using a measure of static local synchrony), and a measure of conversational success (i.e., conversational quality) based on third-party listener observations. Findings were compared to the same analysis with three additional predictive factors: participant gender, partner familiarity, and social competence.

RESULTS

Results revealed a relationship between rhythm perception and speaking rate entrainment. In unfamiliar conversation partners, there was a relationship between speaking rate entrainment and conversational quality. The relationships between entrainment and each of the three additional factors (i.e., gender, partner familiarity, and social competence) were nonsignificant.

CONCLUSIONS

In unfamiliar conversation partners, better rhythm perception abilities were indicative of increased conversational quality mediated by higher levels of speaking rate entrainment. These results support theoretical postulations specifying rhythm perception abilities as a component of acoustic-prosodic entrainment, which, in turn, facilitates conversational success. Knowledge of this relationship contributes to the development of a causal framework for considering a mechanism by which rhythm perception deficits in clinical populations may impact conversational success.

It's alignment all the way down, but not all the way up: Speakers align on some features but not others within a dialogue

During conversation, speakers modulate characteristics of their production to match their interlocutors' characteristics. This behavior is known as alignment. Speakers align at many linguistic levels, including the syntactic, lexical, and phonetic levels. As a result, alignment is often treated as a unitary phenomenon, in which evidence of alignment on one feature is cast as alignment of the entire linguistic level. This experiment investigates whether alignment can occur at some levels but not others, and on some features but not others, within a given dialogue. Participants interacted with two experimenters with highly contrasting acoustic-phonetic and syntactic profiles. The experimenters each described sets of pictures using a consistent acoustic-phonetic and syntactic profile; the participants then described new pictures to each experimenter individually. Alignment was measured as the degree to which subjects matched their current listener's speech (vs. their non-listener's) on each of several individual acoustic-phonetic and syntactic features. Additionally, a holistic measure of phonetic alignment was assessed using 323 acoustic-phonetic features analyzed jointly in a machine learning classifier. Although participants did not align on several individual spectral-phonetic or syntactic features, they did align on individual temporal-phonetic features and as measured by the holistic acoustic-phonetic profile. Thus, alignment can simultaneously occur at some levels but not others within a given dialogue, and is not a single phenomenon but rather a constellation of loosely-related effects. These findings suggest that the mechanism underlying alignment is not a primitive, automatic priming mechanism but rather guided by communicative or social factors.

Healthy Communication Partners Modify Their Speech When Conversing With Individuals With Parkinson's Disease

Purpose For individuals with Parkinson's disease (PD), conversational interactions can be challenging. Efforts to improve the success of these interactions have largely fallen on the individual with PD. Successful communication, however, involves contributions from both the individual with PD and their communication partner. The current study examines whether healthy communication partners naturally engage in different acoustic-prosodic behavior (speech compensations) when conversing with an individual with PD and, further, whether such behavior aids communication success. Method Measures of articulatory precision, speaking rate, and pitch variability were extracted from the speech of healthy speakers engaged in goal-directed dialogue with other healthy speakers (healthy-healthy dyads) and with individuals with PD (healthy-PD dyads). Speech compensations, operationally defined as significant differences in healthy speakers' acoustic-prosodic behavior in healthy-healthy dyads versus healthy-PD dyads, were calculated for the three speech behaviors. Finally, the relationships between speech behaviors and an objective measure of communicative efficiency were examined. Results Healthy speakers engaged in speech characterized by greater articulatory precision and slower speaking rate when conversing with individuals with PD relative to conversations with other healthy individuals. However, these adaptive speech compensations were not predictive of communicative efficiency. Conclusions Evidence that healthy speakers naturally engage in speech compensations when conversing with individuals with PD is novel, yet consistent with findings from studies with other populations in which conversation can be challenging. In the case of PD, these compensatory behaviors did not support communication outcomes. While preliminary in nature, the results raise important questions regarding the speech behavior of healthy communication partners and provide directions for future work.

Prosody-Based Measures for Automatic Severity Assessment of Dysarthric Speech

One of the first cues for many neurological disorders are impairments in speech. The traditional method of diagnosing speech disorders such as dysarthria involves a perceptual evaluation from a trained speech therapist. However, this approach is known to be difficult to use for assessing speech impairments due to the subjective nature of the task. As prosodic impairments are one of the earliest cues of dysarthria, the current study presents an automatic method of assessing dysarthria in a range of severity levels using prosody-based measures. We extract prosodic measures related to pitch, speech rate, and rhythm from speakers with dysarthria and healthy controls in English and Korean datasets, despite the fact that these two languages differ in terms of prosodic characteristics. These prosody-based measures are then used as inputs to random forest, support vector machine and neural network classifiers to automatically assess different severity levels of dysarthria. Compared to baseline MFCC features, 18.13% and 11.22% relative accuracy improvement are achieved for English and Korean datasets, respectively, when including prosody-based features. Furthermore, most improvements are obtained with a better classification of mild dysarthric utterances: a recall improvement from 42.42% to 83.34% for English speakers with mild dysarthria and a recall improvement from 36.73% to 80.00% for Korean speakers with mild dysarthria.

Conversational Coordination of Articulation Responds to Context: A Clinical Test Case With Traumatic Brain Injury

Purpose Coordination of communicative behavior supports shared understanding in conversation. The current study brings together analysis of two speech coordination strategies, entrainment and compensation of articulation, in a preliminary investigation into whether strategy organization is shaped by a challenging communicative context-conversing with a person who has a communication disorder. Method As an initial clinical test case, an automated measure of articulatory precision was analyzed in a corpus of spoken dialogue, where a confederate conversed with participants with traumatic brain injury (n = 28) and participants with no brain injury (n = 48). Results Overall, the confederate engaged in significant entrainment and high compensation (hyperarticulation) in conversations with participants with traumatic brain injury relative to significant entrainment and low compensation (hypoarticulation) in conversations with participants with no brain injury. Furthermore, the confederate's articulatory precision changed over the course of the conversations. Conclusions Findings suggest that the organization of conversational coordination is sensitive to context, supporting synergistic models of spoken dialogue. While corpus limitations are acknowledged, these initial results point to differences in the way in which speech strategies are realized in challenging communicative contexts, highlighting a viable and important target for investigation with clinical populations. A framework for investigating speech coordination strategies in tandem and ideas for advancing this line of inquiry serve as key contributions of this work.

Methodology Matters: The Impact of Research Design on Conversational Entrainment Outcomes

Purpose Conversational entrainment describes the tendency for individuals to alter their communicative behaviors to more closely align with those of their conversation partner. This communication phenomenon has been widely studied, and thus, the methodologies used to examine it are diverse. Here, we summarize key differences in research design and present a test case to examine the effect of methodology on entrainment outcomes. Method Sixty neurotypical adults were randomly assigned to experimental groups formed by a 2 × 2 factorial combination of two independent variables: stimuli organization (blocked vs. random presentation) and stimuli modality (auditory-only vs. audiovisual stimuli). Individuals participated in a quasiconversational design in which the speech of a virtual interlocutor was manipulated to produce fast and slow speech rate conditions. Results There was a significant effect of stimuli organization on entrainment outcomes. Individuals in the blocked, but not the random, groups altered their speech rate to align with the speech rate of the virtual interlocutor. There were no effect of stimuli modality and no interaction between modality and organization on entrainment outcomes. Conclusion Findings highlight the importance of methodological decisions on entrainment outcomes. This underscores the need for more comprehensive research regarding entrainment methodology.

The Role of the Speaker, the Listener, and Their Joint Contributions During Communicative Interactions: A Tripartite View of Intelligibility in Individuals With Dysarthria

Purpose We present a tripartite view of intelligibility in which the contributions of both the speaker and listener, as well as their joint effort during interaction, are considered. While considerable research has examined communicative interactions in situ, there is a critical gap in current knowledge on how speech intelligibility unfolds during such interactions. Here, we argue that research examining speech intelligibility in communicative interactions may provide important groundwork for advancement in clinical interventions for individuals with dysarthria. Method First, we describe the view and argue for its consideration as a powerful way of thinking about speech intelligibility. We then briefly situate the view in the relevant literature on speech intelligibility and existing theoretical frameworks. We then identify suitable methodological paradigms for studying joint contributions to intelligibility and, lastly, discuss the clinical application and potential impact of this tripartite view. Conclusions Speech communication occurs through interaction; however, in the laboratory and clinic, emphasis is usually placed on individual speakers and listeners. We have proposed that it is critical to consider how the joint contributions of speakers and listeners affect speech intelligibility in communicative interaction. This conceptualization is well aligned with the International Classification of Functioning, Disability and Health, and the findings from such an approach will allow us to better understand how to maximize available resources to enhance speech intelligibility.

Sync Pending: Characterizing Conversational Entrainment in Dysarthria Using a Multidimensional, Clinically Informed Approach

Purpose Despite the import of conversational entrainment to successful spoken dialogue, the systematic characterization of this behavioral syncing phenomenon represents a critical gap in the field of speech pathology. The goal of this study was to acoustically characterize conversational entrainment in the context of dysarthria using a multidimensional approach previously validated in healthy populations (healthy conversations; Borrie, Barrett, Willi, & Berisha, 2019). Method A large corpus of goal-oriented conversations between participants with dysarthria and healthy participants (disordered conversations) was elicited using a "spot the difference" task. Expert clinical assessment of entrainment and a measure of conversational success (communicative efficiency) was obtained for each of the audio-recorded conversations. Conversational entrainment of acoustic features representing rhythmic, articulatory, and phonatory dimensions of speech was identified using cross-recurrence quantification analysis with clinically informed model parameters and validated with a sham condition involving conversational participants who did not converse with one another. The relationship between conversational entrainment and communicative efficiency was examined. Results Acoustic evidence of entrainment was observed in phonatory, but not rhythmic and articulatory, behavior, a finding that differs from healthy conversations in which entrainment was observed in all speech signal dimensions. This result, that disordered conversations showed less acoustic entrainment than healthy conversations, is corroborated by clinical assessment of entrainment in which the disordered conversations were rated, overall, as being less in sync than healthy conversations. Furthermore, acoustic entrainment was predictive of communicative efficiency, corroborated by a relationship between clinical assessment and the same outcome measure. Conclusions The findings confirm our hypothesis that the pathological speech production parameters of dysarthria disrupt the seemingly ubiquitous phenomenon of conversational entrainment, thus advancing entrainment deficits as an important variable in dysarthria, one that may have causative effects on the success of everyday communication. Results further reveal that while this approach provides a broad overview, methodologies for characterizing conversational entrainment in dysarthria must continue to be developed and refined, with a focus on clinical utility. Supplemental Material https://osf.io/ktg5q.

Going With the Flow: An Examination of Entrainment in Typically Developing Children

Purpose Conversational entrainment is the tendency for individuals to modify their behavior to more closely converge with the behavior of their communication partner and is an important aspect of successful interaction. Evidence of entrainment in adults is robust, yet research regarding its development in children is sparse. Here, we investigate the emergence of entrainment skills in typically developing children. Method Data were collected from a total of 50 typically developing children between the ages of 5 and 14 years. Children participated in a quasiconversational paradigm with a virtual interlocutor. Speech rate of the interlocutor was digitally manipulated to produce fast and slow speech rate conditions. Results Data from the fast and slow conditions were compared using linear mixed models. Results indicated that children, regardless of age, did not alter their speech to match the rate of the virtual interlocutor. Conclusions Findings suggest that entrainment in children may not be as robust as entrainment in adults and therefore not adequately captured with the current experimental paradigm. Modifications to the current paradigm will help identify a methodology sufficiently sensitive to capture the speech alignment phenomenon in children and provide much needed information regarding the typical stages of entrainment development.

Do Conversational Partners Entrain on Articulatory Precision?

The communication phenomenon known as conversational entrainment occurs when dialogue partners align or adapt their behavior to one another while conversing. Associated with rapport, trust, and communicative efficiency, entrainment appears to facilitate conversational success. In this work, we explore how conversational partners entrain or align on articulatory precision or the clarity with which speakers articulate their spoken productions. Articulatory precision also has implications for conversational success as precise articulation can enhance speech understanding and intelligibility. However, in conversational speech, speakers tend to reduce their articulatory precision, preferring low-cost, imprecise speech. Speakers may adapt their articulation and become more precise depending on feedback from their listeners. Given the potential of entrainment, we are interested in how conversational partners adapt or entrain their articulatory precision to one another. We explore this phenomenon in 57 task-based dialogues. Controlling for the influence of speaking rate, we find that speakers entrain on articulatory precision, with significant alignment on articulation of consonants. We discuss the potential applications that speaker alignment on precision might have for modeling conversation and implementing strategies for enhancing communicative success in human-human and human-computer interactions.

Syncing Up for a Good Conversation: A Clinically Meaningful Methodology for Capturing Conversational Entrainment in the Speech Domain

Purpose Conversational entrainment, the phenomenon whereby communication partners synchronize their behavior, is considered essential for productive and fulfilling conversation. Lack of entrainment could, therefore, negatively impact conversational success. Although studied in many disciplines, entrainment has received limited attention in the field of speech-language pathology, where its implications may have direct clinical relevance. Method A novel computational methodology, informed by expert clinical assessment of conversation, was developed to investigate conversational entrainment across multiple speech dimensions in a corpus of experimentally elicited conversations involving healthy participants. The predictive relationship between the methodology output and an objective measure of conversational success, communicative efficiency, was then examined. Results Using a real versus sham validation procedure, we find evidence of sustained entrainment in rhythmic, articulatory, and phonatory dimensions of speech. We further validate the methodology, showing that models built on speech signal entrainment measures consistently outperform models built on nonentrained speech signal measures in predicting communicative efficiency of the conversations. Conclusions A multidimensional, clinically meaningful methodology for capturing conversational entrainment, validated in healthy populations, has implications for disciplines such as speech-language pathology where conversational entrainment represents a critical knowledge gap in the field, as well as a potential target for remediation.

Speech Rate Entrainment in Children and Adults With and Without Autism Spectrum Disorder

PURPOSE

Conversational entrainment, a phenomenon whereby people modify their behaviors to match their communication partner, has been evidenced as critical to successful conversation. It is plausible that deficits in entrainment contribute to the conversational breakdowns and social difficulties exhibited by people with autism spectrum disorder (ASD). This study examined speech rate entrainment in children and adult populations with and without ASD.

METHOD

Sixty participants including typically developing children, children with ASD, typically developed adults, and adults with ASD participated in a quasi-conversational paradigm with a pseudoconfederate. The confederate's speech rate was digitally manipulated to create slow and fast speech rate conditions.

RESULTS

Typically developed adults entrained their speech rate in the quasi-conversational paradigm, using a faster rate during the fast speech rate conditions and a slower rate during the slow speech rate conditions. This entrainment pattern was not evident in adults with ASD or in children populations.

CONCLUSION

Findings suggest that speech rate entrainment is a developmentally acquired skill and offers preliminary evidence of speech rate entrainment deficits in adults with ASD. Impairments in this area may contribute to the conversational breakdowns and social difficulties experienced by this population. Future work is needed to advance this area of inquiry.