Acoustic Measures of Prosody in Right-Hemisphere Damage: A Systematic Review and Meta-Analysis

How does the human brain process noisy speech in real life? Insights from the second-person neuroscience perspective

Comprehending speech with the existence of background noise is of great importance for human life. In the past decades, a large number of psychological, cognitive and neuroscientific research has explored the neurocognitive mechanisms of speech-in-noise comprehension. However, as limited by the low ecological validity of the speech stimuli and the experimental paradigm, as well as the inadequate attention on the high-order linguistic and extralinguistic processes, there remains much unknown about how the brain processes noisy speech in real-life scenarios. A recently emerging approach, i.e., the second-person neuroscience approach, provides a novel conceptual framework. It measures both of the speaker's and the listener's neural activities, and estimates the speaker-listener neural coupling with regarding of the speaker's production-related neural activity as a standardized reference. The second-person approach not only promotes the use of naturalistic speech but also allows for free communication between speaker and listener as in a close-to-life context. In this review, we first briefly review the previous discoveries about how the brain processes speech in noise; then, we introduce the principles and advantages of the second-person neuroscience approach and discuss its implications to unravel the linguistic and extralinguistic processes during speech-in-noise comprehension; finally, we conclude by proposing some critical issues and calls for more research interests in the second-person approach, which would further extend the present knowledge about how people comprehend speech in noise.

Identifying Medications Underlying Communication Atypicalities in Psychotic and Affective Disorders: A Pharmacovigilance Study Within the FDA Adverse Event Reporting System

PURPOSE

Communication atypicalities are considered promising markers of a broad range of clinical conditions. However, little is known about the mechanisms and confounders underlying them. Medications might have a crucial, relatively unknown role both as potential confounders and offering an insight on the mechanisms at work. The integration of regulatory documents with disproportionality analyses provides a more comprehensive picture to account for in future investigations of communication-related markers. The aim of this study was to identify a list of drugs potentially associated with communicative atypicalities within psychotic and affective disorders.

METHOD

We developed a query using the Medical Dictionary for Regulatory Activities to search for communicative atypicalities within the FDA Adverse Event Reporting System (updated June 2021). A Bonferroni-corrected disproportionality analysis (reporting odds ratio) was separately performed on spontaneous reports involving psychotic, affective, and non-neuropsychiatric disorders, to account for the confounding role of different underlying conditions. Drug-adverse event associations not already reported in the Side Effect Resource database of labeled adverse drug reactions (unexpected) were subjected to further robustness analyses to account for expected biases.

RESULTS

A list of 291 expected and 91 unexpected potential confounding medications was identified, including drugs that may irritate (inhalants) or desiccate (anticholinergics) the larynx, impair speech motor control (antipsychotics), or induce nodules (acitretin) or necrosis (vascular endothelial growth factor receptor inhibitors) on vocal cords; sedatives and stimulants; neurotoxic agents (anti-infectives); and agents acting on neurotransmitter pathways (dopamine agonists).

CONCLUSIONS

We provide a list of medications to account for in future studies of communication-related markers in affective and psychotic disorders. The current test case illustrates rigorous procedures for digital phenotyping, and the methodological tools implemented for large-scale disproportionality analyses can be considered a road map for investigations of communication-related markers in other clinical populations.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23721345.

The right hemisphere's capacity for language: evidence from primary progressive aphasia

The role of the right hemisphere (RH) in core language processes is still a matter of intense debate. Most of the relevant evidence has come from studies of gray matter, with relatively little research on RH white matter (WM) connectivity. Using Diffusion Tensor Imaging-based tractography, the current work examined the role of the two hemispheres in language processing in 33 individuals with Primary Progressive Aphasia (PPA), aiming to better characterize the contribution of the RH to language processing in the context of left hemisphere (LH) damage. The findings confirm the impact of PPA on the integrity of the WM language tracts in the LH. Additionally, an examination of the relationship between tract integrity and language behaviors provides robust evidence of the involvement of the WM language tracts of both hemispheres in language processing in PPA. Importantly, this study provides novel evidence of a unique contribution of the RH to language processing (i.e. a contribution independent from that of the language-dominant LH). Finally, we provide evidence that the RH contribution is specific to language processing rather than being domain general. These findings allow us to better characterize the role of RH in language processing, particularly in the context of LH damage.

Different in different ways: A network-analysis approach to voice and prosody in Autism Spectrum Disorder

The current study investigated whether the difficulty in finding group differences in prosody between speakers with autism spectrum disorder (ASD) and neurotypical (NT) speakers might be explained by identifying different acoustic profiles of speakers which, while still perceived as atypical, might be characterized by different acoustic qualities. We modelled the speech from a selection of speakers (N = 26), with and without ASD, as a network of nodes defined by acoustic features. We used a community-detection algorithm to identify clusters of speakers who were acoustically similar and compared these clusters with atypicality ratings by naïve and expert human raters. Results identified three clusters: one primarily composed of speakers with ASD, one of mostly NT speakers, and one comprised of an even mixture of ASD and NT speakers. The human raters were highly reliable at distinguishing speakers with and without ASD, regardless of which cluster the speaker was in. These results suggest that community-detection methods using a network approach may complement commonly-employed human ratings to improve our understanding of the intonation profiles in ASD.

A systematic review on production and comprehension of linguistic prosody in people with acquired language and communication disorders resulting from unilateral brain lesions

BACKGROUND

Prosody serves central functions in language processing including linguistic functions (linguistic prosody), like structuring the speech signal. Impairments in production and comprehension of linguistic prosody have been described for persons with unilateral right (RHDP) or left hemisphere damage (LHDP). However, reported results differ with respect to the characteristics and severities of these impairments AIMS: We conducted a systematic literature review focusing on production and comprehension of linguistic prosody at the prosody-syntax interface (i.e., phrase or sentence level) in LHDP and RHDP.

METHODS & PROCEDURES

In a systematic literature search we included: (i) empirical studies with (ii) adult RHDP and/or LHDP (iii) investigating production and/or comprehension of linguistic prosody at the (iv) phrase or sentence level (v) reporting quantitative data on prosodic measures. We excluded overview papers; studies involving participants with dysarthria, apraxia of speech, foreign accent syndrome, psychiatric diseases, and/or neurodegenerative diseases; studies focusing primarily on emotional prosody; and on lexical stress / word level; studies of which no full text was available and/or that were published in a language other than English. We searched the databases BIOSIS, MEDLINE, EMBASE, PubMed, Web of Science, CINAHL, Cochrane Library, PSYNDEX, PsycINFO and speechBITE, last searched on January 13^th 2022.We found 2,631 studies without duplicates. We identified 43 studies which were included into our systematic review. For data extraction and synthesis of results, we grouped studies by (i) modality (production vs. comprehension), (ii) function (syntactic structure vs. information structure), and (iii) by experiment task. For production studies, outcome measures were defined as the productive use of the different prosodic cues (lengthening, pause, f0, amplitude). For comprehension studies, performance measures (accuracy and reaction times) were defined as outcome measures. In accordance with the PRISMA 2020 statement (Page et al., 2021), we conducted a quality check to assess study risk of bias. Our review was pre-registered with PROSPERO (CRD42019120308).

OUTCOMES & RESULTS

Of the 43 studies reviewed, 30 studies involved RHDP (n = 309), assessing production in 15 studies and focusing on comprehension of prosody in 16 studies (one study investigated production and comprehension). LHDP (n = 438) were included in 35 studies of which 15 studied production and 21 evaluated comprehension of prosody (one study investigated production and comprehension). Despite the heterogeneity of results in the studies reviewed, our synthesis of results suggests that both LHDP and RHDP show limitations, but no complete impairment, in their production and/or comprehension of linguistic prosody. Prosodic limitations are evident in different areas of processing linguistic prosody, like syntactic disambiguation or the distinction between sentence types. There is a tendency towards more severe limitations in LHDP as compared to RHDP.

CONCLUSIONS

We only included published studies into our review and did not perform an assessment of risk of reporting bias as well as systematic certainty assessments of the outcomes. Despite these limitations, we conclude that both groups show deficits in production and comprehension of linguistic prosody, but neither LHDP nor RHDP are completely impaired in their prosodic processing. This suggests that prosody is a relevant communicative resource for LHDP and RHDP worth being addressed in speech-language-therapy.

Speaker-listener neural coupling reveals a right-lateralized mechanism for non-native speech-in-noise comprehension

While the increasingly globalized world has brought more and more demands for non-native language communication, the prevalence of background noise in everyday life poses a great challenge to non-native speech comprehension. The present study employed an interbrain approach based on functional near-infrared spectroscopy (fNIRS) to explore how people adapt to comprehend non-native speech information in noise. A group of Korean participants who acquired Chinese as their non-native language was invited to listen to Chinese narratives at 4 noise levels (no noise, 2 dB, -6 dB, and - 9 dB). These narratives were real-life stories spoken by native Chinese speakers. Processing of the non-native speech was associated with significant fNIRS-based listener-speaker neural couplings mainly over the right hemisphere at both the listener's and the speaker's sides. More importantly, the neural couplings from the listener's right superior temporal gyrus, the right middle temporal gyrus, as well as the right postcentral gyrus were found to be positively correlated with their individual comprehension performance at the strongest noise level (-9 dB). These results provide interbrain evidence in support of the right-lateralized mechanism for non-native speech processing and suggest that both an auditory-based and a sensorimotor-based mechanism contributed to the non-native speech-in-noise comprehension.

A generalizable speech emotion recognition model reveals depression and remission

OBJECTIVE

Affective disorders are associated with atypical voice patterns; however, automated voice analyses suffer from small sample sizes and untested generalizability on external data. We investigated a generalizable approach to aid clinical evaluation of depression and remission from voice using transfer learning: We train machine learning models on easily accessible non-clinical datasets and test them on novel clinical data in a different language.

METHODS

A Mixture of Experts machine learning model was trained to infer happy/sad emotional state using three publicly available emotional speech corpora in German and US English. We examined the model's predictive ability to classify the presence of depression on Danish speaking healthy controls (N = 42), patients with first-episode major depressive disorder (MDD) (N = 40), and the subset of the same patients who entered remission (N = 25) based on recorded clinical interviews. The model was evaluated on raw, de-noised, and speaker-diarized data.

RESULTS

The model showed separation between healthy controls and depressed patients at the first visit, obtaining an AUC of 0.71. Further, speech from patients in remission was indistinguishable from that of the control group. Model predictions were stable throughout the interview, suggesting that 20-30 s of speech might be enough to accurately screen a patient. Background noise (but not speaker diarization) heavily impacted predictions.

CONCLUSION

A generalizable speech emotion recognition model can effectively reveal changes in speaker depressive states before and after remission in patients with MDD. Data collection settings and data cleaning are crucial when considering automated voice analysis for clinical purposes.

Toward a cumulative science of vocal markers of autism: A cross-linguistic meta-analysis-based investigation of acoustic markers in American and Danish autistic children

Acoustic atypicalities in speech production are argued to be potential markers of clinical features in autism spectrum disorder (ASD). A recent meta-analysis highlighted shortcomings in the field, in particular small sample sizes and study heterogeneity. We showcase a cumulative (i.e., explicitly building on previous studies both conceptually and statistically) yet self-correcting (i.e., critically assessing the impact of cumulative statistical techniques) approach to prosody in ASD to overcome these issues. We relied on the recommendations contained in the meta-analysis to build and analyze a cross-linguistic corpus of multiple speech productions in 77 autistic and 72 neurotypical children and adolescents (>1000 recordings in Danish and US English). We used meta-analytically informed and skeptical priors, with informed priors leading to more generalizable inference. We replicated findings of a minimal cross-linguistically reliable distinctive acoustic profile for ASD (higher pitch and longer pauses) with moderate effect sizes. We identified novel reliable differences between the two groups for normalized amplitude quotient, maxima dispersion quotient, and creakiness. However, the differences were small, and there is likely no one acoustic profile characterizing all autistic individuals. We identified reliable relations of acoustic features with individual differences (age, gender), and clinical features (speech rate and ADOS sub-scores). Besides cumulatively building our understanding of acoustic atypicalities in ASD, the study shows how to use systematic reviews and meta-analyses to guide the design and analysis of follow-up studies. We indicate future directions: larger and more diverse cross-linguistic datasets, focus on heterogeneity, self-critical cumulative approaches, and open science. LAY SUMMARY: Autistic individuals are reported to speak in distinctive ways. Distinctive vocal production can affect social interactions and social development and could represent a noninvasive way to support the assessment of autism spectrum disorder (ASD). We systematically checked whether acoustic atypicalities highlighted in previous articles could be actually found across multiple recordings and two languages. We find a minimal acoustic profile of ASD: higher pitch, longer pauses, increased hoarseness and creakiness of the voice. However, there is much individual variability (by age, sex, language, and clinical characteristics). This suggests that the search for one common "autistic voice" might be naive and more fine-grained approaches are needed.

A Systematic Review of Expressive and Receptive Prosody in People With Dementia

Purpose This review was designed to provide a systematic overview of prosody in people with a primary diagnosis of dementia (PwD) and evaluate the potential use of prosodic features for diagnosis of dementia. Method A systematic search of five databases was conducted using Medical Subject Headings and keywords. Studies included in the review were evaluated for their methodological quality using the modified Joanna Briggs Institute checklist. Results A total of 14 articles were identified as being relevant for this review. Among the 14 articles, the methodological quality ranged, with eight rated as weak, four rated as moderate, and two rated as strong. Ten of the 14 articles had people with Alzheimer's disease (AD) as participants, and the remaining four had people with frontotemporal dementia as participants. Four articles focused on receptive prosody, another six focused on expressive prosody, and the remaining four articles were investigations into both. The 14 articles presented inconsistent findings, and various tasks were used to measure prosodic features in PwD in the articles. Prosody was studied as a diagnostic tool for dementia in four of the articles, all of which were based on expressive prosody in individuals with AD. Among the four articles, three proposed the use of automatic speech analysis for diagnosis of AD. Conclusions This review demonstrates that prosody in PwD is an underinvestigated area. In particular, it was concerning that most articles were of weak methodological quality. Nevertheless, it was found that prosody may be a potential diagnostic tool for assessing dementia. More studies that replicate the existing studies and those with stronger methodology are needed to confirm that receptive and/or expressive prosody can be used for dementia diagnosis.

Emotion-Attention Interaction in the Right Hemisphere

Hemispheric asymmetries in affective and cognitive functions have been extensively studied. While both cerebral hemispheres contribute to most affective and cognitive processes, neuroscientific literature and neuropsychological evidence support an overall right hemispheric dominance for emotion, attention and arousal. Emotional stimuli, especially those with survival value such as threat, tend to be prioritized in attentional resource competition. Arousing unpleasant emotional stimuli have prioritized access, especially to right-lateralized attention networks. Interference of task performance may be observed when limited resources are exhausted by task- and emotion-related processing. Tasks that rely on right hemisphere-dependent processing, like attending to the left visual hemifield or global-level visual features, are especially vulnerable to interference due to attention capture by unpleasant emotional stimuli. The aim of this review is to present literature regarding the special role of the right hemisphere in affective and attentional brain processes and their interaction. Furthermore, clinical and technological implications of this interaction will be presented. Initially, the effects of focal right hemisphere lesion or atrophy on emotional functions will be introduced. Neurological right hemisphere syndromes including aprosodia, anosognosia and neglect, which further point to the predominance of the intact right hemisphere in emotion, attention and arousal will be presented. Then there will be a brief review of electrophysiological evidence, as well as evidence from patients with neglect that support attention capture by emotional stimuli in the right hemisphere. Subsequently, experimental work on the interaction of emotion, attention and cognition in the right hemispheres of healthy subjects will be presented. Finally, clinical implications for better understanding and assessment of alterations in emotion-attention interaction due to brain disorder or treatment, such as neuromodulation, that impact affective brain functions will be discussed. It will be suggested that measuring right hemispheric emotion-attention interactions may provide basis for novel biomarkers of brain health. Such biomarkers allow for improved diagnostics in brain damage and disorders and optimized treatments. To conclude, future technological applications will be outlined regarding brain physiology-based measures that reflect engagement of the right hemisphere in affective and attentional processes.