Speech prosody abnormalities and specific dimensional schizotypy features: are relationships limited to male participants?

Detecting schizophrenia, bipolar disorder, psychosis vulnerability and major depressive disorder from 5 minutes of online-collected speech

Background

Psychosis poses substantial social and healthcare burdens. The analysis of speech is a promising approach for the diagnosis and monitoring of psychosis, capturing symptoms like thought disorder and flattened affect. Recent advancements in Natural Language Processing (NLP) methodologies enable the automated extraction of informative speech features, which has been leveraged for early psychosis detection and assessment of symptomology. However, critical gaps persist, including the absence of standardized sample collection protocols, small sample sizes, and a lack of multi-illness classification, limiting clinical applicability. Our study aimed to (1) identify an optimal assessment approach for the online and remote collection of speech, in the context of assessing the psychosis spectrum and evaluate whether a fully automated, speech-based machine learning (ML) pipeline can discriminate among different conditions on the schizophrenia-bipolar spectrum (SSD-BD-SPE), help-seeking comparison subjects (MDD), and healthy controls (HC) at varying layers of analysis and diagnostic complexity.

Methods

We adopted online data collection methods to collect 20 minutes of speech and demographic information from individuals. Participants were categorized as "healthy" help-seekers (HC), having a schizophrenia-spectrum disorder (SSD), bipolar disorder (BD), major depressive disorder (MDD), or being on the psychosis spectrum with sub-clinical psychotic experiences (SPE). SPE status was determined based on self-reported clinical diagnosis and responses to the PHQ-8 and PQ-16 screening questionnaires, while other diagnoses were determined based on self-report from participants. Linguistic and paralinguistic features were extracted and ensemble learning algorithms (e.g., XGBoost) were used to train models. A 70%-30% train-test split and 30-fold cross-validation was used to validate the model performance.

Results

The final analysis sample included 1140 individuals and 22,650 minutes of speech. Using 5-minutes of speech, our model could discriminate between HC and those with a serious mental illness (SSD or BD) with 86% accuracy (AUC = 0.91, Recall = 0.7, Precision = 0.98). Furthermore, our model could discern among HC, SPE, BD and SSD groups with 86% accuracy (F1 macro = 0.855, Recall Macro = 0.86, Precision Macro = 0.86). Finally, in a 5-class discrimination task including individuals with MDD, our model had 76% accuracy (F1 macro = 0.757, Recall Macro = 0.758, Precision Macro = 0.766).

Conclusion

Our ML pipeline demonstrated disorder-specific learning, achieving excellent or good accuracy across several classification tasks. We demonstrated that the screening of mental disorders is possible via a fully automated, remote speech assessment pipeline. We tested our model on relatively high number conditions (5 classes) in the literature and in a stratified sample of psychosis spectrum, including HC, SPE, SSD and BD (4 classes). We tested our model on a large sample (N = 1150) and demonstrated best-in-class accuracy with remotely collected speech data in the psychosis spectrum, however, further clinical validation is needed to test the reliability of model performance.

Automated analysis of speech as a marker of sub-clinical psychotic experiences

Automated speech analysis techniques, when combined with artificial intelligence and machine learning, show potential in capturing and predicting a wide range of psychosis symptoms, garnering attention from researchers. These techniques hold promise in predicting the transition to clinical psychosis from at-risk states, as well as relapse or treatment response in individuals with clinical-level psychosis. However, challenges in scientific validation hinder the translation of these techniques into practical applications. Although sub-clinical research could aid to tackle most of these challenges, there have been only few studies conducted in speech and psychosis research in non-clinical populations. This work aims to facilitate this work by summarizing automated speech analytical concepts and the intersection of this field with psychosis research. We review psychosis continuum and sub-clinical psychotic experiences, and the benefits of researching them. Then, we discuss the connection between speech and psychotic symptoms. Thirdly, we overview current and state-of-the art approaches to the automated analysis of speech both in terms of language use (text-based analysis) and vocal features (audio-based analysis). Then, we review techniques applied in subclinical population and findings in these samples. Finally, we discuss research challenges in the field, recommend future research endeavors and outline how research in subclinical populations can tackle the listed challenges.

Prosody and schizophrenia. Objective acoustic measurements of monotonous and flat intonation in young Danish people with a schizophrenia diagnosis. A pilot study

PURPOSE

Patients with schizophrenia have a flat and monotonous intonation. The purpose of the study was to find the variables of flat speech that differed in patients from those in healthy controls in Danish.

MATERIALS AND METHODS

We compared drug-naïve schizophrenic patients 5 men, 13 women and 18 controls, aged 18-35 years, which had all grown up in Copenhagen speaking modern Danish standard (rigsdansk). We used two different tasks that lay different demands on the speaker to elicit spontaneous speech: a retelling of a film clip and telling a story from pictures in a book. A linguist used the computer program Praat to extract the phonetic linguistic parameters.

RESULTS

We found different results for the two elicitation tasks (Task 1: a retelling of a film clip, task 2: telling a story from pictures in a book). There was higher intensity variation in task one in controls and higher pitch variation in task two in controls. We found a difference in intensity with higher intensity variation in the stresses in the controls in task one and fewer syllables between each stress in the controls. We also found higher F1 variation in task one and two in the patient group and higher F2 variation in the control group in both tasks.

CONCLUSIONS

The results varied between patients and controls, but the demands also made a difference. Further research is needed to elucidate the possibilities of acoustic measures in diagnostics or linguistic treatment related to schizophrenia.

Online speech assessment of the psychotic spectrum: Exploring the relationship between overlapping acoustic markers of schizotypy, depression and anxiety

BACKGROUND

Remote assessment of acoustic alterations in speech holds promise to increase scalability and validity in research across the psychosis spectrum. A feasible first step in establishing a procedure for online assessments is to assess acoustic alterations in psychometric schizotypy. However, to date, the complex relationship between alterations in speech related to schizotypy and those related to comorbid conditions such as symptoms of depression and anxiety has not been investigated. This study tested whether (1) depression, generalized anxiety and high psychometric schizotypy have similar voice characteristics, (2) which acoustic markers of online collected speech are the strongest predictors of psychometric schizotypy, (3) whether including generalized anxiety and depression symptoms in the model can improve the prediction of schizotypy.

METHODS

We collected cross-sectional, online-recorded speech data from 441 participants, assessing demographics, symptoms of depression, generalized anxiety and psychometric schizotypy.

RESULTS

Speech samples collected online could predict psychometric schizotypy, depression, and anxiety symptoms with weak to moderate predictive power, and with moderate and good predictive power when basic demographic variables were added to the models. Most influential features of these models largely overlapped. The predictive power of speech marker-based models of schizotypy significantly improved after including symptom scores of depression and generalized anxiety in the models (from R2 = 0.296 to R2 = 0. 436).

CONCLUSIONS

Acoustic features of online collected speech are predictive of psychometric schizotypy as well as generalized anxiety and depression symptoms. The acoustic characteristics of schizotypy, depression and anxiety symptoms significantly overlap. Speech models that are designed to predict schizotypy or symptoms of the schizophrenia spectrum might therefore benefit from controlling for symptoms of depression and anxiety.

A Narrative Review of Speech and EEG Features for Schizophrenia Detection: Progress and Challenges

Schizophrenia is a mental illness that affects an estimated 21 million people worldwide. The literature establishes that electroencephalography (EEG) is a well-implemented means of studying and diagnosing mental disorders. However, it is known that speech and language provide unique and essential information about human thought. Semantic and emotional content, semantic coherence, syntactic structure, and complexity can thus be combined in a machine learning process to detect schizophrenia. Several studies show that early identification is crucial to prevent the onset of illness or mitigate possible complications. Therefore, it is necessary to identify disease-specific biomarkers for an early diagnosis support system. This work contributes to improving our knowledge about schizophrenia and the features that can identify this mental illness via speech and EEG. The emotional state is a specific characteristic of schizophrenia that can be identified with speech emotion analysis. The most used features of speech found in the literature review are fundamental frequency (F0), intensity/loudness (I), frequency formants (F1, F2, and F3), Mel-frequency cepstral coefficients (MFCC's), the duration of pauses and sentences (SD), and the duration of silence between words. Combining at least two feature categories achieved high accuracy in the schizophrenia classification. Prosodic and spectral or temporal features achieved the highest accuracy. The work with higher accuracy used the prosodic and spectral features QEVA, SDVV, and SSDL, which were derived from the F0 and spectrogram. The emotional state can be identified with most of the features previously mentioned (F0, I, F1, F2, F3, MFCCs, and SD), linear prediction cepstral coefficients (LPCC), linear spectral features (LSF), and the pause rate. Using the event-related potentials (ERP), the most promissory features found in the literature are mismatch negativity (MMN), P2, P3, P50, N1, and N2. The EEG features with higher accuracy in schizophrenia classification subjects are the nonlinear features, such as Cx, HFD, and Lya.

Schizophrenia identification for phonetic coherence using SVM and blur approaches

The diagnosis cycle of schizophrenia is long, there is no objective diagnostic basis. The over-energy entropy product of the speech fluency rectangular parameter is designed in the paper, the fuzzy clustering is used to double locate speech pause areas and to assist in the diagnosis of schizophrenia. The pause area of speech is located based on the low speech fluency and flat energy in schizophrenia patients, an extraction algorithm is given for speech fluency quantification parameters, support vector machine (SVM) classifier is used in the approach. The fluency acoustic features of speech are taken from 28 schizophrenia patients and 28 normal controls, these are used to verify the effect of the method in schizophrenia recognition, there is a correct rate of over 85% . The automatic schizophrenia identification based on energy entropy product and fuzzy clustering can provide objective, effective and non-invasive auxiliary for clinical diagnosis of schizophrenia.

High Predictive Accuracy of Negative Schizotypy With Acoustic Measures

Negative schizotypal traits potentially can be digitally phenotyped using objective vocal analysis. Prior attempts have shown mixed success in this regard, potentially because acoustic analysis has relied on small, constrained feature sets. We employed machine learning to (a) optimize and cross-validate predictive models of self-reported negative schizotypy using a large acoustic feature set, (b) evaluate model performance as a function of sex and speaking task, (c) understand potential mechanisms underlying negative schizotypal traits by evaluating the key acoustic features within these models, and (d) examine model performance in its convergence with clinical symptoms and cognitive functioning. Accuracy was good (> 80%) and was improved by considering speaking task and sex. However, the features identified as most predictive of negative schizotypal traits were generally not considered critical to their conceptual definitions. Implications for validating and implementing digital phenotyping to understand and quantify negative schizotypy are discussed.

Speech Prosody as a Bridge Between Psychopathology and Linguistics: The Case of the Schizophrenia Spectrum

Patients with schizophrenia spectrum disorders experience severe difficulties in interpersonal communication, as described by traditional psychopathology and current research on social cognition. From a linguistic perspective, pragmatic abilities are crucial for successful communication. Empirical studies have shown that these abilities are significantly impaired in this group of patients. Prosody, the tone of voice with which words and sentences are pronounced, is one of the most important carriers of pragmatic meaning and can serve a range of functions from linguistic to emotional ones. Most of the existing literature on prosody of patients with schizophrenia spectrum disorders focuses on the expression of emotion, generally showing significant impairments. By contrast, the use of non-emotional prosody in these patients is scarcely investigated. In this paper, we first present a linguistic model to classify prosodic functions. Second, we discuss existing studies on the use of non-emotional prosody in these patients, providing an overview of the state of the art. Third, we delineate possible future lines of research in this field, also taking into account some classical psychopathological assumptions, for both diagnostic and therapeutic purposes.

Protocolo de evaluación de la prosodia emocional y la pragmática en personas con esquizofrenia

La esquizofrenia se caracteriza por una distorsión del pensamiento, las percepciones, las emociones, el lenguaje, la conciencia de sí mismo y la conducta. Afecta a más de 21 millones de personas en todo el mundo y es una de las 10 enfermedades más incapacitantes según la Organización Mundial de la Salud, pero existe escasa evidencia sobre el déficit lingüístico con el que cursa la enfermedad. Estudio transversal y cuasi-experimental donde se analizan los datos resultantes de la evaluación por medio de diferentes pruebas de las áreas de pragmática y prosodia en una muestra compuesta por 96 sujetos, de los cuales 48 (50%) no tienen enfermedad mental y componen el grupo control y 48 (50%) tienen enfermedad mental y componen el grupo experimental.De acuerdo con la clasificación de Crow (1978), el 56,7% de los sujetos del grupo experimental (27 sujetos) tiene sintomatología positiva y el 43,8% (21 sujetos) tiene sintomatología negativa. Se observan diferentes patrones de déficit según la sintomatología, así como una afectación de las áreas evaluadas en el estudio (pragmática y prosodia afectiva).La esquizofrenia cursa con déficit en las áreas analizadas, si bien existe una controversia acerca de su origen. En relación con la pragmática, encontramos interpretaciones literales en ambos tipos de pacientes, aunque son más frecuentes en los sujetos con sintomatología positiva. En cuanto a la prosodia, encontramos frecuencias fundamentales excesivas, monotonía en la expresión de las emociones y dificultades para comprender emociones a través de los aspectos prosódicos.

A Transdiagnostic Review of Negative Symptom Phenomenology and Etiology

In the DSM5, negative symptoms are 1 of the 5 core dimensions of psychopathology evaluated for schizophrenia. However, negative symptoms are not pathognomonic-they are also part of the diagnostic criteria for other schizophrenia-spectrum disorders, disorders that sometimes have comorbid psychosis, diagnoses not in the schizophrenia-spectrum, and the general "nonclinical" population. Although etiological models of negative symptoms have been developed for chronic schizophrenia, there has been little attention given to whether these models have transdiagnostic applicability. In the current review, we examine areas of commonality and divergence in the clinical presentation and etiology of negative symptoms across diagnostic categories. It was concluded that negative symptoms are relatively frequent across diagnostic categories, but individual disorders may differ in whether their negative symptoms are persistent/transient or primary/secondary. Evidence for separate dimensions of volitional and expressive symptoms exists, and there may be multiple mechanistic pathways to the same symptom phenomenon among DSM-5 disorders within and outside the schizophrenia-spectrum (ie, equifinality). Evidence for a novel transdiagnostic etiological model is presented based on the Research Domain Criteria (RDoC) constructs, which proposes the existence of 2 such pathways-a hedonic pathway and a cognitive pathway-that can both lead to expressive or volitional symptoms. To facilitate treatment breakthroughs, future transdiagnostic studies on negative symptoms are warranted that explore mechanisms underlying volitional and expressive pathology.

Can RDoC Help Find Order in Thought Disorder?

Thought disorder is a pernicious and nonspecific aspect of numerous serious mental illnesses (SMIs) and related conditions. Despite decades of empirical research on thought disorder, our present understanding of it is poor, our clinical assessments focus on a limited set of extreme behaviors, and treatments are palliative at best. Applying a Research Domain Criteria (RDoC) framework to thought disorder research offers advantages to explicate its phenotype; isolate its mechanisms; and develop more effective assessments, treatments, and potential cures. In this commentary, we discuss ways in which thought disorder can be understood within the RDoC framework. We propose operationalizing thought disorder within the RDoC construct of language using psycholinguistic sciences, to help objectify and quantify language within individuals; technologically sophisticated paradigms, to allow naturalistic behavioral sampling techniques with unprecedented ecological validity; and computational modeling, to account for a network of interconnected and dynamic linguistic, cognitive, affective, and social functions. We also highlight challenges for understanding thought disorder within an RDoC framework. Thought disorder likely does not occur as an isomorphic dysfunction in a single RDoC construct, but rather, as multiple potential dysfunctions in a network of RDoC constructs. Moreover, thought disorder is dynamic over time and context within individuals. In sum, RDoC is a useful framework to integrate multidisciplinary research efforts aimed at operationalizing, understanding, and ameliorating thought disorder.

Emotion processing deficits in the different dimensions of psychometric schizotypy

Schizotypy refers to a personality structure indicating "proneness" to schizophrenia. Around 10% of the general population has increased schizotypal traits, they also share other core features with schizophrenia and are thus at heightened risk for developing schizophrenia and spectrum disorders. A key aspect in schizophrenia-spectrum pathology is the impairment observed in emotion-related processes. This review summarizes findings on impairments related to central aspects of emotional processes, such as emotional disposition, alexithymia, facial affect recognition and speech prosody, in high schizotypal individuals in the general population. Although the studies in the field are not numerous, the current findings indicate that all these aspects of emotional processing are deficient in psychometric schizotypy, in accordance to the schizophrenia-spectrum literature. A disturbed frontotemporal neural network seems to be the critical link between these impairments, schizotypy and schizophrenia. The limitations of the current studies and suggestions for future research are discussed.

Sex Differences in Affective Expression Among Individuals with Psychometrically Defined Schizotypy: Diagnostic Implications

The present investigation uses facial electromyography (fEMG) to measure patterns of affective expression in individuals with psychometrically defined schizotypy during presentation of neutral and negative visual images. Twenty-eight individuals with elevated schizotypal features and 20 healthy controls observed a series of images from the International Affective Picture System (IAPS) and provided self-report ratings of affective valence and arousal while their physiological responses were recorded. The groups were evenly divided by sex. A three-way interaction in fEMG measurement revealed that while males with psychometrically defined schizotypy demonstrated the expected pattern of blunted/constricted facial affective expression relative to male controls in the context of negative images, females displayed the opposite pattern. That is, females with psychometrically defined schizotypy demonstrated significant elevations in negative facial affective expression relative to female controls while viewing negative images. We argue that these findings corroborate previously reported impressions of sex differences in affective expression in schizotypy. We discuss implications for assessment and diagnostic procedures among individuals with disorders along the schizophrenia spectrum.