Automated computerized analysis of speech in psychiatric disorders

Semantic coherence in psychometric schizotypy: An investigation using Latent Semantic Analysis

Technological advancements have led to the development of automated methods for assessing semantic coherence in psychiatric populations. Latent Semantic Analysis (LSA) is an automated method that has been used to quantify semantic coherence in schizophrenia-spectrum disorders. The current study examined whether: 1) Semantic coherence reductions extended to psychometrically-defined schizotypy and 2) Greater cognitive load further reduces semantic coherence. LSA was applied to responses generated during category fluency tasks in baseline and cognitive load conditions. Significant differences between schizotypy and non-schizotypy groups were not observed. Findings suggest that semantic coherence may be relatively preserved at this point on the schizophrenia-spectrum.

A psychometric investigation of "macroscopic" speech measures for clinical and psychological science

The analysis of vocal expression is a critical endeavor for psychological and clinical sciences and is an increasingly popular application for computer-human interfaces. Despite this, and despite advances in the efficiency, affordability, and sophistication of vocal analytic technologies, there is considerable variability across studies regarding what aspects of vocal expression are studied. Vocal signals can be quantified in a myriad of ways, and their underlying structure, at least with respect to "macroscopic" measures from extended speech, is presently unclear. To address this issue, we evaluated the psychometric properties-notably, the structural and construct validity-of a systematically defined set of global vocal features. Our analytic strategy focused on (a) identifying redundant variables among this set, (b) employing principal components analysis (PCA) to identify nonoverlapping domains of vocal expression, (c) examining the degrees to which the vocal variables are modulated as a function of changes in speech task, and (d) evaluating the relationship between the vocal variables and cognitive (i.e., verbal fluency) and clinical (i.e., depression, anxiety, and hostility) variables. Spontaneous speech samples from 11 independent studies of young adults (>60 s in length), employing one of three different speaking tasks, were examined (N = 1,350). Confounding variables (i.e., sex, ethnicity) were statistically controlled for. The PCA identified six distinct domains of vocal expression. Collectively, vocal expression (defined in terms of these domains) was modulated as a function of speech task and was related to the cognitive and clinical variables. These findings provide empirically grounded implications for the study of vocal expression in psychological and clinical sciences.

The effect of limited cognitive resources on communication disturbances in serious mental illness

Semantically incoherent speech is a pernicious clinical feature of serious mental illness (SMI). The precise mechanisms underlying this deficit remain unclear. Prior studies have found that arousal of negative emotion exaggerates the severity of these communication disturbances; this has been coined "affective reactivity". Recent research suggests that "cognitive reactivity" may also occur, namely reflecting reduced "on-line" cognitive resources in SMI. We tested the hypothesis that communication disturbances manifest as a function of limited cognitive resources in SMI above and beyond that associated with state affectivity. We also investigated individual differences in symptoms, cognitive ability, and trait affect that may be related to cognitive reactivity. We compared individuals with SMI (n=52) to nonpsychiatric controls (n=27) on a behavioral-based coding of communication disturbances during separate baseline and experimentally-manipulated high cognitive-load dual tasks. Controlling for state affective reactivity, a significant interaction was observed such that communication disturbances decreased in the SMI group under high cognitive-load. Furthermore, a reduction in communication disturbances was related to lower trait and state positive affectivity in the SMI group. Contrary to our expectations, limited cognitive resources temporarily relieved language dysfunction. Implications, particularly with respect to interventions, are discussed.

Vocal expression in schizophrenia: Less than meets the ear

Abnormalities in nonverbal communication are a hallmark of schizophrenia. Results from studies using symptom rating scales suggest that these abnormalities are profound (i.e., 3-5 SDs) and occur across virtually every channel of vocal expression. Computerized acoustic analytic technologies, used to overcome practical and psychometric limitations with symptom rating scales, have found much more benign and isolated abnormalities. To better understand vocal deficits in schizophrenia and to advance acoustic analytic technologies for clinical and research applications, we examined archived speech samples from 5 separate studies, each using different speaking tasks (patient N = 309; control N = 117). We sought to: (a) use Principal Component Analysis (PCA) to identify independent vocal expression measures from a large set of variables, (b) quantify how patients with schizophrenia are abnormal with respect to these variables, (c) evaluate the impact of demographic and contextual factors (e.g., study site, speaking task), and (d) examine the relationship between clinically-rated psychiatric symptoms and vocal variables. PCA identified 7 independent markers of vocal expression. Most of these vocal variables varied considerably as a function of context and many were associated with demographic factors. After controlling for context and demographics, there were no meaningful differences in vocal expression between patients and controls. Within patients, vocal variables were associated with a range of psychiatric symptoms-though only pause length was significantly associated with clinically rated negative symptoms. The discussion centers on explaining the apparent discordance between clinical and computerized speech measures.

An examination of the language construct in NIMH's research domain criteria: Time for reconceptualization!

The National Institute of Mental Health's Research Domain Criteria (RDoC) Initiative "calls for the development of new ways of classifying psychopathology based on dimensions of observable behavior." As a result of this ambitious initiative, language has been identified as an independent construct in the RDoC matrix. In this article, we frame language within an evolutionary and neuropsychological context and discuss some of the limitations to the current measurements of language. Findings from genomics and the neuroimaging of performance during language tasks are discussed in relation to serious mental illness and within the context of caveats regarding measuring language. Indeed, the data collection and analysis methods employed to assay language have been both aided and constrained by the available technologies, methodologies, and conceptual definitions. Consequently, different fields of language research show inconsistent definitions of language that have become increasingly broad over time. Individually, they have also shown significant improvements in conceptual resolution, as well as in experimental and analytic techniques. More recently, language research has embraced collaborations across disciplines, notably neuroscience, cognitive science, and computational linguistics and has ultimately re-defined classical ideas of language. As we move forward, the new models of language with their remarkably multifaceted constructs force a re-examination of the NIMH RDoC conceptualization of language and thus the neuroscience and genetics underlying this concept. © 2016 The Authors. American Journal of Medical Genetics Part B: Neuropsychiatric Genetics Published by Wiley Periodicals, Inc.

Associations of acoustically measured tongue/jaw movements and portion of time speaking with negative symptom severity in patients with schizophrenia in Italy and the United States

This is the first cross-language study of the effect of schizophrenia on speech as measured by analyzing phonetic parameters with sound spectrography. We hypothesized that reduced variability in pitch and formants would be correlated with negative symptom severity in two samples of patients with schizophrenia, one from Italy, and one from the United States. Audio recordings of spontaneous speech were available from 40 patients. From each speech sample, a file of F0 (pitch) and formant values (F1 and F2, resonance bands indicating the moment-by-moment shape of the oral cavity), and the portion of the recording in which there was speaking ("fraction voiced," FV), was created. Correlations between variability in the phonetic indices and negative symptom severity were tested and further examined using regression analyses. Meaningful negative correlations between Scale for the Assessment of Negative Symptoms (SANS) total score and standard deviation (SD) of F2, as well as variability in pitch (SD F0) were observed in the Italian sample. We also found meaningful associations of SANS affective flattening and SANS alogia with SD F0, and of SANS avolition/apathy and SD F2 in the Italian sample. In both samples, FV was meaningfully correlated with SANS total score, avolition/apathy, and anhedonia/asociality.

The development of the Athens Emotional States Inventory (AESI): collection, validation and automatic processing of emotionally loaded sentences

OBJECTIVES

The development of ecologically valid procedures for collecting reliable and unbiased emotional data towards computer interfaces with social and affective intelligence targeting patients with mental disorders.

METHODS

Following its development, presented with, the Athens Emotional States Inventory (AESI) proposes the design, recording and validation of an audiovisual database for five emotional states: anger, fear, joy, sadness and neutral. The items of the AESI consist of sentences each having content indicative of the corresponding emotion. Emotional content was assessed through a survey of 40 young participants with a questionnaire following the Latin square design. The emotional sentences that were correctly identified by 85% of the participants were recorded in a soundproof room with microphones and cameras. A preliminary validation of AESI is performed through automatic emotion recognition experiments from speech.

RESULTS

The resulting database contains 696 recorded utterances in Greek language by 20 native speakers and has a total duration of approximately 28 min. Speech classification results yield accuracy up to 75.15% for automatically recognizing the emotions in AESI.

CONCLUSIONS

These results indicate the usefulness of our approach for collecting emotional data with reliable content, balanced across classes and with reduced environmental variability.

Can the Acoustic Analysis of Expressive Prosody Discriminate Schizophrenia?

Emotional states, attitudes and intentions are often conveyed by modulations in the tone of voice. Impaired recognition of emotions from a tone of voice (receptive prosody) has been described as characteristic symptoms of schizophrenia. However, the ability to express non-verbal information in speech (expressive prosody) has been understudied. This paper describes a useful technique for quantifying the degree of expressive prosody deficits in schizophrenia, using a semi-automatic method, and evaluates this method's ability to discriminate between patient and control groups. Forty-five medicated patients with a diagnosis of schizophrenia were matched with thirty-five healthy comparison subjects. Production of expressive prosodic speech was analyzed using variation in fundamental frequency (F0) measures on an emotionally neutral reading task. Results revealed that patients with schizophrenia exhibited significantly more pauses (p < .001), were slower (p < .001), and showed less pitch variability in speech (p < .05) and fewer variations in syllable timing (p < .001) than control subjects. These features have been associated with «flat» speech prosody. Signal processing algorithms applied to speech were shown to be capable of discriminating between patients and controls with an accuracy of 93.8%. These speech parameters may have a diagnostic and prognosis value and therefore could be used as a dependent measure in clinical trials.

Speech deficits in serious mental illness: a cognitive resource issue?

Speech deficits, notably those involved in psychomotor retardation, blunted affect, alogia and poverty of content of speech, are pronounced in a wide range of serious mental illnesses (e.g., schizophrenia, unipolar depression, bipolar disorders). The present project evaluated the degree to which these deficits manifest as a function of cognitive resource limitations. We examined natural speech from 52 patients meeting criteria for serious mental illnesses (i.e., severe functional deficits with a concomitant diagnosis of schizophrenia, unipolar and/or bipolar affective disorders) and 30 non-psychiatric controls using a range of objective, computer-based measures tapping speech production ("alogia"), variability ("blunted vocal affect") and content ("poverty of content of speech"). Subjects produced natural speech during a baseline condition and while engaging in an experimentally-manipulated cognitively-effortful task. For correlational analysis, cognitive ability was measured using a standardized battery. Generally speaking, speech deficits did not differ as a function of SMI diagnosis. However, every speech production and content measure was significantly abnormal in SMI versus control groups. Speech variability measures generally did not differ between groups. For both patients and controls as a group, speech during the cognitively-effortful task was sparser and less rich in content. Relative to controls, patients were abnormal under cognitive load with respect only to average pause length. Correlations between the speech variables and cognitive ability were only significant for this same variable: average pause length. Results suggest that certain speech deficits, notably involving pause length, may manifest as a function of cognitive resource limitations. Implications for treatment, research and assessment are discussed.

What do we really know about blunted vocal affect and alogia? A meta-analysis of objective assessments

Deficits in nonverbal vocal expression (e.g., blunted vocal affect, alogia) are a hallmark of schizophrenia and are a focus of the Research Domain Criteria initiative from the National Institute of Mental Health. Results from studies using symptom rating scales suggest that these deficits are profound; on the order of four to six standard deviations. To complement this endeavor, we conducted a meta-analysis of studies employing objective analysis of natural speech in patients with schizophrenia and nonpsychiatric controls. Thirteen studies, collectively including 480 patients with schizophrenia and 326 nonpsychiatric controls, were identified. There was considerable variability across studies in which aspects of vocal communication were examined and in the magnitudes of deficit. Overall, speech production (reflecting alogia) was impaired at a large effects size level (d=-.80; k=13), whereas speech variability (reflecting blunted affect) was much more modest (d=-.36; k=2). Regarding the former, this was largely driven by measures of pause behavior, as opposed to other aspects of speech (e.g., number of words/utterances). On the other hand, ratings of negative symptoms across these studies suggested profound group differences (d=3.54; k=4). These data suggest that only certain aspects of vocal expression are affected in schizophrenia, and highlight major discrepancies between symptom rating and objective-based measures. The discussion centers on advancing objective analysis for understanding vocal expression in schizophrenia and for identifying and defining more homogenous patient subsets for study.