Speech disturbances in schizophrenia: Assessing cross-linguistic generalizability of NLP automated measures of coherence

INTRODUCTION

Language disorders - disorganized and incoherent speech in particular - are distinctive features of schizophrenia. Natural language processing (NLP) offers automated measures of incoherent speech as promising markers for schizophrenia. However, the scientific and clinical impact of NLP markers depends on their generalizability across contexts, samples, and languages, which we systematically assessed in the present study relying on a large, novel, cross-linguistic corpus.

METHODS

We collected a Danish (DK), German (GE), and Chinese (CH) cross-linguistic dataset involving transcripts from 187 participants with schizophrenia (111DK, 25GE, 51CH) and 200 matched controls (129DK, 29GE, 42CH) performing the Animated Triangles Task. Fourteen previously published NLP coherence measures were calculated, and between-groups differences and association with symptoms were tested for cross-linguistic generalizability.

RESULTS

One coherence measure, i.e. second-order coherence, robustly generalized across samples and languages. We found several language-specific effects, some of which partially replicated previous findings (lower coherence in German and Chinese patients), while others did not (higher coherence in Danish patients). We found several associations between symptoms and measures of coherence, but the effects were generally inconsistent across languages and rating scales.

CONCLUSIONS

Using a cumulative approach, we have shown that NLP findings of reduced semantic coherence in schizophrenia have limited generalizability across different languages, samples, and measures. We argue that several factors such as sociodemographic and clinical heterogeneity, cross-linguistic variation, and the different NLP measures reflecting different clinical aspects may be responsible for this variability. Future studies should take this variability into account in order to develop effective clinical applications targeting different patient populations.

Semantic and phonetic similarity of verbal fluency responses in early-stage psychosis

Linguistic abnormalities can emerge early in the course of psychotic illness. Computational tools that quantify similarity of responses in standardized language-based tasks such as the verbal fluency test could efficiently characterize the nature and functional correlates of these disturbances. Participants with early-stage psychosis (n=20) and demographically matched controls without a psychiatric diagnosis (n=20) performed category and letter verbal fluency. Semantic similarity was measured via predicted context co-occurrence in a large text corpus using Word2Vec. Phonetic similarity was measured via edit distance using the VFClust tool. Responses were designated as clusters (related items) or switches (transitions to less related items) using similarity-based thresholds. Results revealed that participants with early-stage psychosis compared to controls had lower fluency scores, lower cluster-related semantic similarity, and fewer switches; mean cluster size and phonetic similarity did not differ by group. Lower fluency semantic similarity was correlated with greater speech disorganization (Communication Disturbances Index), although more strongly in controls, and correlated with poorer social functioning (Global Functioning: Social), primarily in the psychosis group. Findings suggest that search for semantically related words may be impaired soon after psychosis onset. Future work is warranted to investigate the impact of language disturbances on social functioning over the course of psychotic illness.

Sleep reduces the semantic coherence of memory recall: An application of latent semantic analysis to investigate memory reconstruction

Sleep is thought to help consolidate hippocampus-dependent memories by reactivating previously encoded neural representations, promoting both quantitative and qualitative changes in memory representations. However, the qualitative nature of changes to memory representations induced by sleep remains largely uncharacterized. In this study, we investigated how memories are reconstructed by hypothesizing that semantic coherence, defined as conceptual relatedness between statements of free-recall texts and quantified using latent semantic analysis (LSA), is affected by post-encoding sleep. Short naturalistic videos of events featuring six animals were presented to 115 participants who were randomly assigned to either 12- or 24-h delay groups featuring sleep or wakefulness. Participants' free-recall responses were analyzed to test for an effect of sleep on semantic coherence between adjacent statements, and overall. The presence of sleep reduced both forms of semantic coherence, compared to wakefulness. This change was robust and not due to shifts in conciseness or repetitiveness with sleep. These findings support the notion that sleep-dependent consolidation qualitatively changes the features of reconstructed memory representations by reducing semantic coherence.

Language as a biomarker for psychosis: A natural language processing approach

Human ratings of conceptual disorganization, poverty of content, referential cohesion and illogical thinking have been shown to predict psychosis onset in prospective clinical high risk (CHR) cohort studies. The potential value of linguistic biomarkers has been significantly magnified, however, by recent advances in natural language processing (NLP) and machine learning (ML). Such methodologies allow for the rapid and objective measurement of language features, many of which are not easily recognized by human raters. Here we review the key findings on language production disturbance in psychosis. We also describe recent advances in the computational methods used to analyze language data, including methods for the automatic measurement of discourse coherence, syntactic complexity, poverty of content, referential coherence, and metaphorical language. Linguistic biomarkers of psychosis risk are now undergoing cross-validation, with attention to harmonization of methods. Future directions in extended CHR networks include studies of sources of variance, and combination with other promising biomarkers of psychosis risk, such as cognitive and sensory processing impairments likely to be related to language. Implications for the broader study of social communication, including reciprocal prosody, face expression and gesture, are discussed.

Semantic coherence in psychometric schizotypy: An investigation using Latent Semantic Analysis

Technological advancements have led to the development of automated methods for assessing semantic coherence in psychiatric populations. Latent Semantic Analysis (LSA) is an automated method that has been used to quantify semantic coherence in schizophrenia-spectrum disorders. The current study examined whether: 1) Semantic coherence reductions extended to psychometrically-defined schizotypy and 2) Greater cognitive load further reduces semantic coherence. LSA was applied to responses generated during category fluency tasks in baseline and cognitive load conditions. Significant differences between schizotypy and non-schizotypy groups were not observed. Findings suggest that semantic coherence may be relatively preserved at this point on the schizophrenia-spectrum.

Semantic Coherence Facilitates Distributional Learning

Computational models have shown that purely statistical knowledge about words' linguistic contexts is sufficient to learn many properties of words, including syntactic and semantic category. For example, models can infer that "postman" and "mailman" are semantically similar because they have quantitatively similar patterns of association with other words (e.g., they both tend to occur with words like "deliver," "truck," "package"). In contrast to these computational results, artificial language learning experiments suggest that distributional statistics alone do not facilitate learning of linguistic categories. However, experiments in this paradigm expose participants to entirely novel words, whereas real language learners encounter input that contains some known words that are semantically organized. In three experiments, we show that (a) the presence of familiar semantic reference points facilitates distributional learning and (b) this effect crucially depends both on the presence of known words and the adherence of these known words to some semantic organization.