Spared and impaired spoken discourse processing in schizophrenia: effects of local and global language context

Theory-Driven Analysis of Natural Language Processing Measures of Thought Disorder Using Generative Language Modeling

BACKGROUND

Natural language processing (NLP) holds promise to transform psychiatric research and practice. A pertinent example is the success of NLP in the automatic detection of speech disorganization in formal thought disorder (FTD). However, we lack an understanding of precisely what common NLP metrics measure and how they relate to theoretical accounts of FTD. We propose tackling these questions by using deep generative language models to simulate FTD-like narratives by perturbing computational parameters instantiating theory-based mechanisms of FTD.

METHODS

We simulated FTD-like narratives using Generative-Pretrained-Transformer-2 by either increasing word selection stochasticity or limiting the model's memory span. We then examined the sensitivity of common NLP measures of derailment (semantic distance between consecutive words or sentences) and tangentiality (how quickly meaning drifts away from the topic) in detecting and dissociating the 2 underlying impairments.

RESULTS

Both parameters led to narratives characterized by greater semantic distance between consecutive sentences. Conversely, semantic distance between words was increased by increasing stochasticity, but decreased by limiting memory span. An NLP measure of tangentiality was uniquely predicted by limited memory span. The effects of limited memory span were nonmonotonic in that forgetting the global context resulted in sentences that were semantically closer to their local, intermediate context. Finally, different methods for encoding the meaning of sentences varied dramatically in performance.

CONCLUSIONS

This work validates a simulation-based approach as a valuable tool for hypothesis generation and mechanistic analysis of NLP markers in psychiatry. To facilitate dissemination of this approach, we accompany the paper with a hands-on Python tutorial.

From communication dysfunction to treatment options in serious mental illness

The Commentary covers research focused on language dysfunction in schizophrenia, and more broadly in communication dysfunction in this disorder, which I have examined with a variety of both behavioral and imaging methodologies. It briefly outlines how further progress can be achieved in pursuing the goal of a comprehensive understanding of its underlying causes. Possible therapeutic approaches are also briefly discussed.

Mechanisms underlying dorsolateral prefrontal cortex contributions to cognitive dysfunction in schizophrenia

Kraepelin, in his early descriptions of schizophrenia (SZ), characterized the illness as having "an orchestra without a conductor." Kraepelin further speculated that this "conductor" was situated in the frontal lobes. Findings from multiple studies over the following decades have clearly implicated pathology of the dorsolateral prefrontal cortex (DLPFC) as playing a central role in the pathophysiology of SZ, particularly with regard to key cognitive features such as deficits in working memory and cognitive control. Following an overview of the cognitive mechanisms associated with DLPFC function and how they are altered in SZ, we review evidence from an array of neuroscientific approaches addressing how these cognitive impairments may reflect the underlying pathophysiology of the illness. Specifically, we present evidence suggesting that alterations of the DLPFC in SZ are evident across a range of spatial and temporal resolutions: from its cellular and molecular architecture, to its gross structural and functional integrity, and from millisecond to longer timescales. We then present an integrative model based upon how microscale changes in neuronal signaling in the DLPFC can influence synchronized patterns of neural activity to produce macrocircuit-level alterations in DLPFC activation that ultimately influence cognition and behavior. We conclude with a discussion of initial efforts aimed at targeting DLPFC function in SZ, the clinical implications of those efforts, and potential avenues for future development.

Cognitive control mediates age-related changes in flexible anticipatory processing during listening comprehension

Effective listening comprehension not only requires processing local linguistic input, but also necessitates incorporating contextual cues available in the global communicative environment. Local sentence processing can be facilitated by pre-activation of likely upcoming input, or predictive processing. Recent evidence suggests that young adults can flexibly adapt local predictive processes based on cues provided by the global communicative environment, such as the reliability of specific speakers. Whether older comprehenders can also flexibly adapt to global contextual cues is currently unknown. Moreover, it is unclear whether the underlying mechanisms supporting local predictive processing differ from those supporting adaptation to global contextual cues. Critically, it is unclear whether these mechanisms change as a function of typical aging. We examined the flexibility of prediction in young and older adults by presenting sentences from speakers whose utterances were typically more or less predictable (i.e., reliable speakers who produced expected words 80% of the time, versus unreliable speakers who produced expected words 20% of the time). For young listeners, global speaker reliability cues modulated neural effects of local predictability on the N400. In contrast, older adults, on average, did not show global modulation of local processing. Importantly, however, cognitive control (i.e., Stroop interference effects) mediated age-related reductions in sensitivity to the reliability of the speaker. Both young and older adults with high cognitive control showed greater N400 effects of predictability during sentences produced by a reliable speaker, suggesting that cognitive control is required to regulate the strength of top-down predictions based on global contextual information. Critically, cognitive control predicted sensitivity to global speaker-specific information but not local predictability cues, suggesting that predictive processing in local sentence contexts may be supported by separable neural mechanisms from adaptation of prediction as a function of global context. These results have important implications for interpreting age-related change in predictive processing, and for drawing more generalized conclusions regarding domain-general versus language-specific accounts of prediction.

A leopard cannot change its spots: A novel pragmatic account of concretism in schizophrenia

Concretism is a well-known feature of schizophrenia, tracing back to the early descriptions of the disease and commonly associated with the literal interpretation of figurative expressions such as proverbs, metaphors, and idioms. However, figurative expressions are not all alike in terms of linguistic and pragmatic processes. Determining if some figurative types are more impaired than others and if the type of task affects the performance constitutes an open issue with implications for the description of the clinical profile and for treatment. We run a fine-grained assessment of figurative language comprehension by comparing 47 patients with schizophrenia and 39 controls in three figurative types (idioms, metaphors, proverbs) presented in two response formats (multiple-choice, verbal-explanation), considering also the role of cognitive and clinical variables and the impact on quality of life. Mixed-effects models analysis revealed that: i) patients performed worse than controls across figurative types and response formats, indicating a diffuse impairment; ii) there is an interplay of figurative type and response format, which makes verbal-explanation of proverbs especially challenging; iii) in patients, problems in figurative language are largely connected with formal thought disorder and global IQ. Moreover, multiple-choice of metaphors was associated with patients' quality of life. This study offers a novel account of concretism, framed in pragmatics and figurative language processing. Adopting this perspective allowed us to describe the nuances of concretism, which areas in the figurative domain are especially challenging for patients and which ones capture differences in functioning, in order to guide intervention programs and integration in society.

Going the Extra Mile: Effects of Discourse Context on Two Late Positivities During Language Comprehension

During language comprehension, online neural processing is strongly influenced by the constraints of the prior context. While the N400 ERP response (300-500ms) is known to be sensitive to a word's semantic predictability, less is known about a set of late positive-going ERP responses (600-1000ms) that can be elicited when an incoming word violates strong predictions about upcoming content (late frontal positivity) or about what is possible given the prior context (late posterior positivity/P600). Across three experiments, we systematically manipulated the length of the prior context and the source of lexical constraint to determine their influence on comprehenders' online neural responses to these two types of prediction violations. In Experiment 1, within minimal contexts, both lexical prediction violations and semantically anomalous words produced a larger N400 than expected continuations (James unlocked the door/laptop/gardener), but no late positive effects were observed. Critically, the late posterior positivity/P600 to semantic anomalies appeared when these same sentences were embedded within longer discourse contexts (Experiment 2a), and the late frontal positivity appeared to lexical prediction violations when the preceding context was rich and globally constraining (Experiment 2b). We interpret these findings within a hierarchical generative framework of language comprehension. This framework highlights the role of comprehension goals and broader linguistic context, and how these factors influence both top-down prediction and the decision to update or reanalyze the prior context when these predictions are violated.

Spared bottom-up but impaired top-down interactive effects during naturalistic language processing in schizophrenia: evidence from the visual-world paradigm

BACKGROUND

People with schizophrenia process language in unusual ways, but the causes of these abnormalities are unclear. In particular, it has proven difficult to empirically disentangle explanations based on impairments in the top-down processing of higher level information from those based on the bottom-up processing of lower level information.

METHODS

To distinguish these accounts, we used visual-world eye tracking, a paradigm that measures spoken language processing during real-world interactions. Participants listened to and then acted out syntactically ambiguous spoken instructions (e.g. 'tickle the frog with the feather', which could either specify how to tickle a frog, or which frog to tickle). We contrasted how 24 people with schizophrenia and 24 demographically matched controls used two types of lower level information (prosody and lexical representations) and two types of higher level information (pragmatic and discourse-level representations) to resolve the ambiguous meanings of these instructions. Eye tracking allowed us to assess how participants arrived at their interpretation in real time, while recordings of participants' actions measured how they ultimately interpreted the instructions.

RESULTS

We found a striking dissociation in participants' eye movements: the two groups were similarly adept at using lower level information to immediately constrain their interpretations of the instructions, but only controls showed evidence of fast top-down use of higher level information. People with schizophrenia, nonetheless, did eventually reach the same interpretations as controls.

CONCLUSIONS

These data suggest that language abnormalities in schizophrenia partially result from a failure to use higher level information in a top-down fashion, to constrain the interpretation of language as it unfolds in real time.

Multimodal neuroimaging evidence for looser lexico-semantic networks in schizophrenia:Evidence from masked indirect semantic priming

It has been hypothesized that schizophrenia is characterized by overly broad automatic activity within lexico-semantic networks. We used two complementary neuroimaging techniques, Magnetoencephalography (MEG) and functional Magnetic Resonance Imaging (fMRI), in combination with a highly automatic indirect semantic priming paradigm, to spatiotemporally localize this abnormality in the brain. Eighteen people with schizophrenia and 20 demographically-matched control participants viewed target words ("bell") preceded by directly related ("church"), indirectly related ("priest"), or unrelated ("hammer") prime words in MEG and fMRI sessions. To minimize top-down processing, the prime was masked, the target appeared only 140 ms after prime onset, and participants simply monitored for words within a particular semantic category that appeared in filler trials. Both techniques revealed a significantly larger automatic indirect priming effect in people with schizophrenia than in control participants. MEG temporally localized this enhanced effect to the N400 time window (300-500 ms) - the critical stage of accessing meaning from words. fMRI spatially localized the effect to the left temporal fusiform cortex, which plays a role in mapping of orthographic word-form on to meaning. There was no evidence of an enhanced automatic direct semantic priming effect in the schizophrenia group. These findings provide converging neural evidence for abnormally broad highly automatic lexico-semantic activity in schizophrenia. We argue that, rather than arising from an unconstrained spread of automatic activation across semantic memory, this broader automatic lexico-semantic activity stems from looser mappings between the form and meaning of words.

When Proactivity Fails: An Electrophysiological Study of Establishing Reference in Schizophrenia

BACKGROUND

Schizophrenia is characterized by abnormalities in referential communication, which may be linked to more general deficits in proactive cognitive control. We used event-related potentials to probe the timing and nature of the neural mechanisms engaged as people with schizophrenia linked pronouns to their preceding referents during word-by-word sentence comprehension.

METHODS

We measured event-related potentials to pronouns in two-clause sentences in 16 people with schizophrenia and 20 demographically matched control participants. Our design crossed the number of potential referents (1-referent, 2-referent) with whether the pronoun matched the gender of its preceding referent(s) (matching, mismatching). This gave rise to four conditions: 1) 1-referent matching ("Edward took courses in accounting but he . . ."); 2) 2-referent matching ("Edward and Phillip took courses but he . . . "); 3) 1-referent mismatching ("Edward took courses in accounting but she . . ."); and 4) 2-referent mismatching ("Edward and Phillip took courses but she . . .").

RESULTS

Consistent with previous findings, healthy control participants produced a larger left anteriorly distributed negativity between 400 and 600 ms to 2-referent matching than to 1-referent matching pronouns (the "Nref effect"). In contrast, people with schizophrenia produced a larger centroposterior positivity effect between 600 and 800 ms. Both patient and control groups produced a larger positivity between 400 and 800 ms to mismatching than to matching pronouns.

CONCLUSIONS

These findings suggest that proactive mechanisms of referential processing, reflected by the Nref effect, are impaired in schizophrenia, while reactive mechanisms, reflected by the positivity effects, are relatively spared. Indeed, patients may compensate for proactive deficits by retroactively engaging with context to influence the processing of inputs at a later stage of analysis.

Abnormal neural hierarchy in processing of verbal information in patients with schizophrenia

Previous research indicates abnormal comprehension of verbal information in patients with schizophrenia. Yet the neural mechanism underlying the breakdown of verbal information processing in schizophrenia is poorly understood. Imaging studies in healthy populations have shown a network of brain areas involved in hierarchical processing of verbal information over time. Here, we identified critical aspects of this hierarchy, examining patients with schizophrenia. Using functional magnetic resonance imaging, we examined various levels of information comprehension elicited by naturally presented verbal stimuli; from a set of randomly shuffled words to an intact story. Specifically, patients with first episode schizophrenia (N = 15), their non-manifesting siblings (N = 14) and healthy controls (N = 15) listened to a narrated story and randomly scrambled versions of it. To quantify the degree of dissimilarity between the groups, we adopted an inter-subject correlation (inter-SC) approach, which estimates differences in synchronization of neural responses within and between groups. The temporal topography found in healthy and siblings groups were consistent with our previous findings - high synchronization in responses from early sensory toward high order perceptual and cognitive areas. In patients with schizophrenia, stimuli with short and intermediate temporal scales evoked a typical pattern of reliable responses, whereas story condition (long temporal scale) revealed robust and widespread disruption of the inter-SCs. In addition, the more similar the neural activity of patients with schizophrenia was to the average response in the healthy group, the less severe the positive symptoms of the patients. Our findings suggest that system-level neural indication of abnormal verbal information processing in schizophrenia reflects disease manifestations.

Language context processing deficits in schizophrenia: The role of attentional engagement

Individuals with schizophrenia exhibit problems in language comprehension that are most evident during discourse processing. We hypothesized that deficits in cognitive control contribute to these comprehension deficits during discourse processing, and investigated the underlying cognitive-neural mechanisms using EEG (alpha power) and ERPs (N400). N400 amplitudes to globally supported or unsupported target words near the end of stories were used to index sensitivity to previous context. ERPs showed reduced sensitivity to context in patients versus controls. EEG alpha-band activity was used to index attentional engagement while participants listened to the stories. We found that context effects varied with attentional engagement in both groups, as well as with negative symptom severity in patients. Both groups demonstrated trial-to-trial fluctuations in alpha. Relatively high alpha power was associated with compromised discourse processing in participants with schizophrenia when it occurred during any early portion of the story. In contrast, discourse processing was only compromised in controls when alpha was relatively high for longer segments of the stories. Our results indicate that shifts in attention from the story context may be more detrimental to discourse processing for participants with schizophrenia than for controls, most likely due to an impaired ability to benefit from global context.

Non-Arbitrariness in Mapping Word Form to Meaning: Cross-Linguistic Formal Markers of Word Concreteness

Arbitrary symbolism is a linguistic doctrine that predicts an orthogonal relationship between word forms and their corresponding meanings. Recent corpora analyses have demonstrated violations of arbitrary symbolism with respect to concreteness, a variable characterizing the sensorimotor salience of a word. In addition to qualitative semantic differences, abstract and concrete words are also marked by distinct morphophonological structures such as length and morphological complexity. Native English speakers show sensitivity to these markers in tasks such as auditory word recognition and naming. One unanswered question is whether this violation of arbitrariness reflects an idiosyncratic property of the English lexicon or whether word concreteness is a marked phenomenon across other natural languages. We isolated concrete and abstract English nouns (N = 400), and translated each into Russian, Arabic, Dutch, Mandarin, Hindi, Korean, Hebrew, and American Sign Language. We conducted offline acoustic analyses of abstract and concrete word length discrepancies across languages. In a separate experiment, native English speakers (N = 56) with no prior knowledge of these foreign languages judged concreteness of these nouns (e.g., Can you see, hear, feel, or touch this? Yes/No). Each naïve participant heard pre-recorded words presented in randomized blocks of three foreign languages following a brief listening exposure to a narrative sample from each respective language. Concrete and abstract words differed by length across five of eight languages, and prediction accuracy exceeded chance for four of eight languages. These results suggest that word concreteness is a marked phenomenon across several of the world's most widely spoken languages. We interpret these findings as supportive of an adaptive cognitive heuristic that allows listeners to exploit non-arbitrary mappings of word form to word meaning.

A Hierarchical Generative Framework of Language Processing: Linking Language Perception, Interpretation, and Production Abnormalities in Schizophrenia

Language and thought dysfunction are central to the schizophrenia syndrome. They are evident in the major symptoms of psychosis itself, particularly as disorganized language output (positive thought disorder) and auditory verbal hallucinations (AVHs), and they also manifest as abnormalities in both high-level semantic and contextual processing and low-level perception. However, the literatures characterizing these abnormalities have largely been separate and have sometimes provided mutually exclusive accounts of aberrant language in schizophrenia. In this review, we propose that recent generative probabilistic frameworks of language processing can provide crucial insights that link these four lines of research. We first outline neural and cognitive evidence that real-time language comprehension and production normally involve internal generative circuits that propagate probabilistic predictions to perceptual cortices - predictions that are incrementally updated based on prediction error signals as new inputs are encountered. We then explain how disruptions to these circuits may compromise communicative abilities in schizophrenia by reducing the efficiency and robustness of both high-level language processing and low-level speech perception. We also argue that such disruptions may contribute to the phenomenology of thought-disordered speech and false perceptual inferences in the language system (i.e., AVHs). This perspective suggests a number of productive avenues for future research that may elucidate not only the mechanisms of language abnormalities in schizophrenia, but also promising directions for cognitive rehabilitation.