Differential Tracking of Linguistic vs. Mental State Content in Naturalistic Stimuli by Language and Theory of Mind (ToM) Brain Networks

Overlapping neural correlates underpin theory of mind and semantic cognition: Evidence from a meta-analysis of 344 functional neuroimaging studies

Key unanswered questions for cognitive neuroscience include whether social cognition is underpinned by specialised brain regions and to what extent it simultaneously depends on more domain-general systems. Until we glean a better understanding of the full set of contributions made by various systems, theories of social cognition will remain fundamentally limited. In the present study, we evaluate a recent proposal that semantic cognition plays a crucial role in supporting social cognition. While previous brain-based investigations have focused on dissociating these two systems, our primary aim was to assess the degree to which the neural correlates are overlapping, particularly within two key regions, the anterior temporal lobe (ATL) and the temporoparietal junction (TPJ). We focus on activation associated with theory of mind (ToM) and adopt a meta-analytic activation likelihood approach to synthesise a large set of functional neuroimaging studies and compare their results with studies of semantic cognition. As a key consideration, we sought to account for methodological differences across the two sets of studies, including the fact that ToM studies tend to use nonverbal stimuli while the semantics literature is dominated by language-based tasks. Overall, we observed consistent overlap between the two sets of brain regions, especially in the ATL and TPJ. This supports the claim that tasks involving ToM draw upon more general semantic retrieval processes. We also identified activation specific to ToM in the right TPJ, bilateral anterior mPFC, and right precuneus. This is consistent with the view that, nested amongst more domain-general systems, there is specialised circuitry that is tuned to social processes.

The Language Network Reliably "Tracks" Naturalistic Meaningful Nonverbal Stimuli

The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension but shows little or no response during many nonlinguistic cognitive tasks (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is semantics-our conceptual knowledge of the world. Given that the language network responds strongly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? In this study, we used a naturalistic cognition paradigm to test whether the cognitive and neural resources that are responsible for language processing are also recruited for processing semantically rich nonverbal stimuli. To do so, we measured BOLD responses to a set of ∼5-minute-long video and audio clips that consisted of meaningful event sequences but did not contain any linguistic content. We then used the intersubject correlation (ISC) approach (Hasson et al., 2004) to examine the extent to which the language network "tracks" these stimuli, that is, exhibits stimulus-related variation. Across all the regions of the language network, meaningful nonverbal stimuli elicited reliable ISCs. These ISCs were higher than the ISCs elicited by semantically impoverished nonverbal stimuli (e.g., a music clip), but substantially lower than the ISCs elicited by linguistic stimuli. Our results complement earlier findings from controlled experiments (e.g., Ivanova et al., 2021) in providing further evidence that the language network shows some sensitivity to semantic content in nonverbal stimuli.

Language is primarily a tool for communication rather than thought

Language is a defining characteristic of our species, but the function, or functions, that it serves has been debated for centuries. Here we bring recent evidence from neuroscience and allied disciplines to argue that in modern humans, language is a tool for communication, contrary to a prominent view that we use language for thinking. We begin by introducing the brain network that supports linguistic ability in humans. We then review evidence for a double dissociation between language and thought, and discuss several properties of language that suggest that it is optimized for communication. We conclude that although the emergence of language has unquestionably transformed human culture, language does not appear to be a prerequisite for complex thought, including symbolic thought. Instead, language is a powerful tool for the transmission of cultural knowledge; it plausibly co-evolved with our thinking and reasoning capacities, and only reflects, rather than gives rise to, the signature sophistication of human cognition.

Weighing the role of social cognition and executive functioning in pragmatics in the schizophrenia spectrum: A systematic review and meta-analysis

Pragmatic impairment is diffused in schizophrenia spectrum disorders, but the literature still debates its neurocognitive underpinnings. This systematic review and meta-analysis aimed to investigate the neurocognitive correlates of pragmatic disorders in schizophrenia and determine the weight of social cognition and executive functioning on such disorders. Of the 2,668 records retrieved from the literature, 16 papers were included in the systematic review, mostly focused on non-literal meanings and discourse production in schizophrenia. Ten studies were included in the meta-analysis: pragmatics was moderately associated with both social cognition and executive functions (especially inhibition), but the link with social cognition was stronger. The mediation analysis showed that social cognition mediated the relationship between executive functions and pragmatics. Based on this, we proposed a hierarchical neurocognitive model where pragmatics stems from social cognition, while executive functions are the fertile ground supporting the other two domains, and we discuss its theoretical and clinical implications.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

The neural basis of naturalistic semantic and social cognition

Decoding social environments and engaging meaningfully with other people are critical aspects of human cognition. Multiple cognitive systems, including social and semantic cognition, work alongside each other to support these processes. This study investigated shared processing between social and semantic systems using neuroimaging data collected during movie-viewing, which captures the multimodal environment in which social knowledge is exchanged. Semantic and social content from movie events (event-level) and movie transcripts (word-level) were used in parametric modulation analyses to test (1) the degree to which semantic and social information is processed within each respective network and (2) engagement of the same cross-network regions or the same domain-general hub located within the semantic network during semantic and social processing. Semantic word and event-level content engaged the same fronto-temporo-parietal network and a portion of the semantic hub in the anterior temporal lobe (ATL). Social word and event-level content engaged the supplementary motor area and right angular gyrus within the social network, but only social words engaged the domain-general semantic hub in left ATL. There was evidence of shared processing between the social and semantic systems in the dorsolateral portion of right ATL which was engaged by word and event-level semantic and social content. Overlap between the semantic and social word and event results was highly variable within and across participants, with the most consistent loci of overlap occurring in left inferior frontal, bilateral precentral and supramarginal gyri for social and semantic words and in bilateral superior temporal gyrus extending from ATL posteriorly into supramarginal gyri for social and semantic events. These results indicate a complex pattern of shared and distinct regions for social and semantic cognition during naturalistic processing. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on October 11, 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/ACWQY .

Left-Hemisphere Cortical Language Regions Respond Equally to Observed Dialogue and Monologue

Much of the language we encounter in our everyday lives comes in the form of conversation, yet the majority of research on the neural basis of language comprehension has used input from only one speaker at a time. Twenty adults were scanned while passively observing audiovisual conversations using functional magnetic resonance imaging. In a block-design task, participants watched 20 s videos of puppets speaking either to another puppet (the dialogue condition) or directly to the viewer (the monologue condition), while the audio was either comprehensible (played forward) or incomprehensible (played backward). Individually functionally localized left-hemisphere language regions responded more to comprehensible than incomprehensible speech but did not respond differently to dialogue than monologue. In a second task, participants watched videos (1-3 min each) of two puppets conversing with each other, in which one puppet was comprehensible while the other's speech was reversed. All participants saw the same visual input but were randomly assigned which character's speech was comprehensible. In left-hemisphere cortical language regions, the time course of activity was correlated only among participants who heard the same character speaking comprehensibly, despite identical visual input across all participants. For comparison, some individually localized theory of mind regions and right-hemisphere homologues of language regions responded more to dialogue than monologue in the first task, and in the second task, activity in some regions was correlated across all participants regardless of which character was speaking comprehensibly. Together, these results suggest that canonical left-hemisphere cortical language regions are not sensitive to differences between observed dialogue and monologue.

A social-semantic working-memory account for two canonical language areas

Language and social cognition are traditionally studied as separate cognitive domains, yet accumulative studies reveal overlapping neural correlates at the left ventral temporoparietal junction (vTPJ) and the left lateral anterior temporal lobe (lATL), which have been attributed to sentence processing and social concept activation. We propose a common cognitive component underlying both effects: social-semantic working memory. We confirmed two key predictions of our hypothesis using functional MRI. First, the left vTPJ and lATL showed sensitivity to sentences only when the sentences conveyed social meaning; second, these regions showed persistent social-semantic-selective activity after the linguistic stimuli disappeared. We additionally found that both regions were sensitive to the socialness of non-linguistic stimuli and were more tightly connected with the social-semantic-processing areas than with the sentence-processing areas. The converging evidence indicates the social-semantic working-memory function of the left vTPJ and lATL and challenges the general-semantic and/or syntactic accounts for the neural activity of these regions.

The words that little by little revealed everything: Neural response to lexical-semantic content during narrative comprehension

The ease with which narratives are understood belies the complexity of the information being conveyed and the cognitive processes that support comprehension. The meanings of the words must be rapidly accessed and integrated with the reader's mental representation of the overarching, unfolding scenario. A broad, bilateral brain network is engaged by this process, but it is not clear how words that vary on specific semantic dimensions, such as ambiguity, emotion, or socialness, engage the semantic, semantic control, or social cognition systems. In the present study, data from 48 participants who listened to The Little Prince audiobook during MRI scanning were selected from the Le Petit Prince dataset. The lexical and semantic content within the narrative was quantified from the transcript words with factor scores capturing Word Length, Semantic Flexibility, Emotional Strength, and Social Impact. These scores, along with word quantity variables, were used to investigate where these predictors co-vary with activation across the brain. In contrast to studies of isolated word processing, large networks were found to co-vary with the lexical and semantic content within the narrative. An increase in semantic content engaged the ventral portion of ventrolateral ATL, consistent with its role as a semantic hub. Decreased semantic content engaged temporal pole and inferior parietal lobule, which may reflect semantic integration. The semantic control network was engaged by words with low Semantic Flexibility, perhaps due to the demand required to process infrequent, less semantically diverse language. Activation in ATL co-varied with an increase in Social Impact, which is consistent with the claim that social knowledge is housed within the neural architecture of the semantic system. These results suggest that current models of language processing may present an impoverished estimate of the neural systems that coordinate to support narrative comprehension, and, by extension, real-world language processing.

The human language system, including its inferior frontal component in "Broca's area," does not support music perception

Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.

Linguistic markers of psychosis in Mandarin Chinese: Relations to theory of mind

Disorganized and impoverished language is a key feature of schizophrenia (Sz), but whether and which linguistic changes previously observed in Indo-European languages generalize to other languages remains unclear. Targeting Mandarin Chinese, we aimed to profile aspects of grammatical complexity that we hypothesized would be reduced in schizophrenia in a task of verbalizing social events. 51 individuals with Sz and 39 controls participated in the animated triangles task, a standardized measure of theory of mind (ToM), in which participants describe triangles moving in either a random or an 'intentional' condition. Results revealed that clauses embedded as arguments in other clauses were reduced in Sz, and that both groups produced such clauses and grammatical aspect more frequently in the intentional condition. ToM scores specifically correlated with production of embedded argument clauses. These results document grammatical impoverishment in Sz in Chinese across several structural domains, which in some of its specific aspects relate to mentalizing performance.

No evidence of theory of mind reasoning in the human language network

Language comprehension and the ability to infer others' thoughts (theory of mind [ToM]) are interrelated during development and language use. However, neural evidence that bears on the relationship between language and ToM mechanisms is mixed. Although robust dissociations have been reported in brain disorders, brain activations for contrasts that target language and ToM bear similarities, and some have reported overlap. We take another look at the language-ToM relationship by evaluating the response of the language network, as measured with fMRI, to verbal and nonverbal ToM across 151 participants. Individual-participant analyses reveal that all core language regions respond more strongly when participants read vignettes about false beliefs compared to the control vignettes. However, we show that these differences are largely due to linguistic confounds, and no such effects appear in a nonverbal ToM task. These results argue against cognitive and neural overlap between language processing and ToM. In exploratory analyses, we find responses to social processing in the "periphery" of the language network-right-hemisphere homotopes of core language areas and areas in bilateral angular gyri-but these responses are not selectively ToM-related and may reflect general visual semantic processing.

Language Network Dysfunction and Formal Thought Disorder in Schizophrenia

BACKGROUND

Pathophysiological inquiries into schizophrenia require a consideration of one of its most defining features: disorganization and impoverishment in verbal behavior. This feature, often captured using the term Formal Thought Disorder (FTD), still remains to be one of the most poorly understood and understudied dimensions of schizophrenia. In particular, the large-scale network level dysfunction that contributes to FTD remains obscure to date.

STUDY DESIGN

In this narrative review, we consider the various challenges that need to be addressed for us to move towards mapping FTD (construct) to a brain network level account (circuit).

STUDY RESULTS

The construct-to-circuit mapping goal is now becoming more plausible than it ever was, given the parallel advent of brain stimulation and the tools providing objective readouts of human speech. Notwithstanding this, several challenges remain to be overcome before we can decisively map the neural basis of FTD. We highlight the need for phenotype refinement, robust experimental designs, informed analytical choices, and present plausible targets in and beyond the Language Network for brain stimulation studies in FTD.

CONCLUSIONS

Developing a therapeutically beneficial pathophysiological model of FTD is a challenging endeavor, but holds the promise of improving interpersonal communication and reducing social disability in schizophrenia. Addressing the issues raised in this review will be a decisive step in this direction.