Lack of selectivity for syntax relative to word meanings throughout the language network

Functional dissociation of the language network and other cognition in early childhood

Is language distinct from other cognition during development? Does neural machinery for language emerge from general-purpose neural mechanisms, becoming tuned for language after years of experience and maturation? Answering these questions will shed light on the origins of domain-specificity in the brain. We address these questions using precision fMRI, scanning young children (35 months to 9 years of age) on an auditory language localizer, spatial working memory localizer (engaging the domain-general multiple demand [MD] network), and a resting-state scan. We create subject-specific functional regions of interest for each network and examine their selectivity, specificity, and functional connectivity. We find young children show domain-specific, left-lateralized language activation, and that the language network is not responsive to domain-general cognitive load. Additionally, the cortically adjacent MD network is selective to cognitive load, but not to language. These networks show higher within versus between-network functional connectivity. This connectivity is stable across ages (examined cross-sectionally and longitudinally), whereas language responses increase with age and across time within subject, reflecting a domain-specific developmental change. Overall, we provide evidence for a double dissociation of the language and MD network throughout development, in both their function and connectivity. These findings suggest that domain-specificity, even for uniquely human cognition like language, develops early and distinctly from mechanisms that presumably support other human cognition.

Precision fMRI reveals that the language network exhibits adult-like left-hemispheric lateralization by 4 years of age

Left hemisphere damage in adulthood often leads to linguistic deficits, but many cases of early damage leave linguistic processing preserved, and a functional language system can develop in the right hemisphere. To explain this early apparent equipotentiality of the two hemispheres for language, some have proposed that the language system is bilateral during early development and only becomes left-lateralized with age. We examined language lateralization using functional magnetic resonance imaging with two large pediatric cohorts (total n=273 children ages 4-16; n=107 adults). Strong, adult-level left-hemispheric lateralization (in activation volume and response magnitude) was evident by age 4. Thus, although the right hemisphere can take over language function in some cases of early brain damage, and although some features of the language system do show protracted development (magnitude of language response and strength of inter-regional correlations in the language network), the left-hemisphere bias for language is robustly present by 4 years of age. These results call for alternative accounts of early equipotentiality of the two hemispheres for language.

Significance Statement

Language is the most canonical function that shows a strong hemispheric asymmetry in adult brains. However, whether the language system is already lateralized to the left hemisphere early in development has long been debated, given that early left-hemisphere damage often leaves language processing unimpaired. We examined the developmental trajectory of language lateralization in two large-scale pediatric datasets using robust individual-subject fMRI approaches. We found that the language system exhibits adult-like left-hemispheric lateralization by age 4, although other aspects of the neural infrastructure for language show a clear change between age 4 and late childhood. These findings challengethe claim that the language system is bilateral during early development and call for alternative accounts of early hemispheric equipotentiality for language.

Longitudinal associations between language network characteristics in the infant brain and school-age reading abilities are mediated by early-developing phonological skills

Reading acquisition is a prolonged learning process relying on language development starting in utero. Behavioral longitudinal studies reveal prospective associations between infant language abilities and preschool/kindergarten phonological development that relates to subsequent reading performance. While recent pediatric neuroimaging work has begun to characterize the neural network underlying language development in infants, how this neural network scaffolds long-term language and reading acquisition remains unknown. We addressed this question in a 7-year longitudinal study from infancy to school-age. Seventy-six infants completed resting-state fMRI scanning, and underwent standardized language assessments in kindergarten. Of this larger cohort, forty-one were further assessed on their emergent word reading abilities after receiving formal reading instructions. Hierarchical clustering analyses identified a modular infant language network in which functional connectivity (FC) of the inferior frontal module prospectively correlated with kindergarten-age phonological skills and emergent word reading abilities. These correlations were obtained when controlling for infant age at scan, nonverbal IQ and parental education. Furthermore, kindergarten-age phonological skills mediated the relationship between infant FC and school-age reading abilities, implying a critical mid-way milestone for long-term reading development from infancy. Overall, our findings illuminate the neurobiological mechanisms by which infant language capacities could scaffold long-term reading acquisition.

Highlights

Clustering analyses revealed a modular language network in the infant brainInfant language network characteristics associate with school-age reading outcomesThese longitudinal associations are mediated by kindergarten-age phonological skills.

Longitudinal associations between language network characteristics in the infant brain and school-age reading abilities are mediated by early-developing phonological skills

Reading acquisition is a prolonged learning process relying on language development starting in utero. Behavioral longitudinal studies reveal prospective associations between infant language abilities and preschool/kindergarten phonological development that relates to subsequent reading performance. While recent pediatric neuroimaging work has begun to characterize the neural network underlying language development in infants, how this neural network scaffolds long-term language and reading acquisition remains unknown. We addressed this question in a 7-year longitudinal study from infancy to school-age. Seventy-six infants completed resting-state fMRI scanning, and underwent standardized language assessments in kindergarten. Of this larger cohort, forty-one were further assessed on their emergent word reading abilities after receiving formal reading instructions. Hierarchical clustering analyses identified a modular infant language network in which functional connectivity (FC) of the inferior frontal module prospectively correlated with kindergarten-age phonological skills and emergent word reading abilities. These correlations were obtained when controlling for infant age at scan, nonverbal IQ and parental education. Furthermore, kindergarten-age phonological skills mediated the relationship between infant FC and school-age reading abilities, implying a critical mid-way milestone for long-term reading development from infancy. Overall, our findings illuminate the neurobiological mechanisms by which infant language capacities could scaffold long-term reading acquisition.

Dissociating language and thought in large language models

Large language models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence (knowledge of linguistic rules and patterns) and functional linguistic competence (understanding and using language in the world). We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in human-like ways would need to master both of these competence types, which, in turn, could require the emergence of separate mechanisms specialized for formal versus functional linguistic competence.

Distributed Sensitivity to Syntax and Semantics throughout the Language Network

Human language is expressive because it is compositional: The meaning of a sentence (semantics) can be inferred from its structure (syntax). It is commonly believed that language syntax and semantics are processed by distinct brain regions. Here, we revisit this claim using precision fMRI methods to capture separation or overlap of function in the brains of individual participants. Contrary to prior claims, we find distributed sensitivity to both syntax and semantics throughout a broad frontotemporal brain network. Our results join a growing body of evidence for an integrated network for language in the human brain within which internal specialization is primarily a matter of degree rather than kind, in contrast with influential proposals that advocate distinct specialization of different brain areas for different types of linguistic functions.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

Artificial Neural Network Language Models Predict Human Brain Responses to Language Even After a Developmentally Realistic Amount of Training

Artificial neural networks have emerged as computationally plausible models of human language processing. A major criticism of these models is that the amount of training data they receive far exceeds that of humans during language learning. Here, we use two complementary approaches to ask how the models' ability to capture human fMRI responses to sentences is affected by the amount of training data. First, we evaluate GPT-2 models trained on 1 million, 10 million, 100 million, or 1 billion words against an fMRI benchmark. We consider the 100-million-word model to be developmentally plausible in terms of the amount of training data given that this amount is similar to what children are estimated to be exposed to during the first 10 years of life. Second, we test the performance of a GPT-2 model trained on a 9-billion-token dataset to reach state-of-the-art next-word prediction performance on the human benchmark at different stages during training. Across both approaches, we find that (i) the models trained on a developmentally plausible amount of data already achieve near-maximal performance in capturing fMRI responses to sentences. Further, (ii) lower perplexity-a measure of next-word prediction performance-is associated with stronger alignment with human data, suggesting that models that have received enough training to achieve sufficiently high next-word prediction performance also acquire representations of sentences that are predictive of human fMRI responses. In tandem, these findings establish that although some training is necessary for the models' predictive ability, a developmentally realistic amount of training (∼100 million words) may suffice.

Neural correlates of semantic-driven syntactic parsing in sentence comprehension

For sentence comprehension, information carried by semantic relations between constituents must be combined with other information to decode the constituent structure of a sentence, due to atypical and noisy situations of language use. Neural correlates of decoding sentence structure by semantic information have remained largely unexplored. In this functional MRI study, we examine the neural basis of semantic-driven syntactic parsing during sentence reading and compare it with that of other types of syntactic parsing driven by word order and case marking. Chinese transitive sentences of various structures were investigated, differing in word order, case making, and agent-patient semantic relations (i.e., same vs. different in animacy). For the non-canonical unmarked sentences without usable case marking, a semantic-driven effect triggered by agent-patient ambiguity was found in the left inferior frontal gyrus opercularis (IFGoper) and left inferior parietal lobule, with the activity not being modulated by naturalness factors of the sentences. The comparison between each type of non-canonical sentences with canonical sentences revealed that the non-canonicity effect engaged the left posterior frontal and temporal regions, in line with previous studies. No extra neural activity was found responsive to case marking within the non-canonical sentences. A word order effect across all types of sentences was also found in the left IFGoper, suggesting a common neural substrate between different types of parsing. The semantic-driven effect was also observed for the non-canonical marked sentences but not for the canonical sentences, suggesting that semantic information is used in decoding sentence structure in addition to case marking. The current findings illustrate the neural correlates of syntactic parsing with semantics, and provide neural evidence of how semantics facilitates syntax together with other information.

Lexical-Semantic Content, Not Syntactic Structure, Is the Main Contributor to ANN-Brain Similarity of fMRI Responses in the Language Network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI data set of responses to n = 627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we (i) perturbed sentences' word order, (ii) removed different subsets of words, or (iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical-semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust as to whether the mapping model is trained on intact or perturbed stimuli and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

Limited but specific engagement of the mature language network during linguistic statistical learning

Statistical learning (SL) is the ability to detect and learn regularities from input and is foundational to language acquisition. Despite the dominant role of SL as a theoretical construct for language development, there is a lack of direct evidence supporting the shared neural substrates underlying language processing and SL. It is also not clear whether the similarities, if any, are related to linguistic processing, or statistical regularities in general. The current study tests whether the brain regions involved in natural language processing are similarly recruited during auditory, linguistic SL. Twenty-two adults performed an auditory linguistic SL task, an auditory nonlinguistic SL task, and a passive story listening task as their neural activation was monitored. Within the language network, the left posterior temporal gyrus showed sensitivity to embedded speech regularities during auditory, linguistic SL, but not auditory, nonlinguistic SL. Using a multivoxel pattern similarity analysis, we uncovered similarities between the neural representation of auditory, linguistic SL, and language processing within the left posterior temporal gyrus. No other brain regions showed similarities between linguistic SL and language comprehension, suggesting that a shared neurocomputational process for auditory SL and natural language processing within the left posterior temporal gyrus is specific to linguistic stimuli.

High-level language brain regions process sublexical regularities

A network of left frontal and temporal brain regions supports language processing. This "core" language network stores our knowledge of words and constructions as well as constraints on how those combine to form sentences. However, our linguistic knowledge additionally includes information about phonemes and how they combine to form phonemic clusters, syllables, and words. Are phoneme combinatorics also represented in these language regions? Across five functional magnetic resonance imaging experiments, we investigated the sensitivity of high-level language processing brain regions to sublexical linguistic regularities by examining responses to diverse nonwords-sequences of phonemes that do not constitute real words (e.g. punes, silory, flope). We establish robust responses in the language network to visually (experiment 1a, n = 605) and auditorily (experiments 1b, n = 12, and 1c, n = 13) presented nonwords. In experiment 2 (n = 16), we find stronger responses to nonwords that are more well-formed, i.e. obey the phoneme-combinatorial constraints of English. Finally, in experiment 3 (n = 14), we provide suggestive evidence that the responses in experiments 1 and 2 are not due to the activation of real words that share some phonology with the nonwords. The results suggest that sublexical regularities are stored and processed within the same fronto-temporal network that supports lexical and syntactic processes.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of the brain response associated with each sentence. We then use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress the activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also non-invasively control neural activity in higher-level cortical areas, such as the language network.

The organization of the semantic network as reflected by the neural correlates of six semantic dimensions

Multiple sensory-motor and non-sensory-motor dimensions have been proposed for semantic representation, but it remains unclear how the semantic system is organized along them in the human brain. Using naturalistic fMRI data and large-scale semantic ratings, we investigated the overlaps and dissociations between the neural correlates of six semantic dimensions: vision, motor, socialness, emotion, space, and time. Our findings revealed a more complex semantic atlas than what is predicted by current neurobiological models of semantic representation. Brain regions that are selectively sensitive to specific semantic dimensions were found both within and outside the brain networks assumed to represent multimodal general and/or abstract semantics. Overlaps between the neural correlates of different semantic dimensions were mainly found inside the default mode network, concentrated in the left anterior superior temporal gyrus and angular gyrus, which have been proposed as two connector hubs that bridge the multimodal experiential semantic system and the language-supported semantic system.

Neural specificity for semantic and syntactic processing in Spanish-English bilingual children

Brain development for language processing is associated with neural specialization of left perisylvian pathways, but this has not been investigated in young bilinguals. We examined specificity for syntax and semantics in early exposed Spanish-English speaking children (N = 65, ages 7-11) using an auditory sentence judgement task in English, their dominant language of use. During functional near infrared spectroscopy (fNIRS), the morphosyntax task elicited activation in left inferior frontal gyrus (IFG) and the semantic task elicited activation in left posterior middle temporal gyrus (MTG). Task comparisons revealed specialization in left superior temporal (STG) for morphosyntax and left MTG and angular gyrus for semantics. Although skills in neither language were uniquely related to specialization, skills in both languages were related to engagement of the left MTG for semantics and left IFG for syntax. These results are consistent with models suggesting a positive cross-linguistic interaction in those with higher language proficiency.

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots-individuals who speak five or more languages-process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants' high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language network's response magnitude scales with the degree of engagement of linguistic computations (e.g. related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages coexist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

Common and distinct patterns underlying different linguistic tasks: multivariate disconnectome symptom mapping in poststroke patients

Numerous studies have been devoted to neural mechanisms of a variety of linguistic tasks (e.g. speech comprehension and production). To date, however, whether and how the neural patterns underlying different linguistic tasks are similar or differ remains elusive. In this study, we compared the neural patterns underlying 3 linguistic tasks mainly concerning speech comprehension and production. To address this, multivariate regression approaches with lesion/disconnection symptom mapping were applied to data from 216 stroke patients with damage to the left hemisphere. The results showed that lesion/disconnection patterns could predict both poststroke scores of speech comprehension and production tasks; these patterns exhibited shared regions on the temporal pole of the left hemisphere as well as unique regions contributing to the prediction for each domain. Lower scores in speech comprehension tasks were associated with lesions/abnormalities in the superior temporal gyrus and middle temporal gyrus, while lower scores in speech production tasks were associated with lesions/abnormalities in the left inferior parietal lobe and frontal lobe. These results suggested an important role of the ventral and dorsal stream pathways in speech comprehension and production (i.e. supporting the dual stream model) and highlighted the applicability of the novel multivariate disconnectome-based symptom mapping in cognitive neuroscience research.

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots-individuals who speak five or more languages-process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher-proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants' high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language network's response magnitude scales with the degree of engagement of linguistic computations (e.g., related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages co-exist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

Domain-general and language-specific contributions to speech production in a second language: an fMRI study using functional localizers

For bilinguals, speaking in a second language (L2) compared to the native language (L1) is usually more difficult. In this study we asked whether the difficulty in L2 production reflects increased demands imposed on domain-general or core language mechanisms. We compared the brain response to speech production in L1 and L2 within two functionally-defined networks in the brain: the Multiple Demand (MD) network and the language network. We found that speech production in L2 was linked to a widespread increase of brain activity in the domain-general MD network. The language network did not show a similarly robust differences in processing speech in the two languages, however, we found increased response to L2 production in the language-specific portion of the left inferior frontal gyrus (IFG). To further explore our results, we have looked at domain-general and language-specific response within the brain structures postulated to form a Bilingual Language Control (BLC) network. Within this network, we found a robust increase in response to L2 in the domain-general, but also in some language-specific voxels including in the left IFG. Our findings show that L2 production strongly engages domain-general mechanisms, but only affects language sensitive portions of the left IFG. These results put constraints on the current model of bilingual language control by precisely disentangling the domain-general and language-specific contributions to the difficulty in speech production in L2.

Efficient functional localization of language regions in the brain

Important recent advances in the cognitive neuroscience of language have been made using functional localizers to demarcate language-selective regions in individual brains. Although single-subject localizers offer insights that are unavailable in classic group analyses, they require additional scan time that imposes costs on investigators and participants. In particular, the unique practical challenges of scanning children and other special populations has led to less adoption of localizers for neuroimaging research with these theoretically and clinically important groups. Here, we examined how measurements of the spatial extent and functional response profiles of language regions are affected by the duration of an auditory language localizer. We compared how parametrically smaller amounts of data collected from one scanning session affected (i) consistency of group-level whole-brain parcellations, (ii) functional selectivity of subject-level activation in individually defined functional regions of interest (fROIs), (iii) sensitivity and specificity of subject-level whole-brain and fROI activation, and (iv) test-retest reliability of subject-level whole-brain and fROI activation. For many of these metrics, the localizer duration could be reduced by 50-75% while preserving the stability and reliability of both the spatial extent and functional response profiles of language areas. These results indicate that, for most measures relevant to cognitive neuroimaging studies, the brain's language network can be localized just as effectively with 3.5 min of scan time as it can with 12 min. Minimizing the time required to reliably localize the brain's language network allows more effective localizer use in situations where each minute of scan time is particularly precious.

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax, and Context

A fundamental question in neurolinguistics concerns the brain regions involved in syntactic and semantic processing during speech comprehension, both at the lexical (word processing) and supra-lexical levels (sentence and discourse processing). To what extent are these regions separated or intertwined? To address this question, we introduce a novel approach exploiting neural language models to generate high-dimensional feature sets that separately encode semantic and syntactic information. More precisely, we train a lexical language model, GloVe, and a supra-lexical language model, GPT-2, on a text corpus from which we selectively removed either syntactic or semantic information. We then assess to what extent the features derived from these information-restricted models are still able to predict the fMRI time courses of humans listening to naturalistic text. Furthermore, to determine the windows of integration of brain regions involved in supra-lexical processing, we manipulate the size of contextual information provided to GPT-2. The analyses show that, while most brain regions involved in language comprehension are sensitive to both syntactic and semantic features, the relative magnitudes of these effects vary across these regions. Moreover, regions that are best fitted by semantic or syntactic features are more spatially dissociated in the left hemisphere than in the right one, and the right hemisphere shows sensitivity to longer contexts than the left. The novelty of our approach lies in the ability to control for the information encoded in the models' embeddings by manipulating the training set. These "information-restricted" models complement previous studies that used language models to probe the neural bases of language, and shed new light on its spatial organization.

Modeling Brain Representations of Words' Concreteness in Context Using GPT-2 and Human Ratings

The meaning of most words in language depends on their context. Understanding how the human brain extracts contextualized meaning, and identifying where in the brain this takes place, remain important scientific challenges. But technological and computational advances in neuroscience and artificial intelligence now provide unprecedented opportunities to study the human brain in action as language is read and understood. Recent contextualized language models seem to be able to capture homonymic meaning variation ("bat", in a baseball vs. a vampire context), as well as more nuanced differences of meaning-for example, polysemous words such as "book", which can be interpreted in distinct but related senses ("explain a book", information, vs. "open a book", object) whose differences are fine-grained. We study these subtle differences in lexical meaning along the concrete/abstract dimension, as they are triggered by verb-noun semantic composition. We analyze functional magnetic resonance imaging (fMRI) activations elicited by Italian verb phrases containing nouns whose interpretation is affected by the verb to different degrees. By using a contextualized language model and human concreteness ratings, we shed light on where in the brain such fine-grained meaning variation takes place and how it is coded. Our results show that phrase concreteness judgments and the contextualized model can predict BOLD activation associated with semantic composition within the language network. Importantly, representations derived from a complex, nonlinear composition process consistently outperform simpler composition approaches. This is compatible with a holistic view of semantic composition in the brain, where semantic representations are modified by the process of composition itself. When looking at individual brain areas, we find that encoding performance is statistically significant, although with differing patterns of results, suggesting differential involvement, in the posterior superior temporal sulcus, inferior frontal gyrus and anterior temporal lobe, and in motor areas previously associated with processing of concreteness/abstractness.

Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely

Word co-occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs' semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pretrained LLMs (from 2018's BERT to 2023's MPT) assign a higher likelihood to plausible descriptions of agent-patient interactions than to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n = 1215), we found that pretrained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign a higher likelihood to possible versus impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely versus unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i) LLM scores are driven by both plausibility and surface-level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.

Grammatical Parallelism in Aphasia: A Lesion-Symptom Mapping Study

Sentence structure, or syntax, is potentially a uniquely creative aspect of the human mind. Neuropsychological experiments in the 1970s suggested parallel syntactic production and comprehension deficits in agrammatic Broca's aphasia, thought to result from damage to syntactic mechanisms in Broca's area in the left frontal lobe. This hypothesis was sometimes termed overarching agrammatism, converging with developments in linguistic theory concerning central syntactic mechanisms supporting language production and comprehension. However, the evidence supporting an association among receptive syntactic deficits, expressive agrammatism, and damage to frontal cortex is equivocal. In addition, the relationship among a distinct grammatical production deficit in aphasia, paragrammatism, and receptive syntax has not been assessed. We used lesion-symptom mapping in three partially overlapping groups of left-hemisphere stroke patients to investigate these issues: grammatical production deficits in a primary group of 53 subjects and syntactic comprehension in larger sample sizes (N = 130, 218) that overlapped with the primary group. Paragrammatic production deficits were significantly associated with multiple analyses of syntactic comprehension, particularly when incorporating lesion volume as a covariate, but agrammatic production deficits were not. The lesion correlates of impaired performance of syntactic comprehension were significantly associated with damage to temporal lobe regions, which were also implicated in paragrammatism, but not with the inferior and middle frontal regions implicated in expressive agrammatism. Our results provide strong evidence against the overarching agrammatism hypothesis. By contrast, our results suggest the possibility of an alternative grammatical parallelism hypothesis rooted in paragrammatism and a central syntactic system in the posterior temporal lobe.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

The language network is not engaged in object categorization

The relationship between language and thought is the subject of long-standing debate. One claim states that language facilitates categorization of objects based on a certain feature (e.g. color) through the use of category labels that reduce interference from other, irrelevant features. Therefore, language impairment is expected to affect categorization of items grouped by a single feature (low-dimensional categories, e.g. "Yellow Things") more than categorization of items that share many features (high-dimensional categories, e.g. "Animals"). To test this account, we conducted two behavioral studies with individuals with aphasia and an fMRI experiment with healthy adults. The aphasia studies showed that selective low-dimensional categorization impairment was present in some, but not all, individuals with severe anomia and was not characteristic of aphasia in general. fMRI results revealed little activity in language-responsive brain regions during both low- and high-dimensional categorization; instead, categorization recruited the domain-general multiple-demand network (involved in wide-ranging cognitive tasks). Combined, results demonstrate that the language system is not implicated in object categorization. Instead, selective low-dimensional categorization impairment might be caused by damage to brain regions responsible for cognitive control. Our work adds to the growing evidence of the dissociation between the language system and many cognitive tasks in adults.

Contribution and functional connectivity between cerebrum and cerebellum on sub-lexical and lexical-semantic processing of verbs

Language comprehension involves both sub-lexical (e.g., phonological) and lexical-semantic processing. We conducted a task using functional magnetic resonance imaging (fMRI) to compare the processing of verbs in these two domains. Additionally, we examined the representation of concrete-motor and abstract-non-motor concepts by including two semantic categories of verbs: motor and mental. The findings indicate that sub-lexical processing during the reading of pseudo-verbs primarily involves the left dorsal stream of the perisylvian network, while lexical-semantic representation during the reading of verbs predominantly engages the ventral stream. According to the embodied or grounded cognition approach, modality-specific mechanisms (such as sensory-motor systems) and the well-established multimodal left perisylvian network contribute to the semantic representation of both concrete and abstract verbs. Our study identified the visual system as a preferential modality-specific system for abstract-mental verbs, which exhibited functional connectivity with the right crus I/lobule VI of the cerebellum. Taken together, these results confirm the dissociation between sub-lexical and lexical-semantic processing and provide neurobiological evidence of functional coupling between specific visual modality regions and the right cerebellum, forming a network that supports the semantic representation of abstract concepts. Further, the results shed light on the underlying mechanisms of semantic processing and contribute to our understanding of how the brain processes abstract concepts.

Constructed languages are processed by the same brain mechanisms as natural languages

What constitutes a language? Natural languages share some features with other domains: from math, to music, to gesture. However, the brain mechanisms that process linguistic input are highly specialized, showing little or no response to diverse non-linguistic tasks. Here, we examine constructed languages (conlangs) to ask whether they draw on the same neural mechanisms as natural languages, or whether they instead pattern with domains like math and logic. Using individual-subject fMRI analyses, we show that understanding conlangs recruits the same brain areas as natural language comprehension. This result holds for Esperanto (n=19 speakers)- created to resemble natural languages-and fictional conlangs (Klingon (n=10), Na'vi (n=9), High Valyrian (n=3), and Dothraki (n=3)), created to differ from natural languages, and suggests that conlangs and natural languages share critical features and that the notable differences between conlangs and natural language are not consequential for the cognitive and neural mechanisms that they engage.

The human language system, including its inferior frontal component in "Broca's area," does not support music perception

Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.

No evidence of theory of mind reasoning in the human language network

Language comprehension and the ability to infer others' thoughts (theory of mind [ToM]) are interrelated during development and language use. However, neural evidence that bears on the relationship between language and ToM mechanisms is mixed. Although robust dissociations have been reported in brain disorders, brain activations for contrasts that target language and ToM bear similarities, and some have reported overlap. We take another look at the language-ToM relationship by evaluating the response of the language network, as measured with fMRI, to verbal and nonverbal ToM across 151 participants. Individual-participant analyses reveal that all core language regions respond more strongly when participants read vignettes about false beliefs compared to the control vignettes. However, we show that these differences are largely due to linguistic confounds, and no such effects appear in a nonverbal ToM task. These results argue against cognitive and neural overlap between language processing and ToM. In exploratory analyses, we find responses to social processing in the "periphery" of the language network-right-hemisphere homotopes of core language areas and areas in bilateral angular gyri-but these responses are not selectively ToM-related and may reflect general visual semantic processing.

Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI dataset of responses to n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we i) perturbed sentences' word order, ii) removed different subsets of words, or iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust to whether the mapping model is trained on intact or perturbed stimuli, and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

Non-literal language processing is jointly supported by the language and theory of mind networks: Evidence from a novel meta-analytic fMRI approach

Going beyond the literal meaning of language is key to communicative success. However, the mechanisms that support non-literal inferences remain debated. Using a novel meta-analytic approach, we evaluate the contribution of linguistic, social-cognitive, and executive mechanisms to non-literal interpretation. We identified 74 fMRI experiments (n = 1,430 participants) from 2001 to 2021 that contrasted non-literal language comprehension with a literal control condition, spanning ten phenomena (e.g., metaphor, irony, indirect speech). Applying the activation likelihood estimation approach to the 825 activation peaks yielded six left-lateralized clusters. We then evaluated the locations of both the individual-study peaks and the clusters against probabilistic functional atlases (cf. anatomical locations, as is typically done) for three candidate brain networks-the language-selective network (Fedorenko, Behr, & Kanwisher, 2011), which supports language processing, the Theory of Mind (ToM) network (Saxe & Kanwisher, 2003), which supports social inferences, and the domain-general Multiple-Demand (MD) network (Duncan, 2010), which supports executive control. These atlases were created by overlaying individual activation maps of participants who performed robust and extensively validated 'localizer' tasks that selectively target each network in question (n = 806 for language; n = 198 for ToM; n = 691 for MD). We found that both the individual-study peaks and the ALE clusters fell primarily within the language network and the ToM network. These results suggest that non-literal processing is supported by both i) mechanisms that process literal linguistic meaning, and ii) mechanisms that support general social inference. They thus undermine a strong divide between literal and non-literal aspects of language and challenge the claim that non-literal processing requires additional executive resources.

Spatiotemporally distributed frontotemporal networks for sentence reading

Reading a sentence entails integrating the meanings of individual words to infer more complex, higher-order meaning. This highly rapid and complex human behavior is known to engage the inferior frontal gyrus (IFG) and middle temporal gyrus (MTG) in the language-dominant hemisphere, yet whether there are distinct contributions of these regions to sentence reading is still unclear. To probe these neural spatiotemporal dynamics, we used direct intracranial recordings to measure neural activity while reading sentences, meaning-deficient Jabberwocky sentences, and lists of words or pseudowords. We isolated two functionally and spatiotemporally distinct frontotemporal networks, each sensitive to distinct aspects of word and sentence composition. The first distributed network engages the IFG and MTG, with IFG activity preceding MTG. Activity in this network ramps up over the duration of a sentence and is reduced or absent during Jabberwocky and word lists, implying its role in the derivation of sentence-level meaning. The second network engages the superior temporal gyrus and the IFG, with temporal responses leading those in frontal lobe, and shows greater activation for each word in a list than those in sentences, suggesting that sentential context enables greater efficiency in the lexical and/or phonological processing of individual words. These adjacent, yet spatiotemporally dissociable neural mechanisms for word- and sentence-level processes shed light on the richly layered semantic networks that enable us to fluently read. These results imply distributed, dynamic computation across the frontotemporal language network rather than a clear dichotomy between the contributions of frontal and temporal structures.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

Uncertainty aversion predicts the neural expansion of semantic representations

Correctly identifying the meaning of a stimulus requires activating the appropriate semantic representation among many alternatives. One way to reduce this uncertainty is to differentiate semantic representations from each other, thereby expanding the semantic space. Here, in four experiments, we test this semantic-expansion hypothesis, finding that uncertainty-averse individuals exhibit increasingly differentiated and separated semantic representations. This effect is mirrored at the neural level, where uncertainty aversion predicts greater distances between activity patterns in the left inferior frontal gyrus when reading words, and enhanced sensitivity to the semantic ambiguity of these words in the ventromedial prefrontal cortex. Two direct tests of the behavioural consequences of semantic expansion further reveal that uncertainty-averse individuals exhibit reduced semantic interference and poorer generalization. Together, these findings show that the internal structure of our semantic representations acts as an organizing principle to make the world more identifiable.

Intact reading ability despite lacking a canonical visual word form area in an individual born without the left superior temporal lobe

The visual word form area (VWFA), a region canonically located within left ventral temporal cortex (VTC), is specialized for orthography in literate adults presumbly due to its connectivity with frontotemporal language regions. But is a typical, left-lateralized language network critical for the VWFA's emergence? We investigated this question in an individual (EG) born without the left superior temporal lobe but who has normal reading ability. EG showed canonical typical face-selectivity bilateraly but no wordselectivity either in right VWFA or in the spared left VWFA. Moreover, in contrast with the idea that the VWFA is simply part of the language network, no part of EG's VTC showed selectivity to higher-level linguistic processing. Interestingly, EG's VWFA showed reliable multivariate patterns that distinguished words from other categories. These results suggest that a typical left-hemisphere language network is necessary for acanonical VWFA, and that orthographic processing can otherwise be supported by a distributed neural code.

Behavioural and neural structure of fluent speech production deficits in aphasia

Deficits in fluent speech production following left hemisphere stroke are a central concern because of their impact on patients' lives and the insight they provide about the neural organization of language processing. Fluent speech production requires the rapid coordination of phonological, semantic, and syntactic processing, so this study examined how deficits in connected speech relate to these language sub-systems. Behavioural data (N = 69 participants with aphasia following left hemisphere stroke) consisted of a diverse and comprehensive set of narrative speech production measures and measures of overall severity, semantic deficits, and phonological deficits. These measures were entered into a principal component analysis with bifactor rotation-a latent structure model where each item loads on a general factor that reflects what is common among the items, and orthogonal factors that explain variance not accounted for by the general factor. Lesion data were available for 58 of the participants, and each factor score was analysed with multivariate lesion-symptom mapping. Effects of connectivity disruption were evaluated using robust regression with tract disconnection or graph theoretic measures of connectivity as predictors. The principal component analysis produced a four-factor solution that accounted for 70.6% of the variance in the data, with a general factor corresponding to the overall severity and length and complexity of speech output (complexity factor), a lexical syntax factor, and independent factors for Semantics and Phonology. Deficits in the complexity of speech output were associated with a large temporo-parietal region, similar to overall aphasia severity. The lexical syntax factor was associated with damage in a relatively small set of fronto-parietal regions which may reflect the recruitment of control systems to support retrieval and correct usage of lexical items that primarily serve a syntactic rather than semantic function. Tract-based measures of connectivity disruption were not statistically associated with the deficit scores after controlling for overall lesion volume. Language network efficiency and average clustering coefficient within the language network were significantly associated with deficit scores after controlling for overall lesion volume. These results highlight overall severity as the critical contributor to fluent speech in post-stroke aphasia, with a dissociable factor corresponding to lexical syntax.

Agrammatic output in non-fluent, including Broca's, aphasia as a rational behavior

Background

Speech of individuals with non-fluent, including Broca's, aphasia is often characterized as "agrammatic" because their output mostly consists of nouns and, to a lesser extent, verbs and lacks function words, like articles and prepositions, and correct morphological endings. Among the earliest accounts of agrammatic output in the early 1900s was the "economy of effort" idea whereby agrammatic output is construed as a way of coping with increases in the cost of language production. This idea resurfaced in the 1980s, but in general, the field of language research has largely focused on accounts of agrammatism that postulated core deficits in syntactic knowledge.

Aims

We here revisit the economy of effort hypothesis in light of increasing emphasis in cognitive science on rational and efficient behavior.

Main contribution

The critical idea is as follows: there is a cost per unit of linguistic output, and this cost is greater for patients with non-fluent aphasia. For a rational agent, this increase leads to shorter messages. Critically, the informative parts of the message should be preserved and the redundant ones (like the function words and inflectional markers) should be omitted. Although economy of effort is unlikely to provide a unifying account of agrammatic output in all patients-the relevant population is too heterogeneous and the empirical landscape too complex for any single-factor explanation-we argue that the idea of agrammatic output as a rational behavior was dismissed prematurely and appears to provide a plausible explanation for a large subset of the reported cases of expressive aphasia.

Conclusions

The rational account of expressive agrammatism should be evaluated more carefully and systematically. On the basic research side, pursuing this hypothesis may reveal how the human mind and brain optimize communicative efficiency in the presence of production difficulties. And on the applied side, this construal of expressive agrammatism emphasizes the strengths of some patients to flexibly adapt utterances in order to communicate in spite of grammatical difficulties; and focusing on these strengths may be more effective than trying to "fix" their grammar.

Behavioral Signatures of Memory Resources for Language: Looking beyond the Lexicon/Grammar Divide

Although there is a broad consensus that both the procedural and declarative memory systems play a crucial role in language learning, use, and knowledge, the mapping between linguistic types and memory structures remains underspecified: by default, a dual-route mapping of language systems to memory systems is assumed, with declarative memory handling idiosyncratic lexical knowledge and procedural memory handling rule-governed knowledge of grammar. We experimentally contrast the processing of morphology (case and aspect), syntax (subordination), and lexical semantics (collocations) in a healthy L1 population of Polish, a language rich in form distinctions. We study the processing of these four types under two conditions: a single task condition in which the grammaticality of stimuli was judged and a concurrent task condition in which grammaticality judgments were combined with a digit span task. Dividing attention impedes access to declarative memory while leaving procedural memory unaffected and hence constitutes a test that dissociates which types of linguistic information each long-term memory construct subserves. Our findings confirm the existence of a distinction between lexicon and grammar as a generative, dual-route model would predict, but the distinction is graded, as usage-based models assume: the hypothesized grammar-lexicon opposition appears as a continuum on which grammatical phenomena can be placed as being more or less "ruly" or "idiosyncratic." However, usage-based models, too, need adjusting as not all types of linguistic knowledge are proceduralized to the same extent. This move away from a simple dichotomy fundamentally changes how we think about memory for language, and hence how we design and interpret behavioral and neuroimaging studies that probe into the nature of language cognition.

Robust Effects of Working Memory Demand during Naturalistic Language Comprehension in Language-Selective Cortex

To understand language, we must infer structured meanings from real-time auditory or visual signals. Researchers have long focused on word-by-word structure building in working memory as a mechanism that might enable this feat. However, some have argued that language processing does not typically involve rich word-by-word structure building, and/or that apparent working memory effects are underlyingly driven by surprisal (how predictable a word is in context). Consistent with this alternative, some recent behavioral studies of naturalistic language processing that control for surprisal have not shown clear working memory effects. In this fMRI study, we investigate a range of theory-driven predictors of word-by-word working memory demand during naturalistic language comprehension in humans of both sexes under rigorous surprisal controls. In addition, we address a related debate about whether the working memory mechanisms involved in language comprehension are language specialized or domain general. To do so, in each participant, we functionally localize (1) the language-selective network and (2) the "multiple-demand" network, which supports working memory across domains. Results show robust surprisal-independent effects of memory demand in the language network and no effect of memory demand in the multiple-demand network. Our findings thus support the view that language comprehension involves computationally demanding word-by-word structure building operations in working memory, in addition to any prediction-related mechanisms. Further, these memory operations appear to be primarily conducted by the same neural resources that store linguistic knowledge, with no evidence of involvement of brain regions known to support working memory across domains.SIGNIFICANCE STATEMENT This study uses fMRI to investigate signatures of working memory (WM) demand during naturalistic story listening, using a broad range of theoretically motivated estimates of WM demand. Results support a strong effect of WM demand in the brain that is distinct from effects of word predictability. Further, these WM demands register primarily in language-selective regions, rather than in "multiple-demand" regions that have previously been associated with WM in nonlinguistic domains. Our findings support a core role for WM in incremental language processing, using WM resources that are specialized for language.

Probabilistic atlas for the language network based on precision fMRI data from >800 individuals

Two analytic traditions characterize fMRI language research. One relies on averaging activations across individuals. This approach has limitations: because of inter-individual variability in the locations of language areas, any given voxel/vertex in a common brain space is part of the language network in some individuals but in others, may belong to a distinct network. An alternative approach relies on identifying language areas in each individual using a functional ‘localizer’. Because of its greater sensitivity, functional resolution, and interpretability, functional localization is gaining popularity, but it is not always feasible, and cannot be applied retroactively to past studies. To bridge these disjoint approaches, we created a probabilistic functional atlas using fMRI data for an extensively validated language localizer in 806 individuals. This atlas enables estimating the probability that any given location in a common space belongs to the language network, and thus can help interpret group-level activation peaks and lesion locations, or select voxels/electrodes for analysis. More meaningful comparisons of findings across studies should increase robustness and replicability in language research.

Measurement(s)	Brain activity measurement
Technology Type(s)	fMRI
Sample Characteristic - Organism	Homo sapiens

An investigation across 45 languages and 12 language families reveals a universal language network

To understand the architecture of human language, it is critical to examine diverse languages; however, most cognitive neuroscience research has focused on only a handful of primarily Indo-European languages. Here we report an investigation of the fronto-temporo-parietal language network across 45 languages and establish the robustness to cross-linguistic variation of its topography and key functional properties, including left-lateralization, strong functional integration among its brain regions and functional selectivity for language processing.

Differential Tracking of Linguistic vs. Mental State Content in Naturalistic Stimuli by Language and Theory of Mind (ToM) Brain Networks

Language and social cognition, especially the ability to reason about mental states, known as theory of mind (ToM), are deeply related in development and everyday use. However, whether these cognitive faculties rely on distinct, overlapping, or the same mechanisms remains debated. Some evidence suggests that, by adulthood, language and ToM draw on largely distinct-though plausibly interacting-cortical networks. However, the broad topography of these networks is similar, and some have emphasized the importance of social content / communicative intent in the linguistic signal for eliciting responses in the language areas. Here, we combine the power of individual-subject functional localization with the naturalistic-cognition inter-subject correlation approach to illuminate the language-ToM relationship. Using functional magnetic resonance imaging (fMRI), we recorded neural activity as participants (n = 43) listened to stories and dialogues with mental state content (+linguistic, +ToM), viewed silent animations and live action films with mental state content but no language (-linguistic, +ToM), or listened to an expository text (+linguistic, -ToM). The ToM network robustly tracked stimuli rich in mental state information regardless of whether mental states were conveyed linguistically or non-linguistically, while tracking a +linguistic / -ToM stimulus only weakly. In contrast, the language network tracked linguistic stimuli more strongly than (a) non-linguistic stimuli, and than (b) the ToM network, and showed reliable tracking even for the linguistic condition devoid of mental state content. These findings suggest that in spite of their indisputably close links, language and ToM dissociate robustly in their neural substrates-and thus plausibly cognitive mechanisms-including during the processing of rich naturalistic materials.

Effects of Structure and Meaning on Cortical Tracking of Linguistic Units in Naturalistic Speech

Recent research has established that cortical activity "tracks" the presentation rate of syntactic phrases in continuous speech, even though phrases are abstract units that do not have direct correlates in the acoustic signal. We investigated whether cortical tracking of phrase structures is modulated by the extent to which these structures compositionally determine meaning. To this end, we recorded electroencephalography (EEG) of 38 native speakers who listened to naturally spoken Dutch stimuli in different conditions, which parametrically modulated the degree to which syntactic structure and lexical semantics determine sentence meaning. Tracking was quantified through mutual information between the EEG data and either the speech envelopes or abstract annotations of syntax, all of which were filtered in the frequency band corresponding to the presentation rate of phrases (1.1-2.1 Hz). Overall, these mutual information analyses showed stronger tracking of phrases in regular sentences than in stimuli whose lexical-syntactic content is reduced, but no consistent differences in tracking between sentences and stimuli that contain a combination of syntactic structure and lexical content. While there were no effects of compositional meaning on the degree of phrase-structure tracking, analyses of event-related potentials elicited by sentence-final words did reveal meaning-induced differences between conditions. Our findings suggest that cortical tracking of structure in sentences indexes the internal generation of this structure, a process that is modulated by the properties of its input, but not by the compositional interpretation of its output.

The Wernicke conundrum revisited: evidence from connectome-based lesion-symptom mapping

Wernicke's area has been assumed since the 1800s to be the primary region supporting word and sentence comprehension. However, in 2015 and 2019, Mesulam and colleagues raised what they termed the 'Wernicke conundrum', noting widespread variability in the anatomical definition of this area and presenting data from primary progressive aphasia that challenged this classical assumption. To resolve the conundrum, they posited a 'double disconnection' hypothesis: that word and sentence comprehension deficits in stroke-based aphasia result from disconnection of anterior temporal and inferior frontal regions from other parts of the brain due to white matter damage, rather than dysfunction of Wernicke's area itself. To test this hypothesis, we performed lesion-deficit correlations, including connectome-based lesion-symptom mapping, in four large, partially overlapping groups of English-speaking chronic left hemisphere stroke survivors. After removing variance due to object recognition and associative semantic processing, the same middle and posterior temporal lobe regions were implicated in both word comprehension deficits and complex noncanonical sentence comprehension deficits. Connectome lesion-symptom mapping revealed similar temporal-occipital white matter disconnections for impaired word and noncanonical sentence comprehension, including the temporal pole. We found an additional significant temporal-parietal disconnection for noncanonical sentence comprehension deficits, which may indicate a role for phonological working memory in processing complex syntax, but no significant frontal disconnections. Moreover, damage to these middle-posterior temporal lobe regions was associated with both word and noncanonical sentence comprehension deficits even when accounting for variance due to the strongest anterior temporal and inferior frontal white matter disconnections, respectively. Our results largely agree with the classical notion that Wernicke's area, defined here as middle superior temporal gyrus and middle-posterior superior temporal sulcus, supports both word and sentence comprehension, suggest a supporting role for temporal pole in both word and sentence comprehension, and speak against the hypothesis that comprehension deficits in Wernicke's aphasia result from double disconnection.

Gesture is the primary modality for language creation

How language began is one of the oldest questions in science, but theories remain speculative due to a lack of direct evidence. Here, we report two experiments that generate empirical evidence to inform gesture-first and vocal-first theories of language origin; in each, we tested modern humans' ability to communicate a range of meanings (995 distinct words) using either gesture or non-linguistic vocalization. Experiment 1 is a cross-cultural study, with signal Producers sampled from Australia (n = 30, M_age = 32.63, s.d. = 12.42) and Vanuatu (n = 30, M_age = 32.40, s.d. = 11.76). Experiment 2 is a cross-experiential study in which Producers were either sighted (n = 10, M_age = 39.60, s.d. = 11.18) or severely vision-impaired (n = 10, M_age = 39.40, s.d. = 10.37). A group of undergraduate student Interpreters guessed the meaning of the signals created by the Producers (n = 140). Communication success was substantially higher in the gesture modality than the vocal modality (twice as high overall; 61.17% versus 29.04% success). This was true within cultures, across cultures and even for the signals produced by severely vision-impaired participants. The success of gesture is attributed in part to its greater universality (i.e. similarity in form across different Producers). Our results support the hypothesis that gesture is the primary modality for language creation.

Models of Language and Multiword Expressions

Traditional accounts of language postulate two basic components: words stored in a lexicon, and rules that govern how they can be combined into meaningful sentences, a grammar. But, although this words-and-rules framework has proven itself to be useful in natural language processing and cognitive science, it has also shown important shortcomings when faced with actual language use. In this article, we review evidence from language acquisition, sentence processing, and computational modeling that shows how multiword expressions such as idioms, collocations, and other meaningful and common units that comprise more than one word play a key role in the organization of our linguistic knowledge. Importantly, multiword expressions straddle the line between lexicon and grammar, calling into question how useful this distinction is as a foundation for our understanding of language. Nonetheless, finding a replacement for the foundational role the words-and-rules approach has played in our theories is not straightforward. Thus, the second part of our article reviews and synthesizes the diverse approaches that have attempted to account for the central role of multiword expressions in language representation, acquisition, and processing.

Brains and algorithms partially converge in natural language processing

Deep learning algorithms trained to predict masked words from large amount of text have recently been shown to generate activations similar to those of the human brain. However, what drives this similarity remains currently unknown. Here, we systematically compare a variety of deep language models to identify the computational principles that lead them to generate brain-like representations of sentences. Specifically, we analyze the brain responses to 400 isolated sentences in a large cohort of 102 subjects, each recorded for two hours with functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). We then test where and when each of these algorithms maps onto the brain responses. Finally, we estimate how the architecture, training, and performance of these models independently account for the generation of brain-like representations. Our analyses reveal two main findings. First, the similarity between the algorithms and the brain primarily depends on their ability to predict words from context. Second, this similarity reveals the rise and maintenance of perceptual, lexical, and compositional representations within each cortical region. Overall, this study shows that modern language algorithms partially converge towards brain-like solutions, and thus delineates a promising path to unravel the foundations of natural language processing.

Charlotte Caucheteux and Jean-Rémi King examine the ability of transformer neural networks trained on word prediction tasks to fit representations in the human brain measured with fMRI and MEG. Their results provide further insight into the workings of transformer language models and their relevance to brain responses.

Conceptual Combination in the LATL With and Without Syntactic Composition

The relationship among syntactic, semantic, and conceptual processes in language comprehension is a central question to the neurobiology of language. Several studies have suggested that conceptual combination in particular can be localized to the left anterior temporal lobe (LATL), while syntactic processes are more often associated with the posterior temporal lobe or inferior frontal gyrus. However, LATL activity can also correlate with syntactic computations, particularly in narrative comprehension. Here we investigated the degree to which LATL conceptual combination is dependent on syntax, specifically asking whether rapid (∼200 ms) magnetoencephalography effects of conceptual combination in the LATL can occur in the absence of licit syntactic phrase closure and in the absence of a semantically plausible output for the composition. We find that such effects do occur: LATL effects of conceptual combination were observed even when there was no syntactic phrase closure or plausible meaning. But syntactic closure did have an additive effect such that LATL signals were the highest for expressions that composed both conceptually and syntactically. Our findings conform to an account in which LATL conceptual composition is influenced by local syntactic composition but is also able to operate without it.