A functional dissociation between language and multiple-demand systems revealed in patterns of BOLD signal fluctuations

The visual cortex in the blind but not the auditory cortex in the deaf becomes multiple-demand regions

The fate of deprived sensory cortices (visual regions in the blind and auditory regions in the deaf) exemplifies the extent to which experience can change brain regions. These regions are frequently seen to activate during tasks involving other sensory modalities, leading many authors to infer that these regions have started to process sensory information of other modalities. However, such observations can also imply that these regions are now activating in response to any task event, regardless of the sensory modality. Activating in response to task events, irrespective of the sensory modality involved, is a feature of the multiple-demands (MD) network. This is a set of regions within the frontal and parietal cortices that activate in response to any kind of control demand. Thus, demands as diverse as attention, perceptual difficulty, rule-switching, updating working memory, inhibiting responses, decision-making and difficult arithmetic all activate the same set of regions that are thought to instantiate domain-general cognitive control and underpin fluid intelligence. We investigated whether deprived sensory cortices, or foci within them, become part of the MD network. We tested whether the same foci within the visual regions of the blind and auditory regions of the deaf activated in response to different control demands. We found that control demands related to updating auditory working memory, difficult tactile decisions, time-duration judgments and sensorimotor speed all activated the entire bilateral occipital regions in the blind but not in the sighted. These occipital regions in the blind were the only regions outside the canonical frontoparietal MD regions to show such activation in response to multiple control demands. Furthermore, compared with the sighted, these occipital regions in the blind had higher functional connectivity with frontoparietal MD regions. Early deaf, in contrast, did not activate their auditory regions in response to different control demands, showing that auditory regions do not become MD regions in the deaf. We suggest that visual regions in the blind do not take a new sensory role but become part of the MD network, and this is not a response of all deprived sensory cortices but a feature unique to the visual regions.

Cortical cognitive processing during reading captured using functional-near infrared spectroscopy

Neuroimaging studies using functional magnetic resonance imaging (fMRI) have provided unparalleled insights into the fundamental neural mechanisms underlying human cognitive processing, such as high-level linguistic processes during reading. Here, we build upon this prior work to capture sentence reading comprehension outside the MRI scanner using functional near infra-red spectroscopy (fNIRS) in a large sample of participants (n = 82). We observed increased task-related hemodynamic responses in prefrontal and temporal cortical regions during sentence-level reading relative to the control condition (a list of non-words), replicating prior fMRI work on cortical recruitment associated with high-level linguistic processing during reading comprehension. These results lay the groundwork towards developing adaptive systems to support novice readers and language learners by targeting the underlying cognitive processes. This work also contributes to bridging the gap between laboratory findings and more real-world applications in the realm of cognitive neuroscience.

Language in Brains, Minds, and Machines

It has long been argued that only humans could produce and understand language. But now, for the first time, artificial language models (LMs) achieve this feat. Here we survey the new purchase LMs are providing on the question of how language is implemented in the brain. We discuss why, a priori, LMs might be expected to share similarities with the human language system. We then summarize evidence that LMs represent linguistic information similarly enough to humans to enable relatively accurate brain encoding and decoding during language processing. Finally, we examine which LM properties-their architecture, task performance, or training-are critical for capturing human neural responses to language and review studies using LMs as in silico model organisms for testing hypotheses about language. These ongoing investigations bring us closer to understanding the representations and processes that underlie our ability to comprehend sentences and express thoughts in language.

Longitudinal associations between language network characteristics in the infant brain and school-age reading abilities are mediated by early-developing phonological skills

Reading acquisition is a prolonged learning process relying on language development starting in utero. Behavioral longitudinal studies reveal prospective associations between infant language abilities and preschool/kindergarten phonological development that relates to subsequent reading performance. While recent pediatric neuroimaging work has begun to characterize the neural network underlying language development in infants, how this neural network scaffolds long-term language and reading acquisition remains unknown. We addressed this question in a 7-year longitudinal study from infancy to school-age. Seventy-six infants completed resting-state fMRI scanning, and underwent standardized language assessments in kindergarten. Of this larger cohort, forty-one were further assessed on their emergent word reading abilities after receiving formal reading instructions. Hierarchical clustering analyses identified a modular infant language network in which functional connectivity (FC) of the inferior frontal module prospectively correlated with kindergarten-age phonological skills and emergent word reading abilities. These correlations were obtained when controlling for infant age at scan, nonverbal IQ and parental education. Furthermore, kindergarten-age phonological skills mediated the relationship between infant FC and school-age reading abilities, implying a critical mid-way milestone for long-term reading development from infancy. Overall, our findings illuminate the neurobiological mechanisms by which infant language capacities could scaffold long-term reading acquisition.

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward-presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing.

Functional dissociation of the language network and other cognition in early childhood

Is language distinct from other cognition during development? Does neural machinery for language emerge from general-purpose neural mechanisms, becoming tuned for language after years of experience and maturation? Answering these questions will shed light on the origins of domain-specificity in the brain. We address these questions using precision fMRI, scanning young children (35 months to 9 years of age) on an auditory language localizer, spatial working memory localizer (engaging the domain-general multiple demand [MD] network), and a resting-state scan. We create subject-specific functional regions of interest for each network and examine their selectivity, specificity, and functional connectivity. We find young children show domain-specific, left-lateralized language activation, and that the language network is not responsive to domain-general cognitive load. Additionally, the cortically adjacent MD network is selective to cognitive load, but not to language. These networks show higher within versus between-network functional connectivity. This connectivity is stable across ages (examined cross-sectionally and longitudinally), whereas language responses increase with age and across time within subject, reflecting a domain-specific developmental change. Overall, we provide evidence for a double dissociation of the language and MD network throughout development, in both their function and connectivity. These findings suggest that domain-specificity, even for uniquely human cognition like language, develops early and distinctly from mechanisms that presumably support other human cognition.

Precision fMRI reveals that the language network exhibits adult-like left-hemispheric lateralization by 4 years of age

Left hemisphere damage in adulthood often leads to linguistic deficits, but many cases of early damage leave linguistic processing preserved, and a functional language system can develop in the right hemisphere. To explain this early apparent equipotentiality of the two hemispheres for language, some have proposed that the language system is bilateral during early development and only becomes left-lateralized with age. We examined language lateralization using functional magnetic resonance imaging with two large pediatric cohorts (total n=273 children ages 4-16; n=107 adults). Strong, adult-level left-hemispheric lateralization (in activation volume and response magnitude) was evident by age 4. Thus, although the right hemisphere can take over language function in some cases of early brain damage, and although some features of the language system do show protracted development (magnitude of language response and strength of inter-regional correlations in the language network), the left-hemisphere bias for language is robustly present by 4 years of age. These results call for alternative accounts of early equipotentiality of the two hemispheres for language.

Longitudinal associations between language network characteristics in the infant brain and school-age reading abilities are mediated by early-developing phonological skills

Reading acquisition is a prolonged learning process relying on language development starting in utero. Behavioral longitudinal studies reveal prospective associations between infant language abilities and preschool/kindergarten phonological development that relates to subsequent reading performance. While recent pediatric neuroimaging work has begun to characterize the neural network underlying language development in infants, how this neural network scaffolds long-term language and reading acquisition remains unknown. We addressed this question in a 7-year longitudinal study from infancy to school-age. Seventy-six infants completed resting-state fMRI scanning, and underwent standardized language assessments in kindergarten. Of this larger cohort, forty-one were further assessed on their emergent word reading abilities after receiving formal reading instructions. Hierarchical clustering analyses identified a modular infant language network in which functional connectivity (FC) of the inferior frontal module prospectively correlated with kindergarten-age phonological skills and emergent word reading abilities. These correlations were obtained when controlling for infant age at scan, nonverbal IQ and parental education. Furthermore, kindergarten-age phonological skills mediated the relationship between infant FC and school-age reading abilities, implying a critical mid-way milestone for long-term reading development from infancy. Overall, our findings illuminate the neurobiological mechanisms by which infant language capacities could scaffold long-term reading acquisition.

Tracking Components of Bilingual Language Control in Speech Production: An fMRI Study Using Functional Localizers

When bilingual speakers switch back to speaking in their native language (L1) after having used their second language (L2), they often experience difficulty in retrieving words in their L1. This phenomenon is referred to as the L2 after-effect. We used the L2 after-effect as a lens to explore the neural bases of bilingual language control mechanisms. Our goal was twofold: first, to explore whether bilingual language control draws on domain-general or language-specific mechanisms; second, to investigate the precise mechanism(s) that drive the L2 after-effect. We used a precision fMRI approach based on functional localizers to measure the extent to which the brain activity that reflects the L2 after-effect overlaps with the language network (Fedorenko et al., 2010) and the domain-general multiple demand network (Duncan, 2010), as well as three task-specific networks that tap into interference resolution, lexical retrieval, and articulation. Forty-two Polish-English bilinguals participated in the study. Our results show that the L2 after-effect reflects increased engagement of domain-general but not language-specific resources. Furthermore, contrary to previously proposed interpretations, we did not find evidence that the effect reflects increased difficulty related to lexical access, articulation, and the resolution of lexical interference. We propose that difficulty of speech production in the picture naming paradigm-manifested as the L2 after-effect-reflects interference at a nonlinguistic level of task schemas or a general increase of cognitive control engagement during speech production in L1 after L2.

Distributed Sensitivity to Syntax and Semantics throughout the Language Network

Human language is expressive because it is compositional: The meaning of a sentence (semantics) can be inferred from its structure (syntax). It is commonly believed that language syntax and semantics are processed by distinct brain regions. Here, we revisit this claim using precision fMRI methods to capture separation or overlap of function in the brains of individual participants. Contrary to prior claims, we find distributed sensitivity to both syntax and semantics throughout a broad frontotemporal brain network. Our results join a growing body of evidence for an integrated network for language in the human brain within which internal specialization is primarily a matter of degree rather than kind, in contrast with influential proposals that advocate distinct specialization of different brain areas for different types of linguistic functions.

Organization of the human cerebral cortex estimated within individuals: networks, global topography, and function

The cerebral cortex is populated by specialized regions that are organized into networks. Here we estimated networks from functional MRI (fMRI) data in intensively sampled participants. The procedure was developed in two participants (scanned 31 times) and then prospectively applied to 15 participants (scanned 8-11 times). Analysis of the networks revealed a global organization. Locally organized first-order sensory and motor networks were surrounded by spatially adjacent second-order networks that linked to distant regions. Third-order networks possessed regions distributed widely throughout association cortex. Regions of distinct third-order networks displayed side-by-side juxtapositions with a pattern that repeated across multiple cortical zones. We refer to these as supra-areal association megaclusters (SAAMs). Within each SAAM, two candidate control regions were adjacent to three separate domain-specialized regions. Response properties were explored with task data. The somatomotor and visual networks responded to body movements and visual stimulation, respectively. Second-order networks responded to transients in an oddball detection task, consistent with a role in orienting to salient events. The third-order networks, including distinct regions within each SAAM, showed two levels of functional specialization. Regions linked to candidate control networks responded to working memory load across multiple stimulus domains. The remaining regions dissociated across language, social, and spatial/episodic processing domains. These results suggest that progressively higher-order networks nest outward from primary sensory and motor cortices. Within the apex zones of association cortex, there is specialization that repeatedly divides domain-flexible from domain-specialized regions. We discuss implications of these findings, including how repeating organizational motifs may emerge during development.NEW & NOTEWORTHY The organization of cerebral networks was estimated within individuals with intensive, repeat sampling of fMRI data. A hierarchical organization emerged in each individual that delineated first-, second-, and third-order cortical networks. Regions of distinct third-order association networks consistently exhibited side-by-side juxtapositions that repeated across multiple cortical zones, with clear and robust functional specialization among the embedded regions.

Modeling the association between functional connectivity and lateralization with the activity flow framework

The human brain is localized and distributed. On the one hand, each cognitive function tends to involve one hemisphere more than the other, also known as the principle of lateralization. On the other hand, interactions among brain regions in the form of functional connectivity (FC) are indispensable for intact function. Recent years have seen growing interest in the association between lateralization and FC. However, FC metrics vary from spurious correlation to causal associations. If lateralization manifests local processing and causal network interactions, more causally valid FC metrics should predict lateralization index (LI) better than FC based on simple correlations. The present study directly investigates this hypothesis within the activity flow framework to compare the association between lateralization and four brain connectivity metrics: correlation-based FC, multiple-regression FC, partial-correlation FC, and combinedFC. We propose two modeling approaches: the one-step approach, which models the relationship between LI and FC directly, and the two-step approach, which predicts the brain activation and calculates the LI. Our results indicated that multiple-regression FC, partial-correlation FC, and combinedFC could significantly improve the model prediction compared to correlation-based FC, which was consistent in a spatial working memory task (typically right-lateralized) and a language task (typically left-lateralized). The one-step and two-step approach yielded similar conclusions. In addition, the finding was replicated in a clinical sample of schizophrenia (SZ), bipolar disorder (BP), and attention deficit hyperactivity disorder (ADHD). The present study suggests that the causal interactions among brain regions help shape the lateralization pattern.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

Lexical-Semantic Content, Not Syntactic Structure, Is the Main Contributor to ANN-Brain Similarity of fMRI Responses in the Language Network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI data set of responses to n = 627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we (i) perturbed sentences' word order, (ii) removed different subsets of words, or (iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical-semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust as to whether the mapping model is trained on intact or perturbed stimuli and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

High-level language brain regions process sublexical regularities

A network of left frontal and temporal brain regions supports language processing. This "core" language network stores our knowledge of words and constructions as well as constraints on how those combine to form sentences. However, our linguistic knowledge additionally includes information about phonemes and how they combine to form phonemic clusters, syllables, and words. Are phoneme combinatorics also represented in these language regions? Across five functional magnetic resonance imaging experiments, we investigated the sensitivity of high-level language processing brain regions to sublexical linguistic regularities by examining responses to diverse nonwords-sequences of phonemes that do not constitute real words (e.g. punes, silory, flope). We establish robust responses in the language network to visually (experiment 1a, n = 605) and auditorily (experiments 1b, n = 12, and 1c, n = 13) presented nonwords. In experiment 2 (n = 16), we find stronger responses to nonwords that are more well-formed, i.e. obey the phoneme-combinatorial constraints of English. Finally, in experiment 3 (n = 14), we provide suggestive evidence that the responses in experiments 1 and 2 are not due to the activation of real words that share some phonology with the nonwords. The results suggest that sublexical regularities are stored and processed within the same fronto-temporal network that supports lexical and syntactic processes.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of the brain response associated with each sentence. We then use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress the activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also non-invasively control neural activity in higher-level cortical areas, such as the language network.

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots-individuals who speak five or more languages-process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants' high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language network's response magnitude scales with the degree of engagement of linguistic computations (e.g. related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages coexist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

Basis of executive functions in fine-grained architecture of cortical and subcortical human brain networks

Theoretical models suggest that executive functions rely on both domain-general and domain-specific processes. Supporting this view, prior brain imaging studies have revealed that executive activations converge and diverge within broadly characterized brain networks. However, the lack of precise anatomical mappings has impeded our understanding of the interplay between domain-general and domain-specific processes. To address this challenge, we used the high-resolution multimodal magnetic resonance imaging approach of the Human Connectome Project to scan participants performing 3 canonical executive tasks: n-back, rule switching, and stop signal. The results reveal that, at the individual level, different executive activations converge within 9 domain-general territories distributed in frontal, parietal, and temporal cortices. Each task exhibits a unique topography characterized by finely detailed activation gradients within domain-general territory shifted toward adjacent resting-state networks; n-back activations shift toward the default mode, rule switching toward dorsal attention, and stop signal toward cingulo-opercular networks. Importantly, the strongest activations arise at multimodal neurobiological definitions of network borders. Matching results are seen in circumscribed regions of the caudate nucleus, thalamus, and cerebellum. The shifting peaks of local gradients at the intersection of task-specific networks provide a novel mechanistic insight into how partially-specialized networks interact with neighboring domain-general territories to generate distinct executive functions.

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots-individuals who speak five or more languages-process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher-proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants' high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language network's response magnitude scales with the degree of engagement of linguistic computations (e.g., related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages co-exist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

Efficient functional localization of language regions in the brain

Important recent advances in the cognitive neuroscience of language have been made using functional localizers to demarcate language-selective regions in individual brains. Although single-subject localizers offer insights that are unavailable in classic group analyses, they require additional scan time that imposes costs on investigators and participants. In particular, the unique practical challenges of scanning children and other special populations has led to less adoption of localizers for neuroimaging research with these theoretically and clinically important groups. Here, we examined how measurements of the spatial extent and functional response profiles of language regions are affected by the duration of an auditory language localizer. We compared how parametrically smaller amounts of data collected from one scanning session affected (i) consistency of group-level whole-brain parcellations, (ii) functional selectivity of subject-level activation in individually defined functional regions of interest (fROIs), (iii) sensitivity and specificity of subject-level whole-brain and fROI activation, and (iv) test-retest reliability of subject-level whole-brain and fROI activation. For many of these metrics, the localizer duration could be reduced by 50-75% while preserving the stability and reliability of both the spatial extent and functional response profiles of language areas. These results indicate that, for most measures relevant to cognitive neuroimaging studies, the brain's language network can be localized just as effectively with 3.5 min of scan time as it can with 12 min. Minimizing the time required to reliably localize the brain's language network allows more effective localizer use in situations where each minute of scan time is particularly precious.

Left-Hemisphere Cortical Language Regions Respond Equally to Observed Dialogue and Monologue

Much of the language we encounter in our everyday lives comes in the form of conversation, yet the majority of research on the neural basis of language comprehension has used input from only one speaker at a time. Twenty adults were scanned while passively observing audiovisual conversations using functional magnetic resonance imaging. In a block-design task, participants watched 20 s videos of puppets speaking either to another puppet (the dialogue condition) or directly to the viewer (the monologue condition), while the audio was either comprehensible (played forward) or incomprehensible (played backward). Individually functionally localized left-hemisphere language regions responded more to comprehensible than incomprehensible speech but did not respond differently to dialogue than monologue. In a second task, participants watched videos (1-3 min each) of two puppets conversing with each other, in which one puppet was comprehensible while the other's speech was reversed. All participants saw the same visual input but were randomly assigned which character's speech was comprehensible. In left-hemisphere cortical language regions, the time course of activity was correlated only among participants who heard the same character speaking comprehensibly, despite identical visual input across all participants. For comparison, some individually localized theory of mind regions and right-hemisphere homologues of language regions responded more to dialogue than monologue in the first task, and in the second task, activity in some regions was correlated across all participants regardless of which character was speaking comprehensibly. Together, these results suggest that canonical left-hemisphere cortical language regions are not sensitive to differences between observed dialogue and monologue.

Leukoaraiosis Is Not Associated With Recovery From Aphasia in the First Year After Stroke

After a stroke, individuals with aphasia often recover to a certain extent over time. This recovery process may be dependent on the health of surviving brain regions. Leukoaraiosis (white matter hyperintensities on MRI reflecting cerebral small vessel disease) is one indication of compromised brain health and is associated with cognitive and motor impairment. Previous studies have suggested that leukoaraiosis may be a clinically relevant predictor of aphasia outcomes and recovery, although findings have been inconsistent. We investigated the relationship between leukoaraiosis and aphasia in the first year after stroke. We recruited 267 patients with acute left hemispheric stroke and coincident fluid attenuated inversion recovery MRI. Patients were evaluated for aphasia within 5 days of stroke, and 174 patients presented with aphasia acutely. Of these, 84 patients were evaluated at ∼3 months post-stroke or later to assess longer-term speech and language outcomes. Multivariable regression models were fit to the data to identify any relationships between leukoaraiosis and initial aphasia severity, extent of recovery, or longer-term aphasia severity. We found that leukoaraiosis was present to varying degrees in 90% of patients. However, leukoaraiosis did not predict initial aphasia severity, aphasia recovery, or longer-term aphasia severity. The lack of any relationship between leukoaraiosis severity and aphasia recovery may reflect the anatomical distribution of cerebral small vessel disease, which is largely medial to the white matter pathways that are critical for speech and language function.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

Reading Braille by Touch Recruits Posterior Parietal Cortex

Blind readers use a tactile reading system consisting of raised dot arrays: braille/⠃⠗⠇. How do human brains implement reading by touch? The current study looked for signatures of reading-specific orthographic processes in braille, separate from low-level somatosensory responses and semantic processes. Of specific interest were responses in posterior parietal cortices (PPCs), because of their role in high-level tactile perception. Congenitally blind, proficient braille readers read real words and pseudowords by touch while undergoing fMRI. We leveraged the system of contractions in English braille, where one braille cell can represent multiple English print letters (e.g., "ing" ⠬, "one" ⠐⠕), making it possible to separate physical and orthographic word length. All words in the study consisted of four braille cells, but their corresponding Roman letter spellings varied from four to seven letters (e.g., "con-c-er-t" ⠒⠉⠻⠞. contracted: four cells; uncontracted: seven letters). We found that the bilateral supramarginal gyrus in the PPC increased its activity as the uncontracted word length increased. By contrast, in the hand region of primary somatosensory cortex (S1), activity increased as a function of a low-level somatosensory feature: dot-number per word. The PPC also showed greater response to pseudowords than real words and distinguished between real and pseudowords in multivariate-pattern analysis. Parieto-occipital, early visual and ventral occipito-temporal, as well as prefrontal cortices also showed sensitivity to the real-versus-pseudoword distinction. We conclude that PPC is involved in orthographic processing for braille, that is, braille character and word recognition, possibly because of braille's tactile modality.

The language network is not engaged in object categorization

The relationship between language and thought is the subject of long-standing debate. One claim states that language facilitates categorization of objects based on a certain feature (e.g. color) through the use of category labels that reduce interference from other, irrelevant features. Therefore, language impairment is expected to affect categorization of items grouped by a single feature (low-dimensional categories, e.g. "Yellow Things") more than categorization of items that share many features (high-dimensional categories, e.g. "Animals"). To test this account, we conducted two behavioral studies with individuals with aphasia and an fMRI experiment with healthy adults. The aphasia studies showed that selective low-dimensional categorization impairment was present in some, but not all, individuals with severe anomia and was not characteristic of aphasia in general. fMRI results revealed little activity in language-responsive brain regions during both low- and high-dimensional categorization; instead, categorization recruited the domain-general multiple-demand network (involved in wide-ranging cognitive tasks). Combined, results demonstrate that the language system is not implicated in object categorization. Instead, selective low-dimensional categorization impairment might be caused by damage to brain regions responsible for cognitive control. Our work adds to the growing evidence of the dissociation between the language system and many cognitive tasks in adults.

Constructed languages are processed by the same brain mechanisms as natural languages

What constitutes a language? Natural languages share some features with other domains: from math, to music, to gesture. However, the brain mechanisms that process linguistic input are highly specialized, showing little or no response to diverse non-linguistic tasks. Here, we examine constructed languages (conlangs) to ask whether they draw on the same neural mechanisms as natural languages, or whether they instead pattern with domains like math and logic. Using individual-subject fMRI analyses, we show that understanding conlangs recruits the same brain areas as natural language comprehension. This result holds for Esperanto (n=19 speakers)- created to resemble natural languages-and fictional conlangs (Klingon (n=10), Na'vi (n=9), High Valyrian (n=3), and Dothraki (n=3)), created to differ from natural languages, and suggests that conlangs and natural languages share critical features and that the notable differences between conlangs and natural language are not consequential for the cognitive and neural mechanisms that they engage.

The human language system, including its inferior frontal component in "Broca's area," does not support music perception

Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm

Introduction

Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level.

Methods

This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up.

Results

Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated.

Discussion

Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.

No evidence of theory of mind reasoning in the human language network

Language comprehension and the ability to infer others' thoughts (theory of mind [ToM]) are interrelated during development and language use. However, neural evidence that bears on the relationship between language and ToM mechanisms is mixed. Although robust dissociations have been reported in brain disorders, brain activations for contrasts that target language and ToM bear similarities, and some have reported overlap. We take another look at the language-ToM relationship by evaluating the response of the language network, as measured with fMRI, to verbal and nonverbal ToM across 151 participants. Individual-participant analyses reveal that all core language regions respond more strongly when participants read vignettes about false beliefs compared to the control vignettes. However, we show that these differences are largely due to linguistic confounds, and no such effects appear in a nonverbal ToM task. These results argue against cognitive and neural overlap between language processing and ToM. In exploratory analyses, we find responses to social processing in the "periphery" of the language network-right-hemisphere homotopes of core language areas and areas in bilateral angular gyri-but these responses are not selectively ToM-related and may reflect general visual semantic processing.

Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI dataset of responses to n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we i) perturbed sentences' word order, ii) removed different subsets of words, or iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust to whether the mapping model is trained on intact or perturbed stimuli, and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

Non-literal language processing is jointly supported by the language and theory of mind networks: Evidence from a novel meta-analytic fMRI approach

Going beyond the literal meaning of language is key to communicative success. However, the mechanisms that support non-literal inferences remain debated. Using a novel meta-analytic approach, we evaluate the contribution of linguistic, social-cognitive, and executive mechanisms to non-literal interpretation. We identified 74 fMRI experiments (n = 1,430 participants) from 2001 to 2021 that contrasted non-literal language comprehension with a literal control condition, spanning ten phenomena (e.g., metaphor, irony, indirect speech). Applying the activation likelihood estimation approach to the 825 activation peaks yielded six left-lateralized clusters. We then evaluated the locations of both the individual-study peaks and the clusters against probabilistic functional atlases (cf. anatomical locations, as is typically done) for three candidate brain networks-the language-selective network (Fedorenko, Behr, & Kanwisher, 2011), which supports language processing, the Theory of Mind (ToM) network (Saxe & Kanwisher, 2003), which supports social inferences, and the domain-general Multiple-Demand (MD) network (Duncan, 2010), which supports executive control. These atlases were created by overlaying individual activation maps of participants who performed robust and extensively validated 'localizer' tasks that selectively target each network in question (n = 806 for language; n = 198 for ToM; n = 691 for MD). We found that both the individual-study peaks and the ALE clusters fell primarily within the language network and the ToM network. These results suggest that non-literal processing is supported by both i) mechanisms that process literal linguistic meaning, and ii) mechanisms that support general social inference. They thus undermine a strong divide between literal and non-literal aspects of language and challenge the claim that non-literal processing requires additional executive resources.

Spatiotemporally distributed frontotemporal networks for sentence reading

Reading a sentence entails integrating the meanings of individual words to infer more complex, higher-order meaning. This highly rapid and complex human behavior is known to engage the inferior frontal gyrus (IFG) and middle temporal gyrus (MTG) in the language-dominant hemisphere, yet whether there are distinct contributions of these regions to sentence reading is still unclear. To probe these neural spatiotemporal dynamics, we used direct intracranial recordings to measure neural activity while reading sentences, meaning-deficient Jabberwocky sentences, and lists of words or pseudowords. We isolated two functionally and spatiotemporally distinct frontotemporal networks, each sensitive to distinct aspects of word and sentence composition. The first distributed network engages the IFG and MTG, with IFG activity preceding MTG. Activity in this network ramps up over the duration of a sentence and is reduced or absent during Jabberwocky and word lists, implying its role in the derivation of sentence-level meaning. The second network engages the superior temporal gyrus and the IFG, with temporal responses leading those in frontal lobe, and shows greater activation for each word in a list than those in sentences, suggesting that sentential context enables greater efficiency in the lexical and/or phonological processing of individual words. These adjacent, yet spatiotemporally dissociable neural mechanisms for word- and sentence-level processes shed light on the richly layered semantic networks that enable us to fluently read. These results imply distributed, dynamic computation across the frontotemporal language network rather than a clear dichotomy between the contributions of frontal and temporal structures.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

Functional identification of language-responsive channels in individual participants in MEG investigations

Making meaningful inferences about the functional architecture of the language system requires the ability to refer to the same neural units across individuals and studies. Traditional brain imaging approaches align and average brains together in a common space. However, lateral frontal and temporal cortex, where the language system resides, is characterized by high structural and functional inter-individual variability. This variability reduces the sensitivity and functional resolution of group-averaging analyses. This problem is compounded by the fact that language areas often lay in close proximity to regions of other large-scale networks with different functional profiles. A solution inspired by other fields of cognitive neuroscience (e.g., vision) is to identify language areas functionally in each individual brain using a 'localizer' task (e.g., a language comprehension task). This approach has proven productive in fMRI, yielding a number of discoveries about the language system, and has been successfully extended to intracranial recording investigations. Here, we apply this approach to MEG. Across two experiments (one in Dutch speakers, n=19; one in English speakers, n=23), we examined neural responses to the processing of sentences and a control condition (nonword sequences). We demonstrated that the neural response to language is spatially consistent at the individual level. The language-responsive sensors of interest were, as expected, less responsive to the nonwords condition. Clear inter-individual differences were present in the topography of the neural response to language, leading to greater sensitivity when the data were analyzed at the individual level compared to the group level. Thus, as in fMRI, functional localization yields benefits in MEG and thus opens the door to probing fine-grained distinctions in space and time in future MEG investigations of language processing.

Individual Variability in Performance Reflects Selectivity of the Multiple Demand Network among Children and Adults

Executive function (EF) is essential for humans to effectively engage in cognitively demanding tasks. In adults, EF is subserved by frontoparietal regions in the multiple demand (MD) network, which respond to various cognitively demanding tasks. However, children initially show poor EF and prolonged development. Do children recruit the same network as adults? Is it functionally and connectionally distinct from adjacent language cortex, as in adults? And is this activation or connectivity dependent on age or ability? We examine task-dependent (spatial working memory and passive language tasks) and resting state functional data in 44 adults (18-38 years, 68% female) and 37 children (4-12 years, 35% female). Subject-specific functional ROIs (ss-fROIs) show bilateral MD network activation in children. In both children and adults, these MD ss-fROIs are not recruited for linguistic processing and are connectionally distinct from language ss-fROIs. While MD activation was lower in children than in adults (even in motion- and performance-matched groups), both showed increasing MD activation with better performance, especially in right hemisphere ss-fROIs. We observe this relationship even when controlling for age, cross-sectionally and in a small longitudinal sample of children. These data suggest that the MD network is selective to cognitive demand in children, is distinct from adjacent language cortex, and increases in selectivity as performance improves. These findings show that neural structures subserving domain-general EF emerge early and are sensitive to ability even in children. This research advances understanding of how high-level human cognition emerges and could inform interventions targeting cognitive control.SIGNIFICANCE STATEMENT This study provides evidence that young children already show differentiated brain network organization between regions that process cognitive demand and language. These data support the hypothesis that children recruit a similar network as adults to process cognitive demand; and despite immature characteristics, children's selectivity looks more adult-like as their executive function ability increases. Mapping early stages of network organization furthers our understanding of the functional architecture underlying domain-general executive function. Determining typical variability underlying cognitive processing across developmental periods helps establish a threshold for executive dysfunction. Early markers of dysfunction are necessary for effective early identification, prevention, and intervention efforts for individuals struggling with deficits in processing cognitive demand.

Causal Contributions of the Domain-General (Multiple Demand) and the Language-Selective Brain Networks to Perceptual and Semantic Challenges in Speech Comprehension

Listening to spoken language engages domain-general multiple demand (MD; frontoparietal) regions of the human brain, in addition to domain-selective (frontotemporal) language regions, particularly when comprehension is challenging. However, there is limited evidence that the MD network makes a functional contribution to core aspects of understanding language. In a behavioural study of volunteers (n = 19) with chronic brain lesions, but without aphasia, we assessed the causal role of these networks in perceiving, comprehending, and adapting to spoken sentences made more challenging by acoustic-degradation or lexico-semantic ambiguity. We measured perception of and adaptation to acoustically degraded (noise-vocoded) sentences with a word report task before and after training. Participants with greater damage to MD but not language regions required more vocoder channels to achieve 50% word report, indicating impaired perception. Perception improved following training, reflecting adaptation to acoustic degradation, but adaptation was unrelated to lesion location or extent. Comprehension of spoken sentences with semantically ambiguous words was measured with a sentence coherence judgement task. Accuracy was high and unaffected by lesion location or extent. Adaptation to semantic ambiguity was measured in a subsequent word association task, which showed that availability of lower-frequency meanings of ambiguous words increased following their comprehension (word-meaning priming). Word-meaning priming was reduced for participants with greater damage to language but not MD regions. Language and MD networks make dissociable contributions to challenging speech comprehension: Using recent experience to update word meaning preferences depends on language-selective regions, whereas the domain-general MD network plays a causal role in reporting words from degraded speech.

Domain-specific and domain-general neural network engagement during human-robot interactions

To what extent do domain-general and domain-specific neural network engagement generalize across interactions with human and artificial agents? In this exploratory study, we analysed a publicly available functional MRI (fMRI) data set (n = 22) to probe the similarities and dissimilarities in neural architecture while participants conversed with another person or a robot. Incorporating trial-by-trial dynamics of the interactions, listening and speaking, we used whole-brain, region-of-interest and functional connectivity analyses to test response profiles within and across social or non-social, domain-specific and domain-general networks, that is, the person perception, theory-of-mind, object-specific, language and multiple-demand networks. Listening to a robot compared to a human resulted in higher activation in the language network, especially in areas associated with listening comprehension, and in the person perception network. No differences in activity of the theory-of-mind network were found. Results from the functional connectivity analysis showed no difference between interactions with a human or robot in within- and between-network connectivity. Together, these results suggest that although largely similar regions are activated when speaking to a human and to a robot, activity profiles during listening point to a dissociation at a lower level or perceptual level, but not higher order cognitive level.

Robust Effects of Working Memory Demand during Naturalistic Language Comprehension in Language-Selective Cortex

To understand language, we must infer structured meanings from real-time auditory or visual signals. Researchers have long focused on word-by-word structure building in working memory as a mechanism that might enable this feat. However, some have argued that language processing does not typically involve rich word-by-word structure building, and/or that apparent working memory effects are underlyingly driven by surprisal (how predictable a word is in context). Consistent with this alternative, some recent behavioral studies of naturalistic language processing that control for surprisal have not shown clear working memory effects. In this fMRI study, we investigate a range of theory-driven predictors of word-by-word working memory demand during naturalistic language comprehension in humans of both sexes under rigorous surprisal controls. In addition, we address a related debate about whether the working memory mechanisms involved in language comprehension are language specialized or domain general. To do so, in each participant, we functionally localize (1) the language-selective network and (2) the "multiple-demand" network, which supports working memory across domains. Results show robust surprisal-independent effects of memory demand in the language network and no effect of memory demand in the multiple-demand network. Our findings thus support the view that language comprehension involves computationally demanding word-by-word structure building operations in working memory, in addition to any prediction-related mechanisms. Further, these memory operations appear to be primarily conducted by the same neural resources that store linguistic knowledge, with no evidence of involvement of brain regions known to support working memory across domains.SIGNIFICANCE STATEMENT This study uses fMRI to investigate signatures of working memory (WM) demand during naturalistic story listening, using a broad range of theoretically motivated estimates of WM demand. Results support a strong effect of WM demand in the brain that is distinct from effects of word predictability. Further, these WM demands register primarily in language-selective regions, rather than in "multiple-demand" regions that have previously been associated with WM in nonlinguistic domains. Our findings support a core role for WM in incremental language processing, using WM resources that are specialized for language.

Probabilistic atlas for the language network based on precision fMRI data from >800 individuals

Two analytic traditions characterize fMRI language research. One relies on averaging activations across individuals. This approach has limitations: because of inter-individual variability in the locations of language areas, any given voxel/vertex in a common brain space is part of the language network in some individuals but in others, may belong to a distinct network. An alternative approach relies on identifying language areas in each individual using a functional 'localizer'. Because of its greater sensitivity, functional resolution, and interpretability, functional localization is gaining popularity, but it is not always feasible, and cannot be applied retroactively to past studies. To bridge these disjoint approaches, we created a probabilistic functional atlas using fMRI data for an extensively validated language localizer in 806 individuals. This atlas enables estimating the probability that any given location in a common space belongs to the language network, and thus can help interpret group-level activation peaks and lesion locations, or select voxels/electrodes for analysis. More meaningful comparisons of findings across studies should increase robustness and replicability in language research.

Neural evidence for non-orofacial triggers in mild misophonia

Misophonia, an extreme aversion to certain environmental sounds, is a highly prevalent yet understudied condition plaguing roughly 20% of the general population. Although neuroimaging research on misophonia is scant, recent work showing higher resting-state functional connectivity (rs-fMRI) between auditory cortex and orofacial motor cortex in misophonia vs. controls has led researchers to speculate that misophonia is caused by orofacial mirror neurons. Since orofacial motor cortex was defined using rs-fMRI, we attempted to theoretically replicate these findings using orofacial cortex defined by task-based fMRI instead. Further, given our recent work showing that a wide variety of sounds can be triggering (i.e., not just oral/nasal sounds), we investigated whether there is any neural evidence for misophonic aversion to non-orofacial stimuli. Sampling 19 adults with varying misophonia from the community, we collected resting state data and an fMRI task involving phoneme articulation and finger-tapping. We first defined "orofacial" cortex in each participant using rs-fMRI as done previously, producing what we call resting-state regions of interest (rsROIs). Additionally, we functionally defined regions (fROIs) representing "orofacial" or "finger" cortex using phoneme or finger-tapping activation from the fMRI task, respectively. To investigate the motor specificity of connectivity differences, we subdivided the rsROIs and fROIs into separate sensorimotor areas based on their overlap with two common atlases. We then calculated rs-fMRI between each rsROI/fROI and a priori non-sensorimotor ROIs. We found increased connectivity in mild misophonia between rsROIs and both auditory cortex and insula, theoretically replicating previous results, with differences extending across multiple sensorimotor regions. However, the orofacial task-based fROIs did not show this pattern, suggesting the "orofacial" cortex described previously was not capturing true orofacial cortex; in fact, using task-based fMRI evidence, we find no selectivity to orofacial action in these previously described "orofacial" regions. Instead, we observed higher connectivity between finger fROIs and insula in mild misophonia, demonstrating neural evidence for non-orofacial triggers. These results provide support for a neural representation of misophonia beyond merely an orofacial/motor origin, leading to important implications for the conceptualization and treatment of misophonia.

An investigation across 45 languages and 12 language families reveals a universal language network

To understand the architecture of human language, it is critical to examine diverse languages; however, most cognitive neuroscience research has focused on only a handful of primarily Indo-European languages. Here we report an investigation of the fronto-temporo-parietal language network across 45 languages and establish the robustness to cross-linguistic variation of its topography and key functional properties, including left-lateralization, strong functional integration among its brain regions and functional selectivity for language processing.

Differential Tracking of Linguistic vs. Mental State Content in Naturalistic Stimuli by Language and Theory of Mind (ToM) Brain Networks

Language and social cognition, especially the ability to reason about mental states, known as theory of mind (ToM), are deeply related in development and everyday use. However, whether these cognitive faculties rely on distinct, overlapping, or the same mechanisms remains debated. Some evidence suggests that, by adulthood, language and ToM draw on largely distinct-though plausibly interacting-cortical networks. However, the broad topography of these networks is similar, and some have emphasized the importance of social content / communicative intent in the linguistic signal for eliciting responses in the language areas. Here, we combine the power of individual-subject functional localization with the naturalistic-cognition inter-subject correlation approach to illuminate the language-ToM relationship. Using functional magnetic resonance imaging (fMRI), we recorded neural activity as participants (n = 43) listened to stories and dialogues with mental state content (+linguistic, +ToM), viewed silent animations and live action films with mental state content but no language (-linguistic, +ToM), or listened to an expository text (+linguistic, -ToM). The ToM network robustly tracked stimuli rich in mental state information regardless of whether mental states were conveyed linguistically or non-linguistically, while tracking a +linguistic / -ToM stimulus only weakly. In contrast, the language network tracked linguistic stimuli more strongly than (a) non-linguistic stimuli, and than (b) the ToM network, and showed reliable tracking even for the linguistic condition devoid of mental state content. These findings suggest that in spite of their indisputably close links, language and ToM dissociate robustly in their neural substrates-and thus plausibly cognitive mechanisms-including during the processing of rich naturalistic materials.

Individual Differences in Indirect Speech Act Processing Found Outside the Language Network

Face-to-face communication requires skills that go beyond core language abilities. In dialogue, we routinely make inferences beyond the literal meaning of utterances and distinguish between different speech acts based on, e.g., contextual cues. It is, however, not known whether such communicative skills potentially overlap with core language skills or other capacities, such as theory of mind (ToM). In this functional magnetic resonance imaging (fMRI) study we investigate these questions by capitalizing on individual variation in pragmatic skills in the general population. Based on behavioral data from 199 participants, we selected participants with higher vs. lower pragmatic skills for the fMRI study (N = 57). In the scanner, participants listened to dialogues including a direct or an indirect target utterance. The paradigm allowed participants at the whole group level to (passively) distinguish indirect from direct speech acts, as evidenced by a robust activity difference between these speech acts in an extended language network including ToM areas. Individual differences in pragmatic skills modulated activation in two additional regions outside the core language regions (one cluster in the left lateral parietal cortex and intraparietal sulcus and one in the precuneus). The behavioral results indicate segregation of pragmatic skill from core language and ToM. In conclusion, contextualized and multimodal communication requires a set of interrelated pragmatic processes that are neurocognitively segregated: (1) from core language and (2) partly from ToM.

The Domain-General Multiple Demand Network Is More Active in Early Balanced Bilinguals Than Monolinguals During Executive Processing

The bilingual experience may place special cognitive demands on speakers and has been argued to lead to improvements in domain-general executive abilities, like cognitive control and working memory. Such improvements have been argued for based on both behavioral and brain imaging evidence. However, the empirical landscape is complex and ridden with controversy. Here we attempt to shed light on this question through an fMRI investigation of relatively large, relatively homogeneous, and carefully matched samples of early balanced bilinguals (n = 55) and monolinguals (n = 54), using robust, previously validated individual-level markers of neural activity in the domain-general multiple demand (MD) network, which supports executive functions. We find that the bilinguals, compared to the monolinguals, show significantly stronger neural responses to an executive (spatial working memory) task, and a larger difference between a harder and an easier condition of the task, across the MD network. These stronger neural responses are accompanied by better behavioral performance on the working memory task. We further show that the bilingual-vs.-monolingual difference in neural responses is not ubiquitous across the brain as no group difference in magnitude is observed in primary visual areas, which also respond to the task. Although the neural group difference in the MD network appears robust, it remains difficult to causally link it to bilingual experience specifically.

The neural architecture of language: Integrative modeling converges on predictive processing

The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a systematic study taking this approach to higher-level cognition: human language processing, our species' signature cognitive skill. We find that the most powerful "transformer" models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (functional MRI and electrocorticography). Models' neural fits ("brain score") and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

Multiple-brain systems dynamically interact during tonic and phasic states to support language integrity in temporal lobe epilepsy

•

Unique brain dynamics occur during language task in left temporal lobe epilepsy (TLE).

•

Multiple brain systems interact to implement compensated language status in TLE.

•

Tonic/rest dynamics exert influence and may prime the level of phasic/task dynamics.

•

Multi-network integrations are compensatory in patients with lower language skills.

An epileptogenic focus in the dominant temporal lobe can result in the reorganization of language systems in order to compensate for compromised functions. We studied the compensatory reorganization of language in the setting of left temporal lobe epilepsy (TLE), taking into account the interaction of language (L) with key non-language (NL) networks such as dorsal attention (DAN), fronto-parietal (FPN) and cingulo-opercular (COpN), with these systems providing cognitive resources helpful for successful language performance.

We applied tools from dynamic network neuroscience to functional MRI data collected from 23 TLE patients and 23 matched healthy controls during the resting state (RS) and a sentence completion (SC) task to capture how the functional architecture of a language network dynamically changes and interacts with NL systems in these two contexts.

We provided evidence that the brain areas in which core language functions reside dynamically interact with non-language functional networks to carry out linguistic functions. We demonstrated that abnormal integrations between the language and DAN existed in TLE, and were present both in tonic as well as phasic states. This integration was considered to reflect the entrainment of visual attention systems to the systems dedicated to lexical semantic processing. Our data made clear that the level of baseline integrations between the language subsystems and certain NL systems (e.g., DAN, FPN) had a crucial influence on the general level of task integrations between L/NL systems, with this a normative finding not unique to epilepsy. We also revealed that a broad set of task L/NL integrations in TLE are predictive of language competency, indicating that these integrations are compensatory for patients with lower overall language skills.

We concluded that RS establishes the broad set of L/NL integrations available and primed for use during task, but that the actual use of those interactions in the setting of TLE depended on the level of language skill. We believe our analyses are the first to capture the potential compensatory role played by dynamic network reconfigurations between multiple brain systems during performance of a complex language task, in addition to testing for characteristics in both the phasic/task and tonic/resting state that are necessary to achieve language competency in the setting of temporal lobe pathology. Our analyses highlighted the intra- versus inter-system communications that form the basis of unique language processing in TLE, pointing to the dynamic reconfigurations that provided the broad multi-system support needed to maintain language skill and competency.

Precise Topology of Adjacent Domain-General and Sensory-Biased Regions in the Human Brain

Recent functional MRI studies identified sensory-biased regions across much of the association cortices and cerebellum. However, their anatomical relationship to multiple-demand (MD) regions, characterized as domain-general due to their coactivation during multiple cognitive demands, remains unclear. For a better anatomical delineation, we used multimodal MRI techniques of the Human Connectome Project to scan subjects performing visual and auditory versions of a working memory (WM) task. The contrast between hard and easy WM showed strong domain generality, with essentially identical patterns of cortical, subcortical, and cerebellar MD activity for visual and auditory materials. In contrast, modality preferences were shown by contrasting easy WM with baseline; most MD regions showed visual preference while immediately adjacent to cortical MD regions, there were interleaved regions of both visual and auditory preference. The results may exemplify a general motif whereby domain-specific regions feed information into and out of an adjacent, integrative MD core.

Functional specificity and neural integration in the aesthetic appreciation of artworks with implied motion

Although there is growing interest in the neural foundations of aesthetic experience, it remains unclear how particular mental subsystems (e.g. perceptual, affective and cognitive) are involved in different types of aesthetic judgements. Here, we use fMRI to investigate the involvement of different neural networks during aesthetic judgements of visual artworks with implied motion cues. First, a behavioural experiment (N = 45) confirmed a preference for paintings with implied motion over static cues. Subsequently, in a preregistered fMRI experiment (N = 27), participants made aesthetic and motion judgements towards paintings representing human bodies in dynamic and static postures. Using functional region-of-interest and Bayesian multilevel modelling approaches, we provide no compelling evidence for unique sensitivity within or between neural systems associated with body perception, motion and affective processing during the aesthetic evaluation of paintings with implied motion. However, we show suggestive evidence that motion and body-selective systems may integrate signals via functional connections with a separate neural network in dorsal parietal cortex, which may act as a relay or integration site. Our findings clarify the roles of basic visual and affective brain circuitry in evaluating a central aesthetic feature-implied motion-while also pointing towards promising future research directions, which involve modelling aesthetic preferences as hierarchical interplay between visual and affective circuits and integration processes in frontoparietal cortex.

Extended Frontal Networks for Visual and Auditory Working Memory

Working memory (WM) supports the persistent representation of transient sensory information. Visual and auditory stimuli place different demands on WM and recruit different brain networks. Separate auditory- and visual-biased WM networks extend into the frontal lobes, but several challenges confront attempts to parcellate human frontal cortex, including fine-grained organization and between-subject variability. Here, we use differential intrinsic functional connectivity from 2 visual-biased and 2 auditory-biased frontal structures to identify additional candidate sensory-biased regions in frontal cortex. We then examine direct contrasts of task functional magnetic resonance imaging during visual versus auditory 2-back WM to validate those candidate regions. Three visual-biased and 5 auditory-biased regions are robustly activated bilaterally in the frontal lobes of individual subjects (N = 14, 7 women). These regions exhibit a sensory preference during passive exposure to task stimuli, and that preference is stronger during WM. Hierarchical clustering analysis of intrinsic connectivity among novel and previously identified bilateral sensory-biased regions confirms that they functionally segregate into visual and auditory networks, even though the networks are anatomically interdigitated. We also observe that the frontotemporal auditory WM network is highly selective and exhibits strong functional connectivity to structures serving non-WM functions, while the frontoparietal visual WM network hierarchically merges into the multiple-demand cognitive system.