1
|
Lakretz Y, Friedmann N, King JR, Mankin E, Rangel A, Tankus A, Dehaene S, Fried I. Modality-Specific and Amodal Language Processing by Single Neurons. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.11.16.623907. [PMID: 39605371 PMCID: PMC11601528 DOI: 10.1101/2024.11.16.623907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/29/2024]
Abstract
According to psycholinguistic theories, during language processing, spoken and written words are first encoded along independent phonological and orthographic dimensions, then enter into modality-independent syntactic and semantic codes. Non-invasive brain imaging has isolated several cortical regions putatively associated with those processing stages, but lacks the resolution to identify the corresponding neural codes. Here, we describe the firing responses of over 1000 neurons, and mesoscale field potentials from over 1400 microwires and 1500 iEEG contacts in 21 awake neurosurgical patients with implanted electrodes during written and spoken sentence comprehension. Using forward modeling of temporal receptive fields, we determined which sensory or abstract dimensions are encoded. We observed a double dissociation between superior temporal neurons sensitive to phonemes and phonological features and previously unreported ventral occipito-temporal neurons sensitive to letters and orthographic features. We also discovered novel neurons, primarily located in middle temporal and inferior frontal areas, which are modality-independent and show responsiveness to higher linguistic features. Overall, these findings show how language processing can be linked to neural dynamics, across multiple brain regions at various resolutions and down to the level of single neurons.
Collapse
Affiliation(s)
- Yair Lakretz
- Laboratoire des Sciences Cognitives et Psycholinguistiques, Département d’études cognitives, Ecole Normale Supérieure, PSL University, CNRS, Paris, France
- Cognitive Neuroimaging Unit, CEA, INSERM U 992, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, France
| | - Naama Friedmann
- School of Education, Tel-Aviv University, Tel-Aviv, Israel
- Sagol School of Neuroscience, Tel-Aviv University, Tel-Aviv, Israel
| | - Jean-Rémi King
- Laboratoire des Systèmes Perceptifs, Département d’études cognitives, Ecole Normale Supérieure, PSL University, CNRS, Paris, France
| | - Emily Mankin
- Department of Neurosurgery, David Geffen School of Medicine, UCLA, Los-Angeles, California, USA
| | - Anthony Rangel
- Department of Neurosurgery, David Geffen School of Medicine, UCLA, Los-Angeles, California, USA
| | - Ariel Tankus
- Sagol School of Neuroscience, Tel-Aviv University, Tel-Aviv, Israel
- Faculty of Medicine, Tel Aviv University, Tel-Aviv, Israel
- Functional Neurosurgery Unit, Tel Aviv Sourasky Medical Center, Tel-Aviv, Israel
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, CEA, INSERM U 992, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, France
- Collège de France, Université Paris Sciences Lettres (PSL), Paris, France
| | - Itzhak Fried
- Department of Neurosurgery, David Geffen School of Medicine, UCLA, Los-Angeles, California, USA
- Faculty of Medicine, Tel Aviv University, Tel-Aviv, Israel
| |
Collapse
|
2
|
Li T, Feng C, Wang J. Reconfiguration of the costly punishment network architecture in punishment decision-making. Psychophysiology 2024; 61:e14458. [PMID: 37941501 DOI: 10.1111/psyp.14458] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 09/15/2023] [Accepted: 10/02/2023] [Indexed: 11/10/2023]
Abstract
Human costly punishment is rooted in multiple regions across large-scale functional systems, a collection of which constitutes the costly punishment network (CPN). Our previous study found that the CPN is intrinsically organized in an optimized and reliable manner to support individual costly punishment propensity. However, it remains unknown how the CPN is reconfigured in response to external cognitive demands in punishment decision-making. Here, we combined resting-state and task-functional magnetic resonance imaging to examine the task-related reconfigurations of intrinsic organizations of the CPN when participants made decisions of costly punishment in the Ultimatum Game. Although a strong consistency was observed in the overall pattern and each nodal profile between the intrinsic (task-free) and extrinsic (task-evoked) functional connectivity of the CPN, condition-general and condition-specific reconfigurations were also evident. Specifically, both unfair and fair conditions induced increases in functional connectivity between a few specific pairs of regions, and the unfair condition additionally induced increases in network efficiency of the CPN. Intriguingly, the specific changes in global efficiency of the CPN in the unfair condition were associated with individual differences in costly punishment after adjusting for the corresponding results in the fair condition, which were further identified for females but not for males. These findings were largely reproducible on independent samples. Collectively, our findings provide novel insights into how the CPN adaptively reconfigures its network architecture to support costly punishment.
Collapse
Affiliation(s)
- Ting Li
- Institute of Brain and Psychological Sciences, Sichuan Normal University, Chengdu, China
- Sichuan Key Laboratory of Psychology and Behavior of Discipline Inspection and Supervision, Chengdu, China
| | - Chunliang Feng
- School of Psychology, South China Normal University, Guangzhou, China
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education, Guangzhou, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, China
- Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, China
| | - Jinhui Wang
- Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education, Guangzhou, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, China
- Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, China
- Institute of Brain Research and Rehabilitation, South China Normal University, Guangzhou, China
| |
Collapse
|
3
|
Lee JJ, Scott TL, Perrachione TK. Efficient functional localization of language regions in the brain. Neuroimage 2024; 285:120489. [PMID: 38065277 PMCID: PMC10999251 DOI: 10.1016/j.neuroimage.2023.120489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 11/25/2023] [Accepted: 12/05/2023] [Indexed: 12/17/2023] Open
Abstract
Important recent advances in the cognitive neuroscience of language have been made using functional localizers to demarcate language-selective regions in individual brains. Although single-subject localizers offer insights that are unavailable in classic group analyses, they require additional scan time that imposes costs on investigators and participants. In particular, the unique practical challenges of scanning children and other special populations has led to less adoption of localizers for neuroimaging research with these theoretically and clinically important groups. Here, we examined how measurements of the spatial extent and functional response profiles of language regions are affected by the duration of an auditory language localizer. We compared how parametrically smaller amounts of data collected from one scanning session affected (i) consistency of group-level whole-brain parcellations, (ii) functional selectivity of subject-level activation in individually defined functional regions of interest (fROIs), (iii) sensitivity and specificity of subject-level whole-brain and fROI activation, and (iv) test-retest reliability of subject-level whole-brain and fROI activation. For many of these metrics, the localizer duration could be reduced by 50-75% while preserving the stability and reliability of both the spatial extent and functional response profiles of language areas. These results indicate that, for most measures relevant to cognitive neuroimaging studies, the brain's language network can be localized just as effectively with 3.5 min of scan time as it can with 12 min. Minimizing the time required to reliably localize the brain's language network allows more effective localizer use in situations where each minute of scan time is particularly precious.
Collapse
Affiliation(s)
- Jayden J Lee
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Ave., Boston, MA 02215, United States
| | - Terri L Scott
- Department of Neurological Surgery, University of California - San Francisco, San Francisco, CA, United States
| | - Tyler K Perrachione
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Ave., Boston, MA 02215, United States.
| |
Collapse
|
4
|
Pasquiou A, Lakretz Y, Thirion B, Pallier C. Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax, and Context. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2023; 4:611-636. [PMID: 38144237 PMCID: PMC10745090 DOI: 10.1162/nol_a_00125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 09/28/2023] [Indexed: 12/26/2023]
Abstract
A fundamental question in neurolinguistics concerns the brain regions involved in syntactic and semantic processing during speech comprehension, both at the lexical (word processing) and supra-lexical levels (sentence and discourse processing). To what extent are these regions separated or intertwined? To address this question, we introduce a novel approach exploiting neural language models to generate high-dimensional feature sets that separately encode semantic and syntactic information. More precisely, we train a lexical language model, GloVe, and a supra-lexical language model, GPT-2, on a text corpus from which we selectively removed either syntactic or semantic information. We then assess to what extent the features derived from these information-restricted models are still able to predict the fMRI time courses of humans listening to naturalistic text. Furthermore, to determine the windows of integration of brain regions involved in supra-lexical processing, we manipulate the size of contextual information provided to GPT-2. The analyses show that, while most brain regions involved in language comprehension are sensitive to both syntactic and semantic features, the relative magnitudes of these effects vary across these regions. Moreover, regions that are best fitted by semantic or syntactic features are more spatially dissociated in the left hemisphere than in the right one, and the right hemisphere shows sensitivity to longer contexts than the left. The novelty of our approach lies in the ability to control for the information encoded in the models' embeddings by manipulating the training set. These "information-restricted" models complement previous studies that used language models to probe the neural bases of language, and shed new light on its spatial organization.
Collapse
Affiliation(s)
- Alexandre Pasquiou
- Cognitive Neuroimaging Unit (UNICOG), NeuroSpin, National Institute of Health and Medical Research (Inserm) and French Alternative Energies and Atomic Energy Commission (CEA), Frédéric Joliot Life Sciences Institute, Paris-Saclay University, Gif-sur-Yvette, France
- Models and Inference for Neuroimaging Data (MIND), NeuroSpin, French Alternative Energies and Atomic Energy Commission (CEA), Inria Saclay, Frédéric Joliot Life Sciences Institute, Paris-Saclay University, Gif-sur-Yvette, France
| | - Yair Lakretz
- Cognitive Neuroimaging Unit (UNICOG), NeuroSpin, National Institute of Health and Medical Research (Inserm) and French Alternative Energies and Atomic Energy Commission (CEA), Frédéric Joliot Life Sciences Institute, Paris-Saclay University, Gif-sur-Yvette, France
| | - Bertrand Thirion
- Models and Inference for Neuroimaging Data (MIND), NeuroSpin, French Alternative Energies and Atomic Energy Commission (CEA), Inria Saclay, Frédéric Joliot Life Sciences Institute, Paris-Saclay University, Gif-sur-Yvette, France
| | - Christophe Pallier
- Cognitive Neuroimaging Unit (UNICOG), NeuroSpin, National Institute of Health and Medical Research (Inserm) and French Alternative Energies and Atomic Energy Commission (CEA), Frédéric Joliot Life Sciences Institute, Paris-Saclay University, Gif-sur-Yvette, France
| |
Collapse
|
5
|
Kulik V, Reyes LD, Sherwood CC. Coevolution of language and tools in the human brain: An ALE meta-analysis of neural activation during syntactic processing and tool use. PROGRESS IN BRAIN RESEARCH 2023; 275:93-115. [PMID: 36841572 DOI: 10.1016/bs.pbr.2022.10.003] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Language and complex tool use are often cited as behaviors unique to humans and may be evolutionarily linked owing to the underlying cognitive processes they have in common. We executed a quantitative activation likelihood estimation (ALE) meta-analysis (GingerALE 2.3) on published, whole-brain neuroimaging studies to identify areas associated with syntactic processing and/or tool use in humans. Significant clusters related to syntactic processing were identified in areas known to be related to language production and comprehension, including bilateral Broca's area in the inferior frontal gyrus. Tool use activation clusters were all in the left hemisphere and included the primary motor cortex and premotor cortex, in addition to other areas involved with sensorimotor transformation. Activation shared by syntactic processing and tool use was only significant at one cluster, located in the pars opercularis of the left inferior frontal gyrus. This minimal overlap between syntactic processing and tool use activation from our meta-analysis of neuroimaging studies indicates that there is not a widespread common neural network between the two. Broca's area may serve as an important hub that was initially recruited in early human evolution in the context of simple tool use, but was eventually co-opted for linguistic purposes, including the sequential and hierarchical ordering processes that characterize syntax. In the future, meta-analyses of additional components of language may allow for a more comprehensive examination of the functional networks that underlie the coevolution of human language and complex tool use.
Collapse
Affiliation(s)
- Veronika Kulik
- Department of Anthropology and Center for the Advanced Study of Human Paleobiology, The George Washington University, Washington, DC, United States
| | - Laura D Reyes
- Department of Anthropology and Center for the Advanced Study of Human Paleobiology, The George Washington University, Washington, DC, United States
| | - Chet C Sherwood
- Department of Anthropology and Center for the Advanced Study of Human Paleobiology, The George Washington University, Washington, DC, United States.
| |
Collapse
|
6
|
Divjak D, Milin P, Medimorec S, Borowski M. Behavioral Signatures of Memory Resources for Language: Looking beyond the Lexicon/Grammar Divide. Cogn Sci 2022; 46:e13206. [PMID: 36353955 PMCID: PMC9787600 DOI: 10.1111/cogs.13206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Revised: 07/27/2022] [Accepted: 08/27/2022] [Indexed: 11/11/2022]
Abstract
Although there is a broad consensus that both the procedural and declarative memory systems play a crucial role in language learning, use, and knowledge, the mapping between linguistic types and memory structures remains underspecified: by default, a dual-route mapping of language systems to memory systems is assumed, with declarative memory handling idiosyncratic lexical knowledge and procedural memory handling rule-governed knowledge of grammar. We experimentally contrast the processing of morphology (case and aspect), syntax (subordination), and lexical semantics (collocations) in a healthy L1 population of Polish, a language rich in form distinctions. We study the processing of these four types under two conditions: a single task condition in which the grammaticality of stimuli was judged and a concurrent task condition in which grammaticality judgments were combined with a digit span task. Dividing attention impedes access to declarative memory while leaving procedural memory unaffected and hence constitutes a test that dissociates which types of linguistic information each long-term memory construct subserves. Our findings confirm the existence of a distinction between lexicon and grammar as a generative, dual-route model would predict, but the distinction is graded, as usage-based models assume: the hypothesized grammar-lexicon opposition appears as a continuum on which grammatical phenomena can be placed as being more or less "ruly" or "idiosyncratic." However, usage-based models, too, need adjusting as not all types of linguistic knowledge are proceduralized to the same extent. This move away from a simple dichotomy fundamentally changes how we think about memory for language, and hence how we design and interpret behavioral and neuroimaging studies that probe into the nature of language cognition.
Collapse
Affiliation(s)
- Dagmar Divjak
- Department of Modern LanguagesUniversity of BirminghamBirminghamUnited Kingdom,Department of English Language & LinguisticsUniversity of BirminghamBirminghamUnited Kingdom
| | - Petar Milin
- Department of Modern LanguagesUniversity of BirminghamBirminghamUnited Kingdom
| | - Srdan Medimorec
- Department of Modern LanguagesUniversity of BirminghamBirminghamUnited Kingdom,Department of Psychology, Centre for Applied Psychological ScienceTeesside UniversityMiddlesbroughUnited Kingdom
| | - Maciej Borowski
- Department of Modern LanguagesUniversity of BirminghamBirminghamUnited Kingdom
| |
Collapse
|
7
|
Gavard E, Ziegler JC. The Effects of Semantic and Syntactic Prediction on Reading Aloud. Exp Psychol 2022; 69:308-319. [PMID: 36809159 DOI: 10.1027/1618-3169/a000568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023]
Abstract
Semantic and syntactic prediction effects were investigated in a word naming task using semantic or syntactic contexts that varied between three and six words. Participants were asked to read the contexts silently and name a target word, which was indicated by a color change. Semantic contexts were composed of lists of semantically associated words without any syntactic information. Syntactic contexts were composed of semantically neutral sentences, in which the grammatical category but not the lexical identity of the final word was highly predictable. When the presentation time of the context words was long (1,200 ms), both semantically and syntactically related contexts facilitated reading aloud latencies of target words and syntactically related contexts produced larger priming effects than semantically related contexts in two out of three analyses. When the presentation time was short (200 ms), however, syntactic context effects disappeared, while semantic context effects remained significant. Across the three experiments, longer contexts produced faster response latencies, but longer contexts did not produce larger priming effects. The results are discussed in the context of the extant literature on semantic and syntactic priming and more recent evidence, suggesting that syntactic information constrains single word recognition.
Collapse
Affiliation(s)
- Elisa Gavard
- Laboratoire de Psychologie Cognitive (UMR 7290), Aix-Marseille Univ, CNRS, Marseille, France
| | - Johannes C Ziegler
- Laboratoire de Psychologie Cognitive (UMR 7290), Aix-Marseille Univ, CNRS, Marseille, France
| |
Collapse
|
8
|
Tuckute G, Paunov A, Kean H, Small H, Mineroff Z, Blank I, Fedorenko E. Frontal language areas do not emerge in the absence of temporal language areas: A case study of an individual born without a left temporal lobe. Neuropsychologia 2022; 169:108184. [DOI: 10.1016/j.neuropsychologia.2022.108184] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 12/07/2021] [Accepted: 02/15/2022] [Indexed: 10/19/2022]
|
9
|
Examining the transition of novel information toward familiarity. Neuropsychologia 2021; 161:107993. [PMID: 34411595 DOI: 10.1016/j.neuropsychologia.2021.107993] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Revised: 08/14/2021] [Accepted: 08/15/2021] [Indexed: 11/23/2022]
Abstract
Throughout their lives, humans encounter multiple instances of new information that can be inconsistent with prior knowledge (novel). Over time, the once-novel information becomes integrated into their established knowledge base, shifting from novelty to familiarity. In this study, we investigated the processes by which the first steps of this transition take place. We hypothesized that the neural representations of initially novel items gradually change over the course of repeated presentations, expressing a shift toward familiarity. We further assumed that this shift could be traced by examining neural patterns using fMRI. In two experiments, while being scanned, participants read noun-adjective word pairs that were either consistent or inconsistent with their prior knowledge. Stimuli were repeated 3-6 times within the scans. Employing mass univariate and multivariate similarity analyses, we showed that the neural representations associated with the initial presentation of familiar versus novel objects differed in lateral frontal and temporal regions, the medial prefrontal cortex, and the medial temporal lobe. Importantly, the neural representations of novel stimuli gradually changed throughout repetitions until they became indistinguishable from their respective familiar items. We interpret these findings as indicating that an early phase of familiarization can be completed within a few repetitions. This initial familiarization can then serve as the prerequisite to the integration of novel items into existing knowledge. Future empirical and theoretical works can build on the current findings to develop a comprehensive model of the transition from novelty to familiarity.
Collapse
|
10
|
Mesulam MM, Coventry CA, Rader BM, Kuang A, Sridhar J, Martersteck A, Zhang H, Thompson CK, Weintraub S, Rogalski EJ. Modularity and granularity across the language network-A primary progressive aphasia perspective. Cortex 2021; 141:482-496. [PMID: 34153680 PMCID: PMC8319115 DOI: 10.1016/j.cortex.2021.05.002] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 03/22/2021] [Accepted: 05/05/2021] [Indexed: 11/16/2022]
Abstract
Tests of grammar, repetition and semantics were administered to 62 prospectively enrolled right-handed participants with primary progressive aphasia (PPA). Structural brain images were obtained at the time of testing. Regression analyses uncovered 3 clearly delineated non-overlapping left hemisphere clusters where cortical thinning (atrophy) was significantly correlated with impaired performance. A morphosyntactic cluster associated with the grammaticality of sentence construction was located predominantly within the middle and inferior frontal gyri; a phonolexical cluster associated with language repetition was located in the temporoparietal junction; a lexicosemantic cluster associated with object naming and single word comprehension was located within the middle and anterior parts of the temporal lobe and extended into insular, orbitofrontal, and mediotemporal cortices. Commonality analyses were undertaken to explore whether these three clusters were as modular as indicated by the regression analyses or whether some underlying functional granularity could be uncovered. Modularity was defined as the exclusive association of an anatomical cluster with a single type of language task whereas granularity was defined as the association of a single anatomical cluster with more than one type of language task. The commonality analyses revealed a predominantly modular organization with quantitatively minor instances of inter-cluster granularity. The results also reconfirmed previous work on PPA which had shown that Wernicke's area is not essential for word comprehension, that naming impairments can be based either on deficits of lexical retrieval or word comprehension, and that the essential substrates of word comprehension encompass much wider areas of the temporal lobe than the temporal pole. The anatomy of the language network has traditionally been explored through patients with focal cerebrovascular accidents and experiments based on functional activation. Investigations on PPA are showing that focal neurodegenerations can add new perspectives to existing models of the language network.
Collapse
Affiliation(s)
- M-Marsel Mesulam
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA; Davee Department of Neurology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
| | - Christina A Coventry
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Alan Kuang
- Northwestern University Feinberg School of Medicine, Department of Preventive Medicine, Chicago, IL, USA
| | - Jaiashre Sridhar
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Adam Martersteck
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| | - Hui Zhang
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA; Northwestern University Feinberg School of Medicine, Department of Preventive Medicine, Chicago, IL, USA
| | - Cynthia K Thompson
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA; Northwestern University School of Communication, Evanston, IL, USA
| | - Sandra Weintraub
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA; Northwestern University Feinberg School of Medicine, Department of Psychiatry and Behavioral Sciences, Chicago, IL, USA
| | - Emily J Rogalski
- Mesulam Center for Cognitive Neurology and Alzheimer's Disease, Northwestern University Feinberg School of Medicine, Chicago, IL, USA; Northwestern University Feinberg School of Medicine, Department of Psychiatry and Behavioral Sciences, Chicago, IL, USA
| |
Collapse
|
11
|
Lakretz Y, Hupkes D, Vergallito A, Marelli M, Baroni M, Dehaene S. Mechanisms for handling nested dependencies in neural-network language models and humans. Cognition 2021; 213:104699. [PMID: 33941375 DOI: 10.1016/j.cognition.2021.104699] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2020] [Revised: 03/17/2021] [Accepted: 03/22/2021] [Indexed: 11/25/2022]
Abstract
Recursive processing in sentence comprehension is considered a hallmark of human linguistic abilities. However, its underlying neural mechanisms remain largely unknown. We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing, namely the storing of grammatical number and gender information in working memory and its use in long-distance agreement (e.g., capturing the correct number agreement between subject and verb when they are separated by other phrases). Although the network, a recurrent architecture with Long Short-Term Memory units, was solely trained to predict the next word in a large corpus, analysis showed the emergence of a very sparse set of specialized units that successfully handled local and long-distance syntactic agreement for grammatical number. However, the simulations also showed that this mechanism does not support full recursion and fails with some long-range embedded dependencies. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns, with or without embedding. Human and model error patterns were remarkably similar, showing that the model echoes various effects observed in human data. However, a key difference was that, with embedded long-range dependencies, humans remained above chance level, while the model's systematic errors brought it below chance. Overall, our study shows that exploring the ways in which modern artificial neural networks process sentences leads to precise and testable hypotheses about human linguistic performance.
Collapse
Affiliation(s)
- Yair Lakretz
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Saclay, NeuroSpin center, 91191 Gif/Yvette, France.
| | | | | | - Marco Marelli
- Department of Psychology and NeuroMi, University of Milano Bicocca, Milano, Italy
| | - Marco Baroni
- Facebook AI Research, Paris, France; Catalan Institute for Research and Advanced Studies, Barcelona 08010, Spain
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Saclay, NeuroSpin center, 91191 Gif/Yvette, France; Collège de France, Université Paris-Sciences-Lettres (PSL), 11 Place Marcelin Berthelot, 75005 Paris, France
| |
Collapse
|
12
|
Interference Resolution in Nonfluent Variant Primary Progressive Aphasia: Evidence From a Picture-Word Interference Task. Cogn Behav Neurol 2021; 34:11-25. [PMID: 33652466 DOI: 10.1097/wnn.0000000000000255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Accepted: 08/25/2020] [Indexed: 11/26/2022]
Abstract
BACKGROUND Picture-word interference tasks have been used to investigate (a) the time course of lexical access in individuals with primary progressive aphasia (PPA) and (b) how these individuals resolve competition during lexical selection. OBJECTIVE To investigate the time course of Greek-speaking individuals with PPA to produce grammatical gender-marked determiner phrases by examining their picture-naming latencies in the context of distractor words. METHOD Eight individuals with nonfluent variant PPA (nfv-PPA; M age = 62.8 years) and eight cognitively intact controls (M age = 61.1 years) participated in our study. In a picture-word interference task, the study participants named depicted objects by producing determiner + noun sequences. Interference was generated by manipulating the grammatical gender of the depicted objects and distractor words. Two stimulus onset asynchronies were used: +200 ms and +400 ms. RESULTS The individuals with nfv-PPA exhibited longer picture-naming latencies than the controls (P = 0.003). The controls exhibited interference from incongruent distractors at both asynchronies (P < 0.001); the individuals with PPA exhibited interference from incongruent distractors only at the +400-ms interval (P = 0.002). The gender-congruency effect was stronger for the individuals with PPA than for the controls at the +400-ms interval (P = 0.05); the opposite pattern was observed at the +200-ms interval (P = 0.024). CONCLUSION Gender interference resolution was abnormal in the individuals with nfv-PPA. The results point to deficits in lexicosyntactic networks that compromised the time course of picture-naming production.
Collapse
|
13
|
Pinho AL, Amadon A, Fabre M, Dohmatob E, Denghien I, Torre JJ, Ginisty C, Becuwe-Desmidt S, Roger S, Laurier L, Joly-Testault V, Médiouni-Cloarec G, Doublé C, Martins B, Pinel P, Eger E, Varoquaux G, Pallier C, Dehaene S, Hertz-Pannier L, Thirion B. Subject-specific segregation of functional territories based on deep phenotyping. Hum Brain Mapp 2020; 42:841-870. [PMID: 33368868 PMCID: PMC7856658 DOI: 10.1002/hbm.25189] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2020] [Revised: 07/11/2020] [Accepted: 08/04/2020] [Indexed: 11/08/2022] Open
Abstract
Functional magnetic resonance imaging (fMRI) has opened the possibility to investigate how brain activity is modulated by behavior. Most studies so far are bound to one single task, in which functional responses to a handful of contrasts are analyzed and reported as a group average brain map. Contrariwise, recent data-collection efforts have started to target a systematic spatial representation of multiple mental functions. In this paper, we leverage the Individual Brain Charting (IBC) dataset-a high-resolution task-fMRI dataset acquired in a fixed environment-in order to study the feasibility of individual mapping. First, we verify that the IBC brain maps reproduce those obtained from previous, large-scale datasets using the same tasks. Second, we confirm that the elementary spatial components, inferred across all tasks, are consistently mapped within and, to a lesser extent, across participants. Third, we demonstrate the relevance of the topographic information of the individual contrast maps, showing that contrasts from one task can be predicted by contrasts from other tasks. At last, we showcase the benefit of contrast accumulation for the fine functional characterization of brain regions within a prespecified network. To this end, we analyze the cognitive profile of functional territories pertaining to the language network and prove that these profiles generalize across participants.
Collapse
Affiliation(s)
| | - Alexis Amadon
- Université Paris-Saclay, CEA, CNRS, BAOBAB, NeuroSpin, Gif-sur-Yvette, France
| | - Murielle Fabre
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Elvis Dohmatob
- Université Paris-Saclay, Inria, CEA, Palaiseau, France.,Criteo AI Lab, Paris, France
| | - Isabelle Denghien
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | | | | | | | | | | | | | | | | | | | - Philippe Pinel
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Evelyn Eger
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | | | - Christophe Pallier
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France.,Collège de France, Paris, France
| | - Lucie Hertz-Pannier
- CEA Saclay/DRF/IFJ/NeuroSpin/UNIACT, Paris, France.,UMR 1141, NeuroDiderot, Université de Paris, Paris, France
| | | |
Collapse
|
14
|
Friston KJ, Parr T, Yufik Y, Sajid N, Price CJ, Holmes E. Generative models, linguistic communication and active inference. Neurosci Biobehav Rev 2020; 118:42-64. [PMID: 32687883 PMCID: PMC7758713 DOI: 10.1016/j.neubiorev.2020.07.005] [Citation(s) in RCA: 41] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 06/26/2020] [Accepted: 07/08/2020] [Indexed: 11/24/2022]
Abstract
This paper presents a biologically plausible generative model and inference scheme that is capable of simulating communication between synthetic subjects who talk to each other. Building on active inference formulations of dyadic interactions, we simulate linguistic exchange to explore generative models that support dialogues. These models employ high-order interactions among abstract (discrete) states in deep (hierarchical) models. The sequential nature of language processing mandates generative models with a particular factorial structure-necessary to accommodate the rich combinatorics of language. We illustrate linguistic communication by simulating a synthetic subject who can play the 'Twenty Questions' game. In this game, synthetic subjects take the role of the questioner or answerer, using the same generative model. This simulation setup is used to illustrate some key architectural points and demonstrate that many behavioural and neurophysiological correlates of linguistic communication emerge under variational (marginal) message passing, given the right kind of generative model. For example, we show that theta-gamma coupling is an emergent property of belief updating, when listening to another.
Collapse
Affiliation(s)
- Karl J Friston
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Thomas Parr
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Yan Yufik
- Virtual Structures Research, Inc., 12204 Saint James Rd, Potomac, MD 20854, USA.
| | - Noor Sajid
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Catherine J Price
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| | - Emma Holmes
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, 12 Queen Square, London, WC1N 3AR, UK.
| |
Collapse
|
15
|
Blank IA, Fedorenko E. No evidence for differences among language regions in their temporal receptive windows. Neuroimage 2020; 219:116925. [PMID: 32407994 PMCID: PMC9392830 DOI: 10.1016/j.neuroimage.2020.116925] [Citation(s) in RCA: 42] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Revised: 03/20/2020] [Accepted: 05/06/2020] [Indexed: 10/24/2022] Open
Abstract
The "core language network" consists of left frontal and temporal regions that are selectively engaged in linguistic processing. Whereas functional differences among these regions have long been debated, many accounts propose distinctions in terms of representational grain-size-e.g., words vs. phrases/sentences-or processing time-scale, i.e., operating on local linguistic features vs. larger spans of input. Indeed, the topography of language regions appears to overlap with a cortical hierarchy reported by Lerner et al. (2011) wherein mid-posterior temporal regions are sensitive to low-level features of speech, surrounding areas-to word-level information, and inferior frontal areas-to sentence-level information and beyond. However, the correspondence between the language network and this hierarchy of "temporal receptive windows" (TRWs) is difficult to establish because the precise anatomical locations of language regions vary across individuals. To directly test this correspondence, we first identified language regions in each participant with a well-validated task-based localizer, which confers high functional resolution to the study of TRWs (traditionally based on stereotactic coordinates); then, we characterized regional TRWs with the naturalistic story listening paradigm of Lerner et al. (2011), which augments task-based characterizations of the language network by more closely resembling comprehension "in the wild". We find no region-by-TRW interactions across temporal and inferior frontal regions, which are all sensitive to both word-level and sentence-level information. Therefore, the language network as a whole constitutes a unique stage of information integration within a broader cortical hierarchy.
Collapse
Affiliation(s)
- Idan A Blank
- Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| |
Collapse
|
16
|
Fedorenko E, Blank IA, Siegelman M, Mineroff Z. Lack of selectivity for syntax relative to word meanings throughout the language network. Cognition 2020; 203:104348. [PMID: 32569894 DOI: 10.1101/477851] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2018] [Revised: 05/14/2020] [Accepted: 05/31/2020] [Indexed: 05/25/2023]
Abstract
To understand what you are reading now, your mind retrieves the meanings of words and constructions from a linguistic knowledge store (lexico-semantic processing) and identifies the relationships among them to construct a complex meaning (syntactic or combinatorial processing). Do these two sets of processes rely on distinct, specialized mechanisms or, rather, share a common pool of resources? Linguistic theorizing, empirical evidence from language acquisition and processing, and computational modeling have jointly painted a picture whereby lexico-semantic and syntactic processing are deeply inter-connected and perhaps not separable. In contrast, many current proposals of the neural architecture of language continue to endorse a view whereby certain brain regions selectively support syntactic/combinatorial processing, although the locus of such "syntactic hub", and its nature, vary across proposals. Here, we searched for selectivity for syntactic over lexico-semantic processing using a powerful individual-subjects fMRI approach across three sentence comprehension paradigms that have been used in prior work to argue for such selectivity: responses to lexico-semantic vs. morpho-syntactic violations (Experiment 1); recovery from neural suppression across pairs of sentences differing in only lexical items vs. only syntactic structure (Experiment 2); and same/different meaning judgments on such sentence pairs (Experiment 3). Across experiments, both lexico-semantic and syntactic conditions elicited robust responses throughout the left fronto-temporal language network. Critically, however, no regions were more strongly engaged by syntactic than lexico-semantic processing, although some regions showed the opposite pattern. Thus, contra many current proposals of the neural architecture of language, syntactic/combinatorial processing is not separable from lexico-semantic processing at the level of brain regions-or even voxel subsets-within the language network, in line with strong integration between these two processes that has been consistently observed in behavioral and computational language research. The results further suggest that the language network may be generally more strongly concerned with meaning than syntactic form, in line with the primary function of language-to share meanings across minds.
Collapse
Affiliation(s)
- Evelina Fedorenko
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, USA.
| | - Idan Asher Blank
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; Department of Psychology, UCLA, Los Angeles, CA 90095, USA
| | - Matthew Siegelman
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; Department of Psychology, Columbia University, New York, NY 10027, USA
| | - Zachary Mineroff
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; Eberly Center for Teaching Excellence & Educational Innovation, CMU, Pittsburgh, PA 15213, USA
| |
Collapse
|
17
|
Fedorenko E, Blank IA, Siegelman M, Mineroff Z. Lack of selectivity for syntax relative to word meanings throughout the language network. Cognition 2020; 203:104348. [PMID: 32569894 PMCID: PMC7483589 DOI: 10.1016/j.cognition.2020.104348] [Citation(s) in RCA: 77] [Impact Index Per Article: 15.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2018] [Revised: 05/14/2020] [Accepted: 05/31/2020] [Indexed: 12/31/2022]
Abstract
To understand what you are reading now, your mind retrieves the meanings of words and constructions from a linguistic knowledge store (lexico-semantic processing) and identifies the relationships among them to construct a complex meaning (syntactic or combinatorial processing). Do these two sets of processes rely on distinct, specialized mechanisms or, rather, share a common pool of resources? Linguistic theorizing, empirical evidence from language acquisition and processing, and computational modeling have jointly painted a picture whereby lexico-semantic and syntactic processing are deeply inter-connected and perhaps not separable. In contrast, many current proposals of the neural architecture of language continue to endorse a view whereby certain brain regions selectively support syntactic/combinatorial processing, although the locus of such "syntactic hub", and its nature, vary across proposals. Here, we searched for selectivity for syntactic over lexico-semantic processing using a powerful individual-subjects fMRI approach across three sentence comprehension paradigms that have been used in prior work to argue for such selectivity: responses to lexico-semantic vs. morpho-syntactic violations (Experiment 1); recovery from neural suppression across pairs of sentences differing in only lexical items vs. only syntactic structure (Experiment 2); and same/different meaning judgments on such sentence pairs (Experiment 3). Across experiments, both lexico-semantic and syntactic conditions elicited robust responses throughout the left fronto-temporal language network. Critically, however, no regions were more strongly engaged by syntactic than lexico-semantic processing, although some regions showed the opposite pattern. Thus, contra many current proposals of the neural architecture of language, syntactic/combinatorial processing is not separable from lexico-semantic processing at the level of brain regions-or even voxel subsets-within the language network, in line with strong integration between these two processes that has been consistently observed in behavioral and computational language research. The results further suggest that the language network may be generally more strongly concerned with meaning than syntactic form, in line with the primary function of language-to share meanings across minds.
Collapse
Affiliation(s)
- Evelina Fedorenko
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; McGovern Institute for Brain Research, MIT, Cambridge, MA 02139, USA.
| | - Idan Asher Blank
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; Department of Psychology, UCLA, Los Angeles, CA 90095, USA
| | - Matthew Siegelman
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; Department of Psychology, Columbia University, New York, NY 10027, USA
| | - Zachary Mineroff
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USA; Eberly Center for Teaching Excellence & Educational Innovation, CMU, Pittsburgh, PA 15213, USA
| |
Collapse
|
18
|
Diachek E, Blank I, Siegelman M, Affourtit J, Fedorenko E. The Domain-General Multiple Demand (MD) Network Does Not Support Core Aspects of Language Comprehension: A Large-Scale fMRI Investigation. J Neurosci 2020; 40:4536-4550. [PMID: 32317387 PMCID: PMC7275862 DOI: 10.1523/jneurosci.2036-19.2020] [Citation(s) in RCA: 87] [Impact Index Per Article: 17.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2019] [Revised: 03/02/2020] [Accepted: 04/05/2020] [Indexed: 11/21/2022] Open
Abstract
Aside from the language-selective left-lateralized frontotemporal network, language comprehension sometimes recruits a domain-general bilateral frontoparietal network implicated in executive functions: the multiple demand (MD) network. However, the nature of the MD network's contributions to language comprehension remains debated. To illuminate the role of this network in language processing in humans, we conducted a large-scale fMRI investigation using data from 30 diverse word and sentence comprehension experiments (481 unique participants [female and male], 678 scanning sessions). In line with prior findings, the MD network was active during many language tasks. Moreover, similar to the language-selective network, which is robustly lateralized to the left hemisphere, these responses were stronger in the left-hemisphere MD regions. However, in contrast with the language-selective network, the MD network responded more strongly (1) to lists of unconnected words than to sentences, and (2) in paradigms with an explicit task compared with passive comprehension paradigms. Indeed, many passive comprehension tasks failed to elicit a response above the fixation baseline in the MD network, in contrast to strong responses in the language-selective network. Together, these results argue against a role for the MD network in core aspects of sentence comprehension, such as inhibiting irrelevant meanings or parses, keeping intermediate representations active in working memory, or predicting upcoming words or structures. These results align with recent evidence of relatively poor tracking of the linguistic signal by the MD regions during naturalistic comprehension, and instead suggest that the MD network's engagement during language processing reflects effort associated with extraneous task demands.SIGNIFICANCE STATEMENT Domain-general executive processes, such as working memory and cognitive control, have long been implicated in language comprehension, including in neuroimaging studies that have reported activation in domain-general multiple demand (MD) regions for linguistic manipulations. However, much prior evidence has come from paradigms where language interpretation is accompanied by extraneous tasks. Using a large fMRI dataset (30 experiments/481 participants/678 sessions), we demonstrate that MD regions are engaged during language comprehension in the presence of task demands, but not during passive reading/listening, conditions that strongly activate the frontotemporal language network. These results present a fundamental challenge to proposals whereby linguistic computations, such as inhibiting irrelevant meanings, keeping representations active in working memory, or predicting upcoming elements, draw on domain-general executive resources.
Collapse
Affiliation(s)
- Evgeniia Diachek
- Department of Psychology, Vanderbilt University, Nashville, Tennessee 37203
| | - Idan Blank
- Department of Psychology, University of California at Los Angeles, Los Angeles, California 90095
| | - Matthew Siegelman
- Department of Psychology, Columbia University, New York, New York 10027
| | - Josef Affourtit
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
| | - Evelina Fedorenko
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
- Department of Psychiatry, Massachusetts General Hospital, Charlestown, Massachusetts 02129
| |
Collapse
|
19
|
Schwering SC, MacDonald MC. Verbal Working Memory as Emergent from Language Comprehension and Production. Front Hum Neurosci 2020; 14:68. [PMID: 32226368 PMCID: PMC7081770 DOI: 10.3389/fnhum.2020.00068] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2019] [Accepted: 02/13/2020] [Indexed: 12/31/2022] Open
Abstract
This article reviews current models of verbal working memory and considers the role of language comprehension and long-term memory in the ability to maintain and order verbal information for short periods of time. While all models of verbal working memory posit some interaction with long-term memory, few have considered the character of these long-term representations or how they might affect performance on verbal working memory tasks. Similarly, few models have considered how comprehension processes and production processes might affect performance in verbal working memory tasks. Modern theories of comprehension emphasize that people learn a vast web of correlated information about the language and the world and must activate that information from long-term memory to cope with the demands of language input. To date, there has been little consideration in theories of verbal working memory for how this rich input from comprehension would affect the nature of temporary memory. There has also been relatively little attention to the degree to which language production processes naturally manage serial order of verbal information. The authors argue for an emergent model of verbal working memory supported by a rich, distributed long-term memory for language. On this view, comprehension processes provide encoding in verbal working memory tasks, and production processes maintenance, serial ordering, and recall. Moreover, the computational capacity to maintain and order information varies with language experience. Implications for theories of working memory, comprehension, and production are considered.
Collapse
|
20
|
Mollica F, Siegelman M, Diachek E, Piantadosi ST, Mineroff Z, Futrell R, Kean H, Qian P, Fedorenko E. Composition is the Core Driver of the Language-selective Network. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2020; 1:104-134. [PMID: 36794007 PMCID: PMC9923699 DOI: 10.1162/nol_a_00005] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/09/2019] [Accepted: 12/19/2019] [Indexed: 05/11/2023]
Abstract
The frontotemporal language network responds robustly and selectively to sentences. But the features of linguistic input that drive this response and the computations that these language areas support remain debated. Two key features of sentences are typically confounded in natural linguistic input: words in sentences (a) are semantically and syntactically combinable into phrase- and clause-level meanings, and (b) occur in an order licensed by the language's grammar. Inspired by recent psycholinguistic work establishing that language processing is robust to word order violations, we hypothesized that the core linguistic computation is composition, and, thus, can take place even when the word order violates the grammatical constraints of the language. This hypothesis predicts that a linguistic string should elicit a sentence-level response in the language network provided that the words in that string can enter into dependency relationships as in typical sentences. We tested this prediction across two fMRI experiments (total N = 47) by introducing a varying number of local word swaps into naturalistic sentences, leading to progressively less syntactically well-formed strings. Critically, local dependency relationships were preserved because combinable words remained close to each other. As predicted, word order degradation did not decrease the magnitude of the blood oxygen level-dependent response in the language network, except when combinable words were so far apart that composition among nearby words was highly unlikely. This finding demonstrates that composition is robust to word order violations, and that the language regions respond as strongly as they do to naturalistic linguistic input, providing that composition can take place.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Hope Kean
- Brain & Cognitive Sciences Department, MIT
| | - Peng Qian
- Brain & Cognitive Sciences Department, MIT
| | - Evelina Fedorenko
- Brain & Cognitive Sciences Department, MIT
- McGovern Institute for Brain Research, MIT
- Psychiatry Department, Massachusetts General Hospital
| |
Collapse
|