1
|
Giroud J, Trébuchon A, Mercier M, Davis MH, Morillon B. The human auditory cortex concurrently tracks syllabic and phonemic timescales via acoustic spectral flux. SCIENCE ADVANCES 2024; 10:eado8915. [PMID: 39705351 DOI: 10.1126/sciadv.ado8915] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Accepted: 11/15/2024] [Indexed: 12/22/2024]
Abstract
Dynamical theories of speech processing propose that the auditory cortex parses acoustic information in parallel at the syllabic and phonemic timescales. We developed a paradigm to independently manipulate both linguistic timescales, and acquired intracranial recordings from 11 patients who are epileptic listening to French sentences. Our results indicate that (i) syllabic and phonemic timescales are both reflected in the acoustic spectral flux; (ii) during comprehension, the auditory cortex tracks the syllabic timescale in the theta range, while neural activity in the alpha-beta range phase locks to the phonemic timescale; (iii) these neural dynamics occur simultaneously and share a joint spatial location; (iv) the spectral flux embeds two timescales-in the theta and low-beta ranges-across 17 natural languages. These findings help us understand how the human brain extracts acoustic information from the continuous speech signal at multiple timescales simultaneously, a prerequisite for subsequent linguistic processing.
Collapse
Affiliation(s)
- Jérémy Giroud
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
| | - Agnès Trébuchon
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des Systèmes, Marseille, France
- APHM, Clinical Neurophysiology, Timone Hospital, Marseille, France
| | - Manuel Mercier
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des Systèmes, Marseille, France
| | - Matthew H Davis
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, UK
| | - Benjamin Morillon
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des Systèmes, Marseille, France
| |
Collapse
|
2
|
Wang J, Wagley N, Rice M, Gaab N, Booth JR. Syntactic and semantic specialization in 9- to 10-year-old children during auditory sentence processing. Sci Rep 2024; 14:26965. [PMID: 39505932 PMCID: PMC11541780 DOI: 10.1038/s41598-024-76907-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Accepted: 10/17/2024] [Indexed: 11/08/2024] Open
Abstract
Prior literature has debated whether syntax is separable from semantics in the brain. Using functional magnetic resonance imaging and multi-voxel pattern analysis, our previous studies investigated brain activity during morpho-syntactic versus semantic processing. These studies only detected semantic specialization in activation patterns and no syntactic specialization in 5- to 6-year-old and 7- to 8-year-old children. To examine if older children who have mastered morpho-syntactic skills would show specialization for syntax, the current study examined 64 9- to 10-year-old children using the same design and analyses. We observed that only the left IFG pars opercularis was sensitive to syntactic but not semantic information, supporting the hypothesis that this region serves as a core region for syntax. In addition, the left STG which has been implicated in the integration of semantics and syntax, as well as the left MTG and IFG pars triangularis which have been implicated in semantics, were sensitive to both semantic and syntactic information with no evidence of specialization. These findings suggest a lexicalized view of syntax, which argues that semantically sensitive regions are also critical regions for syntactic processing during language comprehension.
Collapse
Affiliation(s)
- Jin Wang
- School of Education and Information Studies, University of California, Los Angeles, CA, USA.
| | - Neelima Wagley
- Speech and Hearing Sciences, Arizona State University, Tempe, AZ, USA
| | - Mabel Rice
- Child Language Doctoral Program, University of Kansas, Lawrence, KS, USA
| | - Nadine Gaab
- Graduate School of Education, Harvard University, Cambridge, MA, USA
| | - James R Booth
- Department of Psychology and Human Development, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
3
|
Murphy E, Rollo PS, Segaert K, Hagoort P, Tandon N. Multiple dimensions of syntactic structure are resolved earliest in posterior temporal cortex. Prog Neurobiol 2024; 241:102669. [PMID: 39332803 DOI: 10.1016/j.pneurobio.2024.102669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2024] [Revised: 08/08/2024] [Accepted: 09/24/2024] [Indexed: 09/29/2024]
Abstract
How we combine minimal linguistic units into larger structures remains an unresolved topic in neuroscience. Language processing involves the abstract construction of 'vertical' and 'horizontal' information simultaneously (e.g., phrase structure, morphological agreement), but previous paradigms have been constrained in isolating only one type of composition and have utilized poor spatiotemporal resolution. Using intracranial recordings, we report multiple experiments designed to separate phrase structure from morphosyntactic agreement. Epilepsy patients (n = 10) were presented with auditory two-word phrases grouped into pseudoword-verb ('trab run') and pronoun-verb either with or without Person agreement ('they run' vs. 'they runs'). Phrase composition and Person violations both resulted in significant increases in broadband high gamma activity approximately 300 ms after verb onset in posterior middle temporal gyrus (pMTG) and posterior superior temporal sulcus (pSTS), followed by inferior frontal cortex (IFC) at 500 ms. While sites sensitive to only morphosyntactic violations were distributed, those sensitive to both composition types were generally confined to pSTS/pMTG and IFC. These results indicate that posterior temporal cortex shows the earliest sensitivity for hierarchical linguistic structure across multiple dimensions, providing neural resources for distinct windows of composition. This region is comprised of sparsely interwoven heterogeneous constituents that afford cortical search spaces for dissociable syntactic relations.
Collapse
Affiliation(s)
- Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States; Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States.
| | - Patrick S Rollo
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States; Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States
| | - Katrien Segaert
- School of Psychology & Centre for Human Brain Health, University of Birmingham, Birmingham B15 2TT, UK; Max Planck Institute for Psycholinguistics, Nijmegen 6525 XD, the Netherlands
| | - Peter Hagoort
- Max Planck Institute for Psycholinguistics, Nijmegen 6525 XD, the Netherlands; Donders Institute for Brain, Cognition and Behaviour, Nijmegen 6525 HR, the Netherlands
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States; Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States; Memorial Hermann Hospital, Texas Medical Center, Houston, TX 77030, United States.
| |
Collapse
|
4
|
Kumar S, Sumers TR, Yamakoshi T, Goldstein A, Hasson U, Norman KA, Griffiths TL, Hawkins RD, Nastase SA. Shared functional specialization in transformer-based language models and the human brain. Nat Commun 2024; 15:5523. [PMID: 38951520 PMCID: PMC11217339 DOI: 10.1038/s41467-024-49173-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Accepted: 05/24/2024] [Indexed: 07/03/2024] Open
Abstract
When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations ("embeddings") generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized "transformations" that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized "attention heads" differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.
Collapse
Affiliation(s)
- Sreejan Kumar
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA.
| | - Theodore R Sumers
- Department of Computer Science, Princeton University, Princeton, NJ, 08540, USA.
| | - Takateru Yamakoshi
- Faculty of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Ariel Goldstein
- Department of Cognitive and Brain Sciences and Business School, Hebrew University, Jerusalem, 9190401, Israel
| | - Uri Hasson
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA
| | - Kenneth A Norman
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA
| | - Thomas L Griffiths
- Department of Computer Science, Princeton University, Princeton, NJ, 08540, USA
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA
| | - Robert D Hawkins
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA
| | - Samuel A Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA.
| |
Collapse
|
5
|
Fahey D, Fridriksson J, Hickok G, Matchin W. Lesion-symptom Mapping of Acceptability Judgments in Chronic Poststroke Aphasia Reveals the Neurobiological Underpinnings of Receptive Syntax. J Cogn Neurosci 2024; 36:1141-1155. [PMID: 38437175 PMCID: PMC11095916 DOI: 10.1162/jocn_a_02134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2024]
Abstract
Disagreements persist regarding the neural basis of syntactic processing, which has been linked both to inferior frontal and posterior temporal regions of the brain. One focal point of the debate concerns the role of inferior frontal areas in receptive syntactic ability, which is mostly assessed using sentence comprehension involving complex syntactic structures, a task that is potentially confounded with working memory. Syntactic acceptability judgments may provide a better measure of receptive syntax by reducing the need to use high working memory load and complex sentences and by enabling assessment of various types of syntactic violations. We therefore tested the perception of grammatical violations by people with poststroke aphasia (n = 25), along with matched controls (n = 16), using English sentences involving errors in word order, agreement, or subcategorization. Lesion data were also collected. Control participants performed near ceiling in accuracy with higher discriminability of agreement and subcategorization violations than word order; aphasia participants were less able to discriminate violations, but, on average, paralleled control participants discriminability of types of violations. Lesion-symptom mapping showed a correlation between discriminability and posterior temporal regions, but not inferior frontal regions. We argue that these results diverge from models holding that frontal areas are amodal core regions in syntactic structure building and favor models that posit a core hierarchical system in posterior temporal regions.
Collapse
|
6
|
Murphy E, Holmes E, Friston K. Natural language syntax complies with the free-energy principle. SYNTHESE 2024; 203:154. [PMID: 38706520 PMCID: PMC11068586 DOI: 10.1007/s11229-024-04566-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Accepted: 03/15/2024] [Indexed: 05/07/2024]
Abstract
Natural language syntax yields an unbounded array of hierarchically structured expressions. We claim that these are used in the service of active inference in accord with the free-energy principle (FEP). While conceptual advances alongside modelling and simulation work have attempted to connect speech segmentation and linguistic communication with the FEP, we extend this program to the underlying computations responsible for generating syntactic objects. We argue that recently proposed principles of economy in language design-such as "minimal search" criteria from theoretical syntax-adhere to the FEP. This affords a greater degree of explanatory power to the FEP-with respect to higher language functions-and offers linguistics a grounding in first principles with respect to computability. While we mostly focus on building new principled conceptual relations between syntax and the FEP, we also show through a sample of preliminary examples how both tree-geometric depth and a Kolmogorov complexity estimate (recruiting a Lempel-Ziv compression algorithm) can be used to accurately predict legal operations on syntactic workspaces, directly in line with formulations of variational free energy minimization. This is used to motivate a general principle of language design that we term Turing-Chomsky Compression (TCC). We use TCC to align concerns of linguists with the normative account of self-organization furnished by the FEP, by marshalling evidence from theoretical linguistics and psycholinguistics to ground core principles of efficient syntactic computation within active inference.
Collapse
Affiliation(s)
- Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center, Houston, TX 77030 USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center, Houston, TX 77030 USA
| | - Emma Holmes
- Department of Speech Hearing and Phonetic Sciences, University College London, London, WC1N 1PF UK
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, London, WC1N 3AR UK
| | - Karl Friston
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, London, WC1N 3AR UK
| |
Collapse
|
7
|
Sugimoto Y, Yoshida R, Jeong H, Koizumi M, Brennan JR, Oseki Y. Localizing Syntactic Composition with Left-Corner Recurrent Neural Network Grammars. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2024; 5:201-224. [PMID: 38645619 PMCID: PMC11025653 DOI: 10.1162/nol_a_00118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Accepted: 07/24/2023] [Indexed: 04/23/2024]
Abstract
In computational neurolinguistics, it has been demonstrated that hierarchical models such as recurrent neural network grammars (RNNGs), which jointly generate word sequences and their syntactic structures via the syntactic composition, better explained human brain activity than sequential models such as long short-term memory networks (LSTMs). However, the vanilla RNNG has employed the top-down parsing strategy, which has been pointed out in the psycholinguistics literature as suboptimal especially for head-final/left-branching languages, and alternatively the left-corner parsing strategy has been proposed as the psychologically plausible parsing strategy. In this article, building on this line of inquiry, we investigate not only whether hierarchical models like RNNGs better explain human brain activity than sequential models like LSTMs, but also which parsing strategy is more neurobiologically plausible, by developing a novel fMRI corpus where participants read newspaper articles in a head-final/left-branching language, namely Japanese, through the naturalistic fMRI experiment. The results revealed that left-corner RNNGs outperformed both LSTMs and top-down RNNGs in the left inferior frontal and temporal-parietal regions, suggesting that there are certain brain regions that localize the syntactic composition with the left-corner parsing strategy.
Collapse
Affiliation(s)
- Yushi Sugimoto
- Graduate School of Arts and Sciences, University of Tokyo, Tokyo, Japan
| | - Ryo Yoshida
- Graduate School of Arts and Sciences, University of Tokyo, Tokyo, Japan
| | - Hyeonjeong Jeong
- Graduate School of International Cultural Studies, Tohoku University, Sendai, Japan
| | - Masatoshi Koizumi
- Department of Linguistics, Graduate School of Arts and Letters, Tohoku University, Sendai, Japan
| | | | - Yohei Oseki
- Graduate School of Arts and Sciences, University of Tokyo, Tokyo, Japan
| |
Collapse
|
8
|
Zioga I, Zhou YJ, Weissbart H, Martin AE, Haegens S. Alpha and Beta Oscillations Differentially Support Word Production in a Rule-Switching Task. eNeuro 2024; 11:ENEURO.0312-23.2024. [PMID: 38490743 PMCID: PMC10988358 DOI: 10.1523/eneuro.0312-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 01/26/2024] [Accepted: 02/22/2024] [Indexed: 03/17/2024] Open
Abstract
Research into the role of brain oscillations in basic perceptual and cognitive functions has suggested that the alpha rhythm reflects functional inhibition while the beta rhythm reflects neural ensemble (re)activation. However, little is known regarding the generalization of these proposed fundamental operations to linguistic processes, such as speech comprehension and production. Here, we recorded magnetoencephalography in participants performing a novel rule-switching paradigm. Specifically, Dutch native speakers had to produce an alternative exemplar from the same category or a feature of a given target word embedded in spoken sentences (e.g., for the word "tuna", an exemplar from the same category-"seafood"-would be "shrimp", and a feature would be "pink"). A cue indicated the task rule-exemplar or feature-either before (pre-cue) or after (retro-cue) listening to the sentence. Alpha power during the working memory delay was lower for retro-cue compared with that for pre-cue in the left hemispheric language-related regions. Critically, alpha power negatively correlated with reaction times, suggestive of alpha facilitating task performance by regulating inhibition in regions linked to lexical retrieval. Furthermore, we observed a different spatiotemporal pattern of beta activity for exemplars versus features in the right temporoparietal regions, in line with the proposed role of beta in recruiting neural networks for the encoding of distinct categories. Overall, our study provides evidence for the generalizability of the role of alpha and beta oscillations from perceptual to more "complex, linguistic processes" and offers a novel task to investigate links between rule-switching, working memory, and word production.
Collapse
Affiliation(s)
- Ioanna Zioga
- Donders Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen 6525 EN, The Netherlands
- Max Planck Institute for Psycholinguistics, Nijmegen 6525 XD, The Netherlands
| | - Ying Joey Zhou
- Donders Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen 6525 EN, The Netherlands
- Department of Psychiatry, Oxford Centre for Human Brain Activity, Oxford, United Kingdom
| | - Hugo Weissbart
- Donders Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen 6525 EN, The Netherlands
| | - Andrea E Martin
- Donders Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen 6525 EN, The Netherlands
- Max Planck Institute for Psycholinguistics, Nijmegen 6525 XD, The Netherlands
| | - Saskia Haegens
- Donders Centre for Cognitive Neuroimaging, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen 6525 EN, The Netherlands
- Department of Psychiatry, Columbia University, New York, New York 10032
- Division of Systems Neuroscience, New York State Psychiatric Institute, New York, New York 10032
| |
Collapse
|
9
|
Desbordes T, King JR, Dehaene S. Tracking the neural codes for words and phrases during semantic composition, working-memory storage, and retrieval. Cell Rep 2024; 43:113847. [PMID: 38412098 DOI: 10.1016/j.celrep.2024.113847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 11/02/2023] [Accepted: 02/07/2024] [Indexed: 02/29/2024] Open
Abstract
The ability to compose successive words into a meaningful phrase is a characteristic feature of human cognition, yet its neural mechanisms remain incompletely understood. Here, we analyze the cortical mechanisms of semantic composition using magnetoencephalography (MEG) while participants read one-word, two-word, and five-word noun phrases and compared them with a subsequent image. Decoding of MEG signals revealed three processing stages. During phrase comprehension, the representation of individual words was sustained for a variable duration depending on phrasal context. During the delay period, the word code was replaced by a working-memory code whose activation increased with semantic complexity. Finally, the speed and accuracy of retrieval depended on semantic complexity and was faster for surface than for deep semantic properties. In conclusion, we propose that the brain initially encodes phrases using factorized dimensions for successive words but later compresses them in working memory and requires a period of decompression to access them.
Collapse
Affiliation(s)
- Théo Desbordes
- Meta AI, Paris, France; Cognitive Neuroimaging Unit, NeuroSpin Center, 91191 Gif-sur-Yvette, France.
| | - Jean-Rémi King
- Meta AI, Paris, France; École Normale Supérieure, PSL University, Paris, France
| | - Stanislas Dehaene
- Université Paris Saclay, INSERM, CEA, Cognitive Neuroimaging Unit, NeuroSpin Center, 91191 Gif-sur-Yvette, France; Collège de France, PSL University, Paris, France
| |
Collapse
|
10
|
Bruera A, Tao Y, Anderson A, Çokal D, Haber J, Poesio M. Modeling Brain Representations of Words' Concreteness in Context Using GPT-2 and Human Ratings. Cogn Sci 2023; 47:e13388. [PMID: 38103208 DOI: 10.1111/cogs.13388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 09/12/2023] [Accepted: 10/27/2023] [Indexed: 12/18/2023]
Abstract
The meaning of most words in language depends on their context. Understanding how the human brain extracts contextualized meaning, and identifying where in the brain this takes place, remain important scientific challenges. But technological and computational advances in neuroscience and artificial intelligence now provide unprecedented opportunities to study the human brain in action as language is read and understood. Recent contextualized language models seem to be able to capture homonymic meaning variation ("bat", in a baseball vs. a vampire context), as well as more nuanced differences of meaning-for example, polysemous words such as "book", which can be interpreted in distinct but related senses ("explain a book", information, vs. "open a book", object) whose differences are fine-grained. We study these subtle differences in lexical meaning along the concrete/abstract dimension, as they are triggered by verb-noun semantic composition. We analyze functional magnetic resonance imaging (fMRI) activations elicited by Italian verb phrases containing nouns whose interpretation is affected by the verb to different degrees. By using a contextualized language model and human concreteness ratings, we shed light on where in the brain such fine-grained meaning variation takes place and how it is coded. Our results show that phrase concreteness judgments and the contextualized model can predict BOLD activation associated with semantic composition within the language network. Importantly, representations derived from a complex, nonlinear composition process consistently outperform simpler composition approaches. This is compatible with a holistic view of semantic composition in the brain, where semantic representations are modified by the process of composition itself. When looking at individual brain areas, we find that encoding performance is statistically significant, although with differing patterns of results, suggesting differential involvement, in the posterior superior temporal sulcus, inferior frontal gyrus and anterior temporal lobe, and in motor areas previously associated with processing of concreteness/abstractness.
Collapse
Affiliation(s)
- Andrea Bruera
- School of Electronic Engineering and Computer Science, Cognitive Science Research Group, Queen Mary University of London
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences
| | - Yuan Tao
- Department of Cognitive Science, Johns Hopkins University
| | | | - Derya Çokal
- Department of German Language and Literature I-Linguistics, University of Cologne
| | - Janosch Haber
- School of Electronic Engineering and Computer Science, Cognitive Science Research Group, Queen Mary University of London
- Chattermill, London
| | - Massimo Poesio
- School of Electronic Engineering and Computer Science, Cognitive Science Research Group, Queen Mary University of London
- Department of Information and Computing Sciences, University of Utrecht
| |
Collapse
|
11
|
Schroën JAM, Gunter TC, Numssen O, Kroczek LOH, Hartwigsen G, Friederici AD. Causal evidence for a coordinated temporal interplay within the language network. Proc Natl Acad Sci U S A 2023; 120:e2306279120. [PMID: 37963247 PMCID: PMC10666120 DOI: 10.1073/pnas.2306279120] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 10/06/2023] [Indexed: 11/16/2023] Open
Abstract
Recent neurobiological models on language suggest that auditory sentence comprehension is supported by a coordinated temporal interplay within a left-dominant brain network, including the posterior inferior frontal gyrus (pIFG), posterior superior temporal gyrus and sulcus (pSTG/STS), and angular gyrus (AG). Here, we probed the timing and causal relevance of the interplay between these regions by means of concurrent transcranial magnetic stimulation and electroencephalography (TMS-EEG). Our TMS-EEG experiments reveal region- and time-specific causal evidence for a bidirectional information flow from left pSTG/STS to left pIFG and back during auditory sentence processing. Adapting a condition-and-perturb approach, our findings further suggest that the left pSTG/STS can be supported by the left AG in a state-dependent manner.
Collapse
Affiliation(s)
- Joëlle A. M. Schroën
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Thomas C. Gunter
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Ole Numssen
- Methods and Development Group Brain Networks, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Leon O. H. Kroczek
- Department of Psychology, Clinical Psychology and Psychotherapy, Universität Regensburg, Regensburg93053, Germany
| | - Gesa Hartwigsen
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
- Cognitive and Biological Psychology, Wilhelm Wundt Institute for Psychology, Leipzig04109, Germany
| | - Angela D. Friederici
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| |
Collapse
|
12
|
Matchin W, den Ouden DB, Basilakos A, Stark BC, Fridriksson J, Hickok G. Grammatical Parallelism in Aphasia: A Lesion-Symptom Mapping Study. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2023; 4:550-574. [PMID: 37946730 PMCID: PMC10631800 DOI: 10.1162/nol_a_00117] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 07/19/2023] [Indexed: 11/12/2023]
Abstract
Sentence structure, or syntax, is potentially a uniquely creative aspect of the human mind. Neuropsychological experiments in the 1970s suggested parallel syntactic production and comprehension deficits in agrammatic Broca's aphasia, thought to result from damage to syntactic mechanisms in Broca's area in the left frontal lobe. This hypothesis was sometimes termed overarching agrammatism, converging with developments in linguistic theory concerning central syntactic mechanisms supporting language production and comprehension. However, the evidence supporting an association among receptive syntactic deficits, expressive agrammatism, and damage to frontal cortex is equivocal. In addition, the relationship among a distinct grammatical production deficit in aphasia, paragrammatism, and receptive syntax has not been assessed. We used lesion-symptom mapping in three partially overlapping groups of left-hemisphere stroke patients to investigate these issues: grammatical production deficits in a primary group of 53 subjects and syntactic comprehension in larger sample sizes (N = 130, 218) that overlapped with the primary group. Paragrammatic production deficits were significantly associated with multiple analyses of syntactic comprehension, particularly when incorporating lesion volume as a covariate, but agrammatic production deficits were not. The lesion correlates of impaired performance of syntactic comprehension were significantly associated with damage to temporal lobe regions, which were also implicated in paragrammatism, but not with the inferior and middle frontal regions implicated in expressive agrammatism. Our results provide strong evidence against the overarching agrammatism hypothesis. By contrast, our results suggest the possibility of an alternative grammatical parallelism hypothesis rooted in paragrammatism and a central syntactic system in the posterior temporal lobe.
Collapse
Affiliation(s)
- William Matchin
- Department of Communication Sciences and Disorders, University of South Carolina, Columbia, SC, USA
| | - Dirk-Bart den Ouden
- Department of Communication Sciences and Disorders, University of South Carolina, Columbia, SC, USA
| | - Alexandra Basilakos
- Department of Communication Sciences and Disorders, University of South Carolina, Columbia, SC, USA
| | - Brielle Caserta Stark
- Department of Speech, Language and Hearing Sciences, Program for Neuroscience, Indiana University Bloomington, Bloomington, IN, USA
| | - Julius Fridriksson
- Department of Communication Sciences and Disorders, University of South Carolina, Columbia, SC, USA
| | - Gregory Hickok
- Department of Cognitive Sciences, Department of Language Science, University of California, Irvine, Irvine, CA, USA
| |
Collapse
|
13
|
Murphy E, Forseth KJ, Donos C, Snyder KM, Rollo PS, Tandon N. The spatiotemporal dynamics of semantic integration in the human brain. Nat Commun 2023; 14:6336. [PMID: 37875526 PMCID: PMC10598228 DOI: 10.1038/s41467-023-42087-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Accepted: 09/28/2023] [Indexed: 10/26/2023] Open
Abstract
Language depends critically on the integration of lexical information across multiple words to derive semantic concepts. Limitations of spatiotemporal resolution have previously rendered it difficult to isolate processes involved in semantic integration. We utilized intracranial recordings in epilepsy patients (n = 58) who read written word definitions. Descriptions were either referential or non-referential to a common object. Semantically referential sentences enabled high frequency broadband gamma activation (70-150 Hz) of the inferior frontal sulcus (IFS), medial parietal cortex, orbitofrontal cortex (OFC) and medial temporal lobe in the left, language-dominant hemisphere. IFS, OFC and posterior middle temporal gyrus activity was modulated by the semantic coherence of non-referential sentences, exposing semantic effects that were independent of task-based referential status. Components of this network, alongside posterior superior temporal sulcus, were engaged for referential sentences that did not clearly reduce the lexical search space by the final word. These results indicate the existence of complementary cortical mosaics for semantic integration in posterior temporal and inferior frontal cortex.
Collapse
Affiliation(s)
- Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA.
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA.
| | - Kiefer J Forseth
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA
| | - Cristian Donos
- Faculty of Physics, University of Bucharest, Măgurele, 077125, Bucharest, Romania
| | - Kathryn M Snyder
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA
| | - Patrick S Rollo
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA.
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX, 77030, USA.
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX, 77030, USA.
| |
Collapse
|
14
|
Wang J, Wang X, Zou J, Duan J, Shen Z, Xu N, Chen Y, Zhang J, He H, Bi Y, Ding N. Neural substrate underlying the learning of a passage with unfamiliar vocabulary and syntax. Cereb Cortex 2023; 33:10036-10046. [PMID: 37491998 DOI: 10.1093/cercor/bhad263] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 06/26/2023] [Accepted: 06/27/2023] [Indexed: 07/27/2023] Open
Abstract
Speech comprehension is a complex process involving multiple stages, such as decoding of phonetic units, recognizing words, and understanding sentences and passages. In this study, we identify cortical networks beyond basic phonetic processing using a novel passage learning paradigm. Participants learn to comprehend a story composed of syllables of their native language, but containing unfamiliar vocabulary and syntax. Three learning methods are employed, each resulting in some degree of learning within a 12-min learning session. Functional magnetic resonance imaging results reveal that, when listening to the same story, the classic temporal-frontal language network is significantly enhanced by learning. Critically, activation of the left anterior and posterior temporal lobe correlates with the learning outcome that is assessed behaviorally through, e.g. word recognition and passage comprehension tests. This study demonstrates that a brief learning session is sufficient to induce neural plasticity in the left temporal lobe, which underlies the transformation from phonetic units to the units of meaning, such as words and sentences.
Collapse
Affiliation(s)
- Jing Wang
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Xiaosha Wang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Jiajie Zou
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Jipeng Duan
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Zhuowen Shen
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Nannan Xu
- School of Linguistic Sciences and Arts, Jiangsu Normal University, Xuzhou 221009, China
| | - Yan Chen
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Jianfeng Zhang
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Hongjian He
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
| | - Yanchao Bi
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China
| | - Nai Ding
- Key Laboratory for Biomedical Engineering of Ministry of Education, Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou 310027, China
- MOE Frontier Science Center for Brain Science & Brain-machine Integration, Zhejiang University, Hangzhou 310027, China
| |
Collapse
|
15
|
Thomas TM, Singh A, Bullock LP, Liang D, Morse CW, Scherschligt X, Seymour JP, Tandon N. Decoding articulatory and phonetic components of naturalistic continuous speech from the distributed language network. J Neural Eng 2023; 20:046030. [PMID: 37487487 DOI: 10.1088/1741-2552/ace9fb] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 07/24/2023] [Indexed: 07/26/2023]
Abstract
Objective.The speech production network relies on a widely distributed brain network. However, research and development of speech brain-computer interfaces (speech-BCIs) has typically focused on decoding speech only from superficial subregions readily accessible by subdural grid arrays-typically placed over the sensorimotor cortex. Alternatively, the technique of stereo-electroencephalography (sEEG) enables access to distributed brain regions using multiple depth electrodes with lower surgical risks, especially in patients with brain injuries resulting in aphasia and other speech disorders.Approach.To investigate the decoding potential of widespread electrode coverage in multiple cortical sites, we used a naturalistic continuous speech production task. We obtained neural recordings using sEEG from eight participants while they read aloud sentences. We trained linear classifiers to decode distinct speech components (articulatory components and phonemes) solely based on broadband gamma activity and evaluated the decoding performance using nested five-fold cross-validation.Main Results.We achieved an average classification accuracy of 18.7% across 9 places of articulation (e.g. bilabials, palatals), 26.5% across 5 manner of articulation (MOA) labels (e.g. affricates, fricatives), and 4.81% across 38 phonemes. The highest classification accuracies achieved with a single large dataset were 26.3% for place of articulation, 35.7% for MOA, and 9.88% for phonemes. Electrodes that contributed high decoding power were distributed across multiple sulcal and gyral sites in both dominant and non-dominant hemispheres, including ventral sensorimotor, inferior frontal, superior temporal, and fusiform cortices. Rather than finding a distinct cortical locus for each speech component, we observed neural correlates of both articulatory and phonetic components in multiple hubs of a widespread language production network.Significance.These results reveal the distributed cortical representations whose activity can enable decoding speech components during continuous speech through the use of this minimally invasive recording method, elucidating language neurobiology and neural targets for future speech-BCIs.
Collapse
Affiliation(s)
- Tessy M Thomas
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
| | - Aditya Singh
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
| | - Latané P Bullock
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
| | - Daniel Liang
- Department of Computer Science, Rice University, Houston, TX 77005, United States of America
| | - Cale W Morse
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
| | - Xavier Scherschligt
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
| | - John P Seymour
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Department of Electrical & Computer Engineering, Rice University, Houston, TX 77005, United States of America
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, United States of America
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX 77030, United States of America
| |
Collapse
|
16
|
McCarty MJ, Murphy E, Scherschligt X, Woolnough O, Morse CW, Snyder K, Mahon BZ, Tandon N. Intraoperative cortical localization of music and language reveals signatures of structural complexity in posterior temporal cortex. iScience 2023; 26:107223. [PMID: 37485361 PMCID: PMC10362292 DOI: 10.1016/j.isci.2023.107223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 06/01/2023] [Accepted: 06/22/2023] [Indexed: 07/25/2023] Open
Abstract
Language and music involve the productive combination of basic units into structures. It remains unclear whether brain regions sensitive to linguistic and musical structure are co-localized. We report an intraoperative awake craniotomy in which a left-hemispheric language-dominant professional musician underwent cortical stimulation mapping (CSM) and electrocorticography of music and language perception and production during repetition tasks. Musical sequences were melodic or amelodic, and differed in algorithmic compressibility (Lempel-Ziv complexity). Auditory recordings of sentences differed in syntactic complexity (single vs. multiple phrasal embeddings). CSM of posterior superior temporal gyrus (pSTG) disrupted music perception and production, along with speech production. pSTG and posterior middle temporal gyrus (pMTG) activated for language and music (broadband gamma; 70-150 Hz). pMTG activity was modulated by musical complexity, while pSTG activity was modulated by syntactic complexity. This points to shared resources for music and language comprehension, but distinct neural signatures for the processing of domain-specific structural features.
Collapse
Affiliation(s)
- Meredith J. McCarty
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Xavier Scherschligt
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Oscar Woolnough
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Cale W. Morse
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Kathryn Snyder
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Bradford Z. Mahon
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX 77030, USA
| |
Collapse
|
17
|
Stanojević M, Brennan JR, Dunagan D, Steedman M, Hale JT. Modeling Structure-Building in the Brain With CCG Parsing and Large Language Models. Cogn Sci 2023; 47:e13312. [PMID: 37417470 DOI: 10.1111/cogs.13312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Revised: 06/07/2023] [Accepted: 06/17/2023] [Indexed: 07/08/2023]
Abstract
To model behavioral and neural correlates of language comprehension in naturalistic environments, researchers have turned to broad-coverage tools from natural-language processing and machine learning. Where syntactic structure is explicitly modeled, prior work has relied predominantly on context-free grammars (CFGs), yet such formalisms are not sufficiently expressive for human languages. Combinatory categorial grammars (CCGs) are sufficiently expressive directly compositional models of grammar with flexible constituency that affords incremental interpretation. In this work, we evaluate whether a more expressive CCG provides a better model than a CFG for human neural signals collected with functional magnetic resonance imaging (fMRI) while participants listen to an audiobook story. We further test between variants of CCG that differ in how they handle optional adjuncts. These evaluations are carried out against a baseline that includes estimates of next-word predictability from a transformer neural network language model. Such a comparison reveals unique contributions of CCG structure-building predominantly in the left posterior temporal lobe: CCG-derived measures offer a superior fit to neural signals compared to those derived from a CFG. These effects are spatially distinct from bilateral superior temporal effects that are unique to predictability. Neural effects for structure-building are thus separable from predictability during naturalistic listening, and those effects are best characterized by a grammar whose expressive power is motivated on independent linguistic grounds.
Collapse
Affiliation(s)
| | | | | | | | - John T Hale
- Google DeepMind
- Department of Linguistics, University of Georgia
| |
Collapse
|
18
|
Woolnough O, Donos C, Murphy E, Rollo PS, Roccaforte ZJ, Dehaene S, Tandon N. Spatiotemporally distributed frontotemporal networks for sentence reading. Proc Natl Acad Sci U S A 2023; 120:e2300252120. [PMID: 37068244 PMCID: PMC10151604 DOI: 10.1073/pnas.2300252120] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 03/14/2023] [Indexed: 04/19/2023] Open
Abstract
Reading a sentence entails integrating the meanings of individual words to infer more complex, higher-order meaning. This highly rapid and complex human behavior is known to engage the inferior frontal gyrus (IFG) and middle temporal gyrus (MTG) in the language-dominant hemisphere, yet whether there are distinct contributions of these regions to sentence reading is still unclear. To probe these neural spatiotemporal dynamics, we used direct intracranial recordings to measure neural activity while reading sentences, meaning-deficient Jabberwocky sentences, and lists of words or pseudowords. We isolated two functionally and spatiotemporally distinct frontotemporal networks, each sensitive to distinct aspects of word and sentence composition. The first distributed network engages the IFG and MTG, with IFG activity preceding MTG. Activity in this network ramps up over the duration of a sentence and is reduced or absent during Jabberwocky and word lists, implying its role in the derivation of sentence-level meaning. The second network engages the superior temporal gyrus and the IFG, with temporal responses leading those in frontal lobe, and shows greater activation for each word in a list than those in sentences, suggesting that sentential context enables greater efficiency in the lexical and/or phonological processing of individual words. These adjacent, yet spatiotemporally dissociable neural mechanisms for word- and sentence-level processes shed light on the richly layered semantic networks that enable us to fluently read. These results imply distributed, dynamic computation across the frontotemporal language network rather than a clear dichotomy between the contributions of frontal and temporal structures.
Collapse
Affiliation(s)
- Oscar Woolnough
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX77030
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX77030
| | - Cristian Donos
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX77030
- Faculty of Physics, University of Bucharest, 050663Bucharest, Romania
| | - Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX77030
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX77030
| | - Patrick S. Rollo
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX77030
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX77030
| | - Zachary J. Roccaforte
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX77030
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX77030
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, Université Paris-Saclay, INSERM, CEA, NeuroSpin Center, 91191Gif-sur-Yvette, France
- Collège de France, 75005Paris, France
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School at UT Health Houston, Houston, TX77030
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX77030
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX77030
| |
Collapse
|
19
|
Murphy E. ROSE: A Neurocomputational Architecture for Syntax. ARXIV 2023:arXiv:2303.08877v1. [PMID: 36994166 PMCID: PMC10055479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
Abstract
A comprehensive model of natural language processing in the brain must accommodate four components: representations, operations, structures and encoding. It further requires a principled account of how these different components mechanistically, and causally, relate to each another. While previous models have isolated regions of interest for structure-building and lexical access, and have utilized specific neural recording measures to expose possible signatures of syntax, many gaps remain with respect to bridging distinct scales of analysis that map onto these four components. By expanding existing accounts of how neural oscillations can index various linguistic processes, this article proposes a neurocomputational architecture for syntax, termed the ROSE model (Representation, Operation, Structure, Encoding). Under ROSE, the basic data structures of syntax are atomic features, types of mental representations (R), and are coded at the single-unit and ensemble level. Elementary computations (O) that transform these units into manipulable objects accessible to subsequent structure-building levels are coded via high frequency broadband γ activity. Low frequency synchronization and cross-frequency coupling code for recursive categorial inferences (S). Distinct forms of low frequency coupling and phase-amplitude coupling (δ-θ coupling via pSTS-IFG; θ-γ coupling via IFG to conceptual hubs in lateral and ventral temporal cortex) then encode these structures onto distinct workspaces (E). Causally connecting R to O is spike-phase/LFP coupling; connecting O to S is phase-amplitude coupling; connecting S to E is a system of frontotemporal traveling oscillations; connecting E back to lower levels is low-frequency phase resetting of spike-LFP coupling. This compositional neural code has important implications for algorithmic accounts, since it makes concrete predictions for the appropriate level of study for psycholinguistic parsing models. ROSE is reliant on neurophysiologically plausible mechanisms, is supported at all four levels by a range of recent empirical research, and provides an anatomically precise and falsifiable grounding for the basic property of natural language syntax: hierarchical, recursive structure-building.
Collapse
Affiliation(s)
- Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, UTHealth, Houston, TX, USA
- Texas Institute for Restorative Neurotechnologies, UTHealth, Houston, TX, USA
| |
Collapse
|
20
|
Syntax through the looking glass: A review on two-word linguistic processing across behavioral, neuroimaging and neurostimulation studies. Neurosci Biobehav Rev 2022; 142:104881. [DOI: 10.1016/j.neubiorev.2022.104881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Revised: 09/13/2022] [Accepted: 09/15/2022] [Indexed: 11/23/2022]
|
21
|
Cometa A, Falasconi A, Biasizzo M, Carpaneto J, Horn A, Mazzoni A, Micera S. Clinical neuroscience and neurotechnology: An amazing symbiosis. iScience 2022; 25:105124. [PMID: 36193050 PMCID: PMC9526189 DOI: 10.1016/j.isci.2022.105124] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
In the last decades, clinical neuroscience found a novel ally in neurotechnologies, devices able to record and stimulate electrical activity in the nervous system. These technologies improved the ability to diagnose and treat neural disorders. Neurotechnologies are concurrently enabling a deeper understanding of healthy and pathological dynamics of the nervous system through stimulation and recordings during brain implants. On the other hand, clinical neurosciences are not only driving neuroengineering toward the most relevant clinical issues, but are also shaping the neurotechnologies thanks to clinical advancements. For instance, understanding the etiology of a disease informs the location of a therapeutic stimulation, but also the way stimulation patterns should be designed to be more effective/naturalistic. Here, we describe cases of fruitful integration such as Deep Brain Stimulation and cortical interfaces to highlight how this symbiosis between clinical neuroscience and neurotechnology is closer to a novel integrated framework than to a simple interdisciplinary interaction.
Collapse
Affiliation(s)
- Andrea Cometa
- The Biorobotics Institute, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, 56127 Pisa, Italy
| | - Antonio Falasconi
- Friedrich Miescher Institute for Biomedical Research, 4058 Basel, Switzerland
- Biozentrum, University of Basel, 4056 Basel, Switzerland
| | - Marco Biasizzo
- The Biorobotics Institute, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, 56127 Pisa, Italy
| | - Jacopo Carpaneto
- The Biorobotics Institute, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, 56127 Pisa, Italy
| | - Andreas Horn
- Center for Brain Circuit Therapeutics Department of Neurology Brigham & Women’s Hospital, Harvard Medical School, Boston, MA 02115, USA
- MGH Neurosurgery & Center for Neurotechnology and Neurorecovery (CNTR) at MGH Neurology Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Movement Disorder and Neuromodulation Unit, Department of Neurology, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt- Universität zu Berlin, Department of Neurology, 10117 Berlin, Germany
| | - Alberto Mazzoni
- The Biorobotics Institute, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, 56127 Pisa, Italy
| | - Silvestro Micera
- The Biorobotics Institute, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, 56127 Pisa, Italy
- Translational Neural Engineering Lab, School of Engineering, École Polytechnique Fèdèrale de Lausanne, 1015 Lausanne, Switzerland
| |
Collapse
|
22
|
Integrity of the Left Arcuate Fasciculus Segments Significantly Affects Language Performance in Individuals with Acute/Subacute Post-Stroke Aphasia: A Cross-Sectional Diffusion Tensor Imaging Study. Brain Sci 2022; 12:brainsci12070907. [PMID: 35884714 PMCID: PMC9313217 DOI: 10.3390/brainsci12070907] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 07/05/2022] [Accepted: 07/08/2022] [Indexed: 01/05/2023] Open
Abstract
Objective: To investigate the correlation between the left arcuate fasciculus (AF) segments and acute/subacute post-stroke aphasia (PSA). Methods: Twenty-six patients underwent language assessment and MRI scanning. The integrity of the AF based on a three-segment model was evaluated using diffusion tensor imaging. All patients were classified into three groups according to the reconstruction of the left AF: completely reconstructed (group A, 8 cases), non-reconstructed (group B, 6 cases), and partially reconstructed (group C, 12 cases). The correlations and intergroup differences in language performance and diffusion indices were comprehensively estimated. Results: A correlation analyses showed that the lesion load of the language areas and diffusion indices on the left AF posterior and long segments was significantly related to some language subsets, respectively. When controlled lesion load was variable, significant correlations between diffusion indices on the posterior and long segments and comprehension, repetition, naming, and aphasia quotient were retained. Multiple comparison tests revealed intergroup differences in diffusion indices on the left AF posterior and long segments, as well as these language subsets. No significant correlation was found between the anterior segment and language performance. Conclusions: The integrity of the left AF segments, particularly the posterior segment, is crucial for the residual comprehension and repetition abilities in individuals with acute/subacute PSA, and lesion load in cortical language areas is an important factor that should be taken into account when illustrating the contributions of damage to special fiber tracts to language impairments.
Collapse
|
23
|
Martín-Fernández J, Gabarrós A, Fernandez-Coello A. Intraoperative Brain Mapping in Multilingual Patients: What Do We Know and Where Are We Going? Brain Sci 2022; 12:brainsci12050560. [PMID: 35624947 PMCID: PMC9139515 DOI: 10.3390/brainsci12050560] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2022] [Revised: 04/25/2022] [Accepted: 04/26/2022] [Indexed: 02/06/2023] Open
Abstract
In this review, we evaluate the knowledge gained so far about the neural bases of multilingual language processing obtained mainly through imaging and electrical stimulation mapping (ESM). We attempt to answer some key questions about multilingualism in the light of recent literature evidence, such as the degree of anatomical–functional integration of two or more languages in a multilingual brain, how the age of L2-acquisition affects language organization in the human brain, or how the brain controls more than one language. Finally, we highlight the future trends in multilingual language mapping.
Collapse
Affiliation(s)
- Jesús Martín-Fernández
- Hospital Universitario Nuestra Señora de Candelaria (HUNSC), Neurosurgery Section, 38010 Santa Cruz de Tenerife, Spain;
| | - Andreu Gabarrós
- Hospital Universitari de Bellvitge (HUB), Neurosurgery Section, Campus Bellvitge, University of Barcelona—IDIBELL, 08097 L’Hospitalet de Llobregat, Spain;
| | - Alejandro Fernandez-Coello
- Hospital Universitari de Bellvitge (HUB), Neurosurgery Section, Campus Bellvitge, University of Barcelona—IDIBELL, 08097 L’Hospitalet de Llobregat, Spain;
- CIBER de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), 08025 Barcelona, Spain
- Correspondence:
| |
Collapse
|