1
|
Wang JX, Li Y, Mu Y, Zhuang JY. Common and unique neural mechanisms of social and nonsocial conflict resolving and adaptation. Cereb Cortex 2022; 33:3773-3786. [PMID: 35989309 PMCID: PMC10068294 DOI: 10.1093/cercor/bhac306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 06/28/2022] [Accepted: 07/29/2022] [Indexed: 11/12/2022] Open
Abstract
Humans often need to deal with various forms of information conflicts that arise when they receive inconsistent information. However, it remains unclear how we resolve them and whether the brain may recruit similar or distinct brain mechanisms to process different domains (e.g. social vs. nonsocial) of conflicts. To address this, we used functional magnetic resonance imaging and scanned 50 healthy participants when they were asked to perform 2 Stroop tasks with different forms of conflicts: social (i.e. face-gender incongruency) and nonsocial (i.e. color-word incongruency) conflicts. Neuroimaging results revealed that the ventral lateral prefrontal cortex was generally activated in processing incongruent versus congruent stimuli regardless of the task type, serving as a common mechanism for conflict resolving across domains. Notably, trial-based and model-based results jointly demonstrated that the dorsal and rostral medial prefrontal cortices were uniquely engaged in processing social incongruent stimuli, suggesting distinct neural substrates of social conflict resolving and adaptation. The findings uncover that the common but unique brain mechanisms are recruited when humans resolve and adapt to social conflicts.
Collapse
Affiliation(s)
- Jia-Xi Wang
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China
| | - Yuhe Li
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China
| | - Yan Mu
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China.,Department of Psychology, University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Jin-Ying Zhuang
- School of Psychology and Cognitive Science, East China Normal University, Shanghai, 200062, China
| |
Collapse
|
2
|
Schmitt LM, Erb J, Tune S, Rysop AU, Hartwigsen G, Obleser J. Predicting speech from a cortical hierarchy of event-based time scales. SCIENCE ADVANCES 2021. [PMID: 34860554 DOI: 10.1101/2020.12.19.423616] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
How do predictions in the brain incorporate the temporal unfolding of context in our natural environment? We here provide evidence for a neural coding scheme that sparsely updates contextual representations at the boundary of events. This yields a hierarchical, multilayered organization of predictive language comprehension. Training artificial neural networks to predict the next word in a story at five stacked time scales and then using model-based functional magnetic resonance imaging, we observe an event-based “surprisal hierarchy” evolving along a temporoparietal pathway. Along this hierarchy, surprisal at any given time scale gated bottom-up and top-down connectivity to neighboring time scales. In contrast, surprisal derived from continuously updated context influenced temporoparietal activity only at short time scales. Representing context in the form of increasingly coarse events constitutes a network architecture for making predictions that is both computationally efficient and contextually diverse.
Collapse
Affiliation(s)
- Lea-Maria Schmitt
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Julia Erb
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Sarah Tune
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Anna U Rysop
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1 A, 04103 Leipzig, Germany
| | - Gesa Hartwigsen
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1 A, 04103 Leipzig, Germany
| | - Jonas Obleser
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| |
Collapse
|
3
|
Schmitt LM, Erb J, Tune S, Rysop AU, Hartwigsen G, Obleser J. Predicting speech from a cortical hierarchy of event-based time scales. SCIENCE ADVANCES 2021; 7:eabi6070. [PMID: 34860554 PMCID: PMC8641937 DOI: 10.1126/sciadv.abi6070] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Accepted: 10/15/2021] [Indexed: 05/30/2023]
Abstract
How do predictions in the brain incorporate the temporal unfolding of context in our natural environment? We here provide evidence for a neural coding scheme that sparsely updates contextual representations at the boundary of events. This yields a hierarchical, multilayered organization of predictive language comprehension. Training artificial neural networks to predict the next word in a story at five stacked time scales and then using model-based functional magnetic resonance imaging, we observe an event-based “surprisal hierarchy” evolving along a temporoparietal pathway. Along this hierarchy, surprisal at any given time scale gated bottom-up and top-down connectivity to neighboring time scales. In contrast, surprisal derived from continuously updated context influenced temporoparietal activity only at short time scales. Representing context in the form of increasingly coarse events constitutes a network architecture for making predictions that is both computationally efficient and contextually diverse.
Collapse
Affiliation(s)
- Lea-Maria Schmitt
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Julia Erb
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Sarah Tune
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| | - Anna U. Rysop
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1 A, 04103 Leipzig, Germany
| | - Gesa Hartwigsen
- Lise Meitner Research Group Cognition and Plasticity, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1 A, 04103 Leipzig, Germany
| | - Jonas Obleser
- Department of Psychology, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany
| |
Collapse
|
4
|
Listening to speech with a guinea pig-to-human brain-to-brain interface. Sci Rep 2021; 11:12231. [PMID: 34112826 PMCID: PMC8192924 DOI: 10.1038/s41598-021-90823-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Accepted: 05/12/2021] [Indexed: 11/30/2022] Open
Abstract
Nicolelis wrote in his 2003 review on brain-machine interfaces (BMIs) that the design of a successful BMI relies on general physiological principles describing how neuronal signals are encoded. Our study explored whether neural information exchanged between brains of different species is possible, similar to the information exchange between computers. We show for the first time that single words processed by the guinea pig auditory system are intelligible to humans who receive the processed information via a cochlear implant. We recorded the neural response patterns to single-spoken words with multi-channel electrodes from the guinea inferior colliculus. The recordings served as a blueprint for trains of biphasic, charge-balanced electrical pulses, which a cochlear implant delivered to the cochlear implant user’s ear. Study participants completed a four-word forced-choice test and identified the correct word in 34.8% of trials. The participants' recognition, defined by the ability to choose the same word twice, whether right or wrong, was 53.6%. For all sessions, the participants received no training and no feedback. The results show that lexical information can be transmitted from an animal to a human auditory system. In the discussion, we will contemplate how learning from the animals might help developing novel coding strategies.
Collapse
|
5
|
Sheng J, Zheng L, Lyu B, Cen Z, Qin L, Tan LH, Huang MX, Ding N, Gao JH. The Cortical Maps of Hierarchical Linguistic Structures during Speech Perception. Cereb Cortex 2020; 29:3232-3240. [PMID: 30137249 DOI: 10.1093/cercor/bhy191] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2018] [Revised: 06/27/2018] [Accepted: 07/20/2018] [Indexed: 11/14/2022] Open
Abstract
The hierarchical nature of language requires human brain to internally parse connected-speech and incrementally construct abstract linguistic structures. Recent research revealed multiple neural processing timescales underlying grammar-based configuration of linguistic hierarchies. However, little is known about where in the whole cerebral cortex such temporally scaled neural processes occur. This study used novel magnetoencephalography source imaging techniques combined with a unique language stimulation paradigm to segregate cortical maps synchronized to 3 levels of linguistic units (i.e., words, phrases, and sentences). Notably, distinct ensembles of cortical loci were identified to feature structures at different levels. The superior temporal gyrus was found to be involved in processing all 3 linguistic levels while distinct ensembles of other brain regions were recruited to encode each linguistic level. Neural activities in the right motor cortex only followed the rhythm of monosyllabic words which have clear acoustic boundaries, whereas the left anterior temporal lobe and the left inferior frontal gyrus were selectively recruited in processing phrases or sentences. Our results ground a multi-timescale hierarchical neural processing of speech in neuroanatomical reality with specific sets of cortices responsible for different levels of linguistic units.
Collapse
Affiliation(s)
- Jingwei Sheng
- Beijing City Key Lab for Medical Physics and Engineering, Institution of Heavy Ion Physics, School of Physics, Peking University, Beijing, China.,Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.,McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Li Zheng
- Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.,McGovern Institute for Brain Research, Peking University, Beijing, China.,Department of Biomedical Engineering, Peking University, Beijing, China
| | - Bingjiang Lyu
- Centre for Speech, Language and the Brain, Department of Psychology, University of Cambridge, Cambridge, UK
| | - Zhehang Cen
- Beijing City Key Lab for Medical Physics and Engineering, Institution of Heavy Ion Physics, School of Physics, Peking University, Beijing, China.,Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.,McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Lang Qin
- Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.,Department of Linguistics, The University of Hong Kong, Hong Kong, China
| | - Li Hai Tan
- Center for Brain Disorders and Cognitive Science, Shenzhen University, Shenzhen, Guangdong, China.,Center for Language and Brain, Shenzhen Institute of Neuroscience, Shenzhen, Guangdong, China
| | - Ming-Xiong Huang
- Department of Radiology, University of California, San Diego, CA, USA.,Radiology, Research, and Psychiatry Services, VA San Diego Healthcare System, San Diego, CA, USA
| | - Nai Ding
- College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, Zhejiang, China.,Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou, Zhejiang, China.,State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, Zhejiang, China
| | - Jia-Hong Gao
- Beijing City Key Lab for Medical Physics and Engineering, Institution of Heavy Ion Physics, School of Physics, Peking University, Beijing, China.,Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.,McGovern Institute for Brain Research, Peking University, Beijing, China.,Center for Language and Brain, Shenzhen Institute of Neuroscience, Shenzhen, Guangdong, China.,Shenzhen Key Laboratory of Affective and Social Cognitive Science, Institute of Affective and Social Neuroscience, Shenzhen University, Shenzhen, Guangdong, China
| |
Collapse
|
6
|
Liu Z, Shu S, Lu L, Ge J, Gao JH. Spatiotemporal dynamics of predictive brain mechanisms during speech processing: an MEG study. BRAIN AND LANGUAGE 2020; 203:104755. [PMID: 32007671 DOI: 10.1016/j.bandl.2020.104755] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2019] [Revised: 12/09/2019] [Accepted: 01/15/2020] [Indexed: 06/10/2023]
Abstract
Rapid and efficient speech processing benefits from the prediction derived from prior expectations based on the identification of individual words. It is known that speech processing is carried out within a distributed frontotemporal network. However, the spatiotemporal causal dynamics of predictive brain mechanisms in sound-to-meaning mapping within this network remain unclear. Using magnetoencephalography, we adopted a semantic anomaly paradigm which consists of expected, unexpected and time-reversed Mandarin Chinese speech, and localized the effects of violated expectation in frontotemporal brain regions, the sensorimotor cortex and the supramarginal gyrus from 250 ms relative to the target words. By further investigating the causal cortical dynamics, we provided the description of the causal dynamic network within the framework of the dual stream model, and highlighted the importance of the connections within the ventral pathway, the top-down modulation from the left inferior frontal gyrus and the cross-stream integration during the speech processing of violated expectation.
Collapse
Affiliation(s)
- Zhaowei Liu
- Beijing City Key Lab for Medical Physics and Engineering, Institution of Heavy Ion Physics, School of Physics, Peking University, Beijing, China; Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Su Shu
- Beijing City Key Lab for Medical Physics and Engineering, Institution of Heavy Ion Physics, School of Physics, Peking University, Beijing, China; Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Lingxi Lu
- Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Jianqiao Ge
- Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; McGovern Institute for Brain Research, Peking University, Beijing, China.
| | - Jia-Hong Gao
- Beijing City Key Lab for Medical Physics and Engineering, Institution of Heavy Ion Physics, School of Physics, Peking University, Beijing, China; Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; McGovern Institute for Brain Research, Peking University, Beijing, China.
| |
Collapse
|
7
|
Abstract
There are functional and anatomical distinctions between the neural systems involved in the recognition of sounds in the environment and those involved in the sensorimotor guidance of sound production and the spatial processing of sound. Evidence for the separation of these processes has historically come from disparate literatures on the perception and production of speech, music and other sounds. More recent evidence indicates that there are computational distinctions between the rostral and caudal primate auditory cortex that may underlie functional differences in auditory processing. These functional differences may originate from differences in the response times and temporal profiles of neurons in the rostral and caudal auditory cortex, suggesting that computational accounts of primate auditory pathways should focus on the implications of these temporal response differences.
Collapse
|
8
|
Fei N, Ge J, Wang Y, Gao JH. Aging-related differences in the cortical network subserving intelligible speech. BRAIN AND LANGUAGE 2020; 201:104713. [PMID: 31759299 DOI: 10.1016/j.bandl.2019.104713] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Revised: 10/29/2019] [Accepted: 10/30/2019] [Indexed: 06/10/2023]
Abstract
Language communication is crucial throughout the lifespan. The current study investigated how aging affects the brain network subserving intelligible speech. Using functional magnetic resonance imaging, we compared brain responses to intelligible and unintelligible speech between older and young adults. Univariate and multivariate analyses revealed reduced brain activation and lower regional pattern distinctions in response to intelligible versus unintelligible speech in the left anterior superior temporal gyrus (aSTG) and the left inferior frontal gyrus (IFG) in the older compared with young adults. Notably, the functional connectivity between the left IFG and the left angular gyrus (AG) was increased and a significantly enhanced bidirectional effective connectivity between the left aSTG and the left AG was observed in the older adults for processing speech intelligibility. Our study revealed aging-related differences in the cortical activity for intelligible speech and suggested that increased frontal-temporal-parietal functional integration may help facilitate spoken language processing in older adults.
Collapse
Affiliation(s)
- Nanxi Fei
- Beijing City Key Lab for Medical Physics and Engineering, Institute of Heavy Ion Physics, School of Physics, Peking University, Beijing, China; Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China
| | - Jianqiao Ge
- Beijing City Key Lab for Medical Physics and Engineering, Institute of Heavy Ion Physics, School of Physics, Peking University, Beijing, China; Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China.
| | - Yi Wang
- Public Health Science and Engineering College, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Jia-Hong Gao
- Beijing City Key Lab for Medical Physics and Engineering, Institute of Heavy Ion Physics, School of Physics, Peking University, Beijing, China; Center for MRI Research, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; McGovern Institute for Brain Research, Peking University, Beijing, China.
| |
Collapse
|
9
|
Wu Y, Guo X, Gao Y, Wang Z, Wang X. Meaning enhances discrimination of merged phonemes: A mismatch negativity study. Brain Res 2019; 1724:146433. [DOI: 10.1016/j.brainres.2019.146433] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Revised: 08/26/2019] [Accepted: 09/02/2019] [Indexed: 11/30/2022]
|
10
|
Siman-Tov T, Granot RY, Shany O, Singer N, Hendler T, Gordon CR. Is there a prediction network? Meta-analytic evidence for a cortical-subcortical network likely subserving prediction. Neurosci Biobehav Rev 2019; 105:262-275. [PMID: 31437478 DOI: 10.1016/j.neubiorev.2019.08.012] [Citation(s) in RCA: 41] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2019] [Revised: 07/25/2019] [Accepted: 08/17/2019] [Indexed: 01/24/2023]
Abstract
Predictive coding is an increasingly influential and ambitious concept in neuroscience viewing the brain as a 'hypothesis testing machine' that constantly strives to minimize prediction error, the gap between its predictions and the actual sensory input. Despite the invaluable contribution of this framework to the formulation of brain function, its neuroanatomical foundations have not been fully defined. To address this gap, we conducted activation likelihood estimation (ALE) meta-analysis of 39 neuroimaging studies of three functional domains (action perception, language and music) inherently involving prediction. The ALE analysis revealed a widely distributed brain network encompassing regions within the inferior and middle frontal gyri, anterior insula, premotor cortex, pre-supplementary motor area, temporoparietal junction, striatum, thalamus/subthalamus and the cerebellum. This network is proposed to subserve domain-general prediction and its relevance to motor control, attention, implicit learning and social cognition is discussed in light of the predictive coding scheme. Better understanding of the presented network may help advance treatments of neuropsychiatric conditions related to aberrant prediction processing and promote cognitive enhancement in healthy individuals.
Collapse
Affiliation(s)
- Tali Siman-Tov
- Sagol Brain Institute Tel Aviv, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Roni Y Granot
- Musicology Department, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Ofir Shany
- Sagol Brain Institute Tel Aviv, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Neomi Singer
- Sagol Brain Institute Tel Aviv, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel; Montreal Neurological Institute, Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec, Canada
| | - Talma Hendler
- Sagol Brain Institute Tel Aviv, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel; School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Carlos R Gordon
- Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel; Department of Neurology, Meir Medical Center, Kfar Saba, Israel
| |
Collapse
|
11
|
Barnaud ML, Bessière P, Diard J, Schwartz JL. Reanalyzing neurocognitive data on the role of the motor system in speech perception within COSMO, a Bayesian perceptuo-motor model of speech communication. BRAIN AND LANGUAGE 2018; 187:19-32. [PMID: 29241588 PMCID: PMC6286382 DOI: 10.1016/j.bandl.2017.12.003] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2017] [Revised: 07/17/2017] [Accepted: 12/02/2017] [Indexed: 06/07/2023]
Abstract
While neurocognitive data provide clear evidence for the involvement of the motor system in speech perception, its precise role and the way motor information is involved in perceptual decision remain unclear. In this paper, we discuss some recent experimental results in light of COSMO, a Bayesian perceptuo-motor model of speech communication. COSMO enables us to model both speech perception and speech production with probability distributions relating phonological units with sensory and motor variables. Speech perception is conceived as a sensory-motor architecture combining an auditory and a motor decoder thanks to a Bayesian fusion process. We propose the sketch of a neuroanatomical architecture for COSMO, and we capitalize on properties of the auditory vs. motor decoders to address three neurocognitive studies of the literature. Altogether, this computational study reinforces functional arguments supporting the role of a motor decoding branch in the speech perception process.
Collapse
Affiliation(s)
- Marie-Lou Barnaud
- Univ. Grenoble Alpes, Gipsa-lab, F-38000 Grenoble, France; CNRS, Gipsa-lab, F-38000 Grenoble, France; Univ. Grenoble Alpes, LPNC, F-38000 Grenoble, France; CNRS, LPNC, F-38000 Grenoble, France.
| | | | - Julien Diard
- Univ. Grenoble Alpes, LPNC, F-38000 Grenoble, France; CNRS, LPNC, F-38000 Grenoble, France
| | - Jean-Luc Schwartz
- Univ. Grenoble Alpes, Gipsa-lab, F-38000 Grenoble, France; CNRS, Gipsa-lab, F-38000 Grenoble, France.
| |
Collapse
|
12
|
Focal versus distributed temporal cortex activity for speech sound category assignment. Proc Natl Acad Sci U S A 2018; 115:E1299-E1308. [PMID: 29363598 PMCID: PMC5819402 DOI: 10.1073/pnas.1714279115] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023] Open
Abstract
When listening to speech, phonemes are represented in a distributed fashion in our temporal and prefrontal cortices. How these representations are selected in a phonemic decision context, and in particular whether distributed or focal neural information is required for explicit phoneme recognition, is unclear. We hypothesized that focal and early neural encoding of acoustic signals is sufficiently informative to access speech sound representations and permit phoneme recognition. We tested this hypothesis by combining a simple speech-phoneme categorization task with univariate and multivariate analyses of fMRI, magnetoencephalography, intracortical, and clinical data. We show that neural information available focally in the temporal cortex prior to decision-related neural activity is specific enough to account for human phonemic identification. Percepts and words can be decoded from distributed neural activity measures. However, the existence of widespread representations might conflict with the more classical notions of hierarchical processing and efficient coding, which are especially relevant in speech processing. Using fMRI and magnetoencephalography during syllable identification, we show that sensory and decisional activity colocalize to a restricted part of the posterior superior temporal gyrus (pSTG). Next, using intracortical recordings, we demonstrate that early and focal neural activity in this region distinguishes correct from incorrect decisions and can be machine-decoded to classify syllables. Crucially, significant machine decoding was possible from neuronal activity sampled across different regions of the temporal and frontal lobes, despite weak or absent sensory or decision-related responses. These findings show that speech-sound categorization relies on an efficient readout of focal pSTG neural activity, while more distributed activity patterns, although classifiable by machine learning, instead reflect collateral processes of sensory perception and decision.
Collapse
|