1
|
Hudson Kam CL, Sadlier-Brown E, Clark S, Jang C, Demmans Epp C, Thomson J. Evaluating English-language morphological awareness assessments. FIRST LANGUAGE 2024; 44:327-344. [PMID: 39100688 PMCID: PMC11293997 DOI: 10.1177/01427237241245500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 08/06/2024]
Abstract
Many studies have shown that morphological knowledge has effects on reading comprehension separate from other aspects of language knowledge. This has implications for reading instruction and assessment: it suggests that children could have reading comprehension difficulties that are due to a lack of morphological knowledge, and thus, that explicit instruction of morphology might be helpful for them, indeed for all children. To find children who might especially benefit from specific instruction in morphology, we would need good tests of morphological knowledge. We evaluated a set of morphological awareness assessments to determine whether they conclusively tapped into morphological knowledge, and found that it was not possible to be certain that they were accurately targeting morphological knowledge.
Collapse
Affiliation(s)
- Carla L. Hudson Kam
- Carla L. Hudson Kam, Department of Linguistics, The University of British Columbia, 2613 West Mall, Vancouver, BC V6T 1Z4, Canada.
| | | | | | | | | | | |
Collapse
|
2
|
Michaelov JA, Bardolph MD, Van Petten CK, Bergen BK, Coulson S. Strong Prediction: Language Model Surprisal Explains Multiple N400 Effects. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2024; 5:107-135. [PMID: 38645623 PMCID: PMC11025652 DOI: 10.1162/nol_a_00105] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Accepted: 03/24/2023] [Indexed: 04/23/2024]
Abstract
Theoretical accounts of the N400 are divided as to whether the amplitude of the N400 response to a stimulus reflects the extent to which the stimulus was predicted, the extent to which the stimulus is semantically similar to its preceding context, or both. We use state-of-the-art machine learning tools to investigate which of these three accounts is best supported by the evidence. GPT-3, a neural language model trained to compute the conditional probability of any word based on the words that precede it, was used to operationalize contextual predictability. In particular, we used an information-theoretic construct known as surprisal (the negative logarithm of the conditional probability). Contextual semantic similarity was operationalized by using two high-quality co-occurrence-derived vector-based meaning representations for words: GloVe and fastText. The cosine between the vector representation of the sentence frame and final word was used to derive contextual cosine similarity estimates. A series of regression models were constructed, where these variables, along with cloze probability and plausibility ratings, were used to predict single trial N400 amplitudes recorded from healthy adults as they read sentences whose final word varied in its predictability, plausibility, and semantic relationship to the likeliest sentence completion. Statistical model comparison indicated GPT-3 surprisal provided the best account of N400 amplitude and suggested that apparently disparate N400 effects of expectancy, plausibility, and contextual semantic similarity can be reduced to variation in the predictability of words. The results are argued to support predictive coding in the human language network.
Collapse
Affiliation(s)
- James A. Michaelov
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Megan D. Bardolph
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Cyma K. Van Petten
- Department of Psychology, Binghamton University, State University of New York, Binghamton, NY, USA
| | - Benjamin K. Bergen
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Seana Coulson
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| |
Collapse
|
3
|
Tomasello R, Grisoni L, Boux I, Sammler D, Pulvermüller F. OUP accepted manuscript. Cereb Cortex 2022; 32:4885-4901. [PMID: 35136980 PMCID: PMC9626830 DOI: 10.1093/cercor/bhab522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Revised: 11/16/2021] [Accepted: 12/17/2021] [Indexed: 11/20/2022] Open
Abstract
During conversations, speech prosody provides important clues about the speaker’s communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. Here, the neurophysiological basis of intonation and speech act understanding were investigated with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level. Already approximately 100 ms after the sentence-final word differing in prosody, questions, and statements expressed with the same sentences led to different neurophysiological activity recorded in the event-related potential. Interestingly, low-pass filtered sentences and acoustically matched nonvocal musical signals failed to show any neurophysiological dissociations, thus suggesting that the physical intonation alone cannot explain this modulation. Our results show rapid neurophysiological indexes of prosodic communicative information processing that emerge only when pragmatic and lexico-semantic information are fully expressed. The early enhancement of question-related activity compared with statements was due to sources in the articulatory-motor region, which may reflect the richer action knowledge immanent to questions, namely the expectation of the partner action of answering the question. The present findings demonstrate a neurophysiological correlate of prosodic communicative information processing, which enables humans to rapidly detect and understand speaker intentions in linguistic interactions.
Collapse
Affiliation(s)
- Rosario Tomasello
- Address correspondence to Rosario Tomasello, Brain Language Laboratory, Department of Philosophy and Humanities, WE4, Freie Universität Berlin, Habelschwerdter Allee 45, 14195 Berlin, Germany.
| | - Luigi Grisoni
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität Berlin, 14195 Berlin, Germany
- Cluster of Excellence ‘Matters of Activity. Image Space Material’, Humboldt Universität zu Berlin, 10099 Berlin, Germany
| | - Isabella Boux
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität Berlin, 14195 Berlin, Germany
- Berlin School of Mind and Brain, Humboldt Universität zu Berlin, 10117 Berlin, Germany
- Einstein Center for Neurosciences, 10117 Berlin, Germany
| | - Daniela Sammler
- Research Group ‘Neurocognition of Music and Language’, Max Planck Institute for Empirical Aesthetics, 60322 Frankfurt am Main, Germany
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
| | - Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität Berlin, 14195 Berlin, Germany
- Cluster of Excellence ‘Matters of Activity. Image Space Material’, Humboldt Universität zu Berlin, 10099 Berlin, Germany
- Berlin School of Mind and Brain, Humboldt Universität zu Berlin, 10117 Berlin, Germany
- Einstein Center for Neurosciences, 10117 Berlin, Germany
| |
Collapse
|
4
|
Boux I, Tomasello R, Grisoni L, Pulvermüller F. Brain signatures predict communicative function of speech production in interaction. Cortex 2020; 135:127-145. [PMID: 33360757 DOI: 10.1016/j.cortex.2020.11.008] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2020] [Revised: 11/05/2020] [Accepted: 11/18/2020] [Indexed: 10/22/2022]
Abstract
People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.
Collapse
Affiliation(s)
- Isabella Boux
- Brain Language Laboratory, Department of Philosophy and Humanities, WE4 Freie Universität Berlin, Berlin, Germany; Einstein Center for Neurosciences, Berlin, Germany; Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany.
| | - Rosario Tomasello
- Brain Language Laboratory, Department of Philosophy and Humanities, WE4 Freie Universität Berlin, Berlin, Germany; Cluster of Excellence 'Matters of Activity. Image Space Material', Humboldt Universität zu Berlin, Berlin, Germany.
| | - Luigi Grisoni
- Brain Language Laboratory, Department of Philosophy and Humanities, WE4 Freie Universität Berlin, Berlin, Germany
| | - Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and Humanities, WE4 Freie Universität Berlin, Berlin, Germany; Einstein Center for Neurosciences, Berlin, Germany; Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany; Cluster of Excellence 'Matters of Activity. Image Space Material', Humboldt Universität zu Berlin, Berlin, Germany
| |
Collapse
|
5
|
Lecaignard F, Bertrand O, Caclin A, Mattout J. Empirical Bayes evaluation of fused EEG-MEG source reconstruction: Application to auditory mismatch evoked responses. Neuroimage 2020; 226:117468. [PMID: 33075561 DOI: 10.1016/j.neuroimage.2020.117468] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2020] [Revised: 09/08/2020] [Accepted: 10/09/2020] [Indexed: 12/12/2022] Open
Abstract
We here turn the general and theoretical question of the complementarity of EEG and MEG for source reconstruction, into a practical empirical one. Precisely, we address the challenge of evaluating multimodal data fusion on real data. For this purpose, we build on the flexibility of Parametric Empirical Bayes, namely for EEG-MEG data fusion, group level inference and formal hypothesis testing. The proposed approach follows a two-step procedure by first using unimodal or multimodal inference to derive a cortical solution at the group level; and second by using this solution as a prior model for single subject level inference based on either unimodal or multimodal data. Interestingly, for inference based on the same data (EEG, MEG or both), one can then formally compare, as alternative hypotheses, the relative plausibility of the two unimodal and the multimodal group priors. Using auditory data, we show that this approach enables to draw important conclusions, namely on (i) the superiority of multimodal inference, (ii) the greater spatial sensitivity of MEG compared to EEG, (iii) the ability of EEG data alone to source reconstruct temporal lobe activity, (iv) the usefulness of EEG to improve MEG based source reconstruction. Importantly, we largely reproduce those findings over two different experimental conditions. We here focused on Mismatch Negativity (MMN) responses for which generators have been extensively investigated with little homogeneity in the reported results. Our multimodal inference at the group level revealed spatio-temporal activity within the supratemporal plane with a precision which, to our knowledge, has never been achieved before with non-invasive recordings.
Collapse
Affiliation(s)
- Françoise Lecaignard
- Lyon Neuroscience Research Center, CRNL; INSERM, U1028; CNRS, UMR5292; Brain Dynamics and Cognition Team, Lyon, F-69000, France; University Lyon 1, Lyon, F-69000, France.
| | - Olivier Bertrand
- Lyon Neuroscience Research Center, CRNL; INSERM, U1028; CNRS, UMR5292; Brain Dynamics and Cognition Team, Lyon, F-69000, France; University Lyon 1, Lyon, F-69000, France
| | - Anne Caclin
- Lyon Neuroscience Research Center, CRNL; INSERM, U1028; CNRS, UMR5292; Brain Dynamics and Cognition Team, Lyon, F-69000, France; University Lyon 1, Lyon, F-69000, France
| | - Jérémie Mattout
- Lyon Neuroscience Research Center, CRNL; INSERM, U1028; CNRS, UMR5292; Brain Dynamics and Cognition Team, Lyon, F-69000, France; University Lyon 1, Lyon, F-69000, France
| |
Collapse
|
6
|
Zhang LL, Zhong YQ, Sun JW, Chen L, Sun JQ, Hou XY, Chen JW, Guo XT. Deficit of long-term memory traces for words in children with cochlear implants. Clin Neurophysiol 2020; 131:1323-1331. [DOI: 10.1016/j.clinph.2020.03.012] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2019] [Revised: 03/04/2020] [Accepted: 03/09/2020] [Indexed: 01/06/2023]
|
7
|
Leminen M, Leminen A, Smolander S, Arkkila E, Shtyrov Y, Laasonen M, Kujala T. Quick reorganization of memory traces for morphologically complex words in young children. Neuropsychologia 2019; 138:107309. [PMID: 31857117 DOI: 10.1016/j.neuropsychologia.2019.107309] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2018] [Revised: 10/10/2019] [Accepted: 12/15/2019] [Indexed: 10/25/2022]
Abstract
Formation of neural mechanisms for morphosyntactic processing in young children is still poorly understood. Here, we addressed neural processing and rapid online acquisition of familiar and unfamiliar combinations of morphemes. Three different types of morphologically complex words - derived, inflected, and novel (pseudostem + real suffix) - were presented in a passive listening setting to 16 typically developing 3-4-year old children (as part of a longitudinal Helsinki SLI follow-up study). The mismatch negativity (MMN) component of event-related potentials (ERP), an established index of long-term linguistic memory traces in the brain, was analysed separately for the initial and final periods of the exposure to these items. We found MMN response enhancement for the inflected words towards the end of the recording session, whereas no response change was observed for the derived or novel complex forms. This enhancement indicates rapid build-up of a new memory trace for the combination of real morphemes, suggesting a capacity for online formation of whole-form lexicalized representations as one of the morphological mechanisms in the developing brain. Furthermore, this enhancement increased with age, suggesting the development of automatic morphological processing circuits in the age range of 3-4 years.
Collapse
Affiliation(s)
- Miika Leminen
- Department of Otorhinolaryngology and Phoniatrics, Helsinki University Hospital and University of Helsinki, PO Box 250, FIN-00029, HUS, Finland; Cognitive Brain Research Unit, Department of Psychology and Logopedics, Faculty of Medicine, PO Box 21, FIN-00014, University of Helsinki, Finland.
| | - Alina Leminen
- Cognitive Brain Research Unit, Department of Psychology and Logopedics, Faculty of Medicine, PO Box 21, FIN-00014, University of Helsinki, Finland; Cognitive Science, Department of Digital Humanitiers, Faculty of Arts, PO Box 9, FIN-00014, University of Helsinki, Finland.
| | - Sini Smolander
- Department of Otorhinolaryngology and Phoniatrics, Helsinki University Hospital and University of Helsinki, PO Box 250, FIN-00029, HUS, Finland; Research Unit of Logopedics, PO Box 8000, FIN-90014, University of Oulu, Finland.
| | - Eva Arkkila
- Department of Otorhinolaryngology and Phoniatrics, Helsinki University Hospital and University of Helsinki, PO Box 250, FIN-00029, HUS, Finland.
| | - Yury Shtyrov
- Center of Functionally Integrative Neuroscience (CFIN), Institute for Clinical Medicine, Aarhus University, DK-8000, Aarhus C, Denmark; Laboratory of Behavioural Neurodynamics, St.Petersburg State University, Makarova emb, 6, St.Petersburg, 199034, Russian Federation.
| | - Marja Laasonen
- Department of Otorhinolaryngology and Phoniatrics, Helsinki University Hospital and University of Helsinki, PO Box 250, FIN-00029, HUS, Finland; Department of Psychology and Speech-Language Pathology, Faculty of Social Sciences, FIN-20014, University of Turku, Finland; Department of Psychology and Logopedics, Faculty of Clinical Medicine, PO Box 63, FIN-00014 University of Helsinki, Finland.
| | - Teija Kujala
- Cognitive Brain Research Unit, Department of Psychology and Logopedics, Faculty of Medicine, PO Box 21, FIN-00014, University of Helsinki, Finland.
| |
Collapse
|
8
|
Neurophysiological evidence for rapid processing of verbal and gestural information in understanding communicative actions. Sci Rep 2019; 9:16285. [PMID: 31705052 PMCID: PMC6841672 DOI: 10.1038/s41598-019-52158-w] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2019] [Accepted: 10/12/2019] [Indexed: 11/08/2022] Open
Abstract
During everyday social interaction, gestures are a fundamental part of human communication. The communicative pragmatic role of hand gestures and their interaction with spoken language has been documented at the earliest stage of language development, in which two types of indexical gestures are most prominent: the pointing gesture for directing attention to objects and the give-me gesture for making requests. Here we study, in adult human participants, the neurophysiological signatures of gestural-linguistic acts of communicating the pragmatic intentions of naming and requesting by simultaneously presenting written words and gestures. Already at ~150 ms, brain responses diverged between naming and request actions expressed by word-gesture combination, whereas the same gestures presented in isolation elicited their earliest neurophysiological dissociations significantly later (at ~210 ms). There was an early enhancement of request-evoked brain activity as compared with naming, which was due to sources in the frontocentral cortex, consistent with access to action knowledge in request understanding. In addition, an enhanced N400-like response indicated late semantic integration of gesture-language interaction. The present study demonstrates that word-gesture combinations used to express communicative pragmatic intentions speed up the brain correlates of comprehension processes – compared with gesture-only understanding – thereby calling into question current serial linguistic models viewing pragmatic function decoding at the end of a language comprehension cascade. Instead, information about the social-interactive role of communicative acts is processed instantaneously.
Collapse
|
9
|
Grisoni L, Mohr B, Pulvermüller F. Prediction mechanisms in motor and auditory areas and their role in sound perception and language understanding. Neuroimage 2019; 199:206-216. [DOI: 10.1016/j.neuroimage.2019.05.071] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2018] [Revised: 05/02/2019] [Accepted: 05/27/2019] [Indexed: 10/26/2022] Open
|
10
|
Acquisition of L2 morphology by adult language learners. Cortex 2019; 116:74-90. [DOI: 10.1016/j.cortex.2019.01.012] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2017] [Revised: 10/11/2018] [Accepted: 01/11/2019] [Indexed: 11/17/2022]
|
11
|
Schuster S, Lahiri A. Gradations of interpretability in spoken complex word recognition. Neuropsychologia 2019; 133:107065. [PMID: 31004691 DOI: 10.1016/j.neuropsychologia.2019.04.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2018] [Revised: 04/11/2019] [Accepted: 04/13/2019] [Indexed: 11/29/2022]
Abstract
In an ERP experiment with cross-modal priming, we found evidence for graded interpretability of complex pseudowords, expressed as differences in the degree of priming between a complex pseudoword and its embedded base word. Only pseudowords that were constructed according to the morphological constraints of the language (for instance that nouns ending in {-ung} in German must be derived from a verbal base) led to significant attenuation of the N400 component relative to an unrelated control. Novel formations that consisted of the concatenation of a stem and an incompatible affix (e.g. wirr A.,'confused' and {-lein}, which can only add to nouns) did not modulate the N400 component. Moreover, gradations in degree of priming were observed with novel pseudowords that were the result of several morphological operations as a function of the composition of the full sequence of morphological operations. Any positions within the sequence that were filled by possible, but non-existing formations (thus compare hübsch (A., 'pretty') > *hübschen (V., 'beautify') > *Hübschung (N., 'beautification') versus spitz (A., 'sharp') > spitzen (V., 'sharpen') > *Spitzung (N., 'sharpening'), whereby in the former the intermediate position is expressed through a possible, but non-existing verb form) led to reduced activation of the base word. These findings corroborate previous behavioural and neuroimaging results in visual word recognition that found support for step-wise morphological decomposition in multi-affixed pseudowords and reveal that the composition of the derivational path can lead to gradations of interpretability in the recognition of spoken complex pseudowords.
Collapse
|
12
|
Leminen A, Smolka E, Duñabeitia JA, Pliatsikas C. Morphological processing in the brain: The good (inflection), the bad (derivation) and the ugly (compounding). Cortex 2018; 116:4-44. [PMID: 30268324 DOI: 10.1016/j.cortex.2018.08.016] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2018] [Revised: 07/01/2018] [Accepted: 08/22/2018] [Indexed: 11/26/2022]
Abstract
There is considerable behavioral evidence that morphologically complex words such as 'tax-able' and 'kiss-es' are processed and represented combinatorially. In other words, they are decomposed into their constituents 'tax' and '-able' during comprehension (reading or listening), and producing them might also involve on-the-spot combination of these constituents (especially for inflections). However, despite increasing amount of neurocognitive research, the neural mechanisms underlying these processes are still not fully understood. The purpose of this critical review is to offer a comprehensive overview on the state-of-the-art of the research on the neural mechanisms of morphological processing. In order to take into account all types of complex words, we include findings on inflected, derived, and compound words presented both visually and aurally. More specifically, we cover a wide range of electro- and magnetoencephalography (EEG and MEG, respectively) as well as structural/functional magnetic resonance imaging (s/fMRI) studies that focus on morphological processing. We present the findings with respect to the temporal course and localization of morphologically complex word processing. We summarize the observed findings, their interpretations with respect to current psycholinguistic models, and discuss methodological approaches as well as their possible limitations.
Collapse
Affiliation(s)
- Alina Leminen
- Cognitive Science, Department of Digital Humanities, Faculty of Arts, University of Helsinki, Finland; Cognitive Brain Research Unit, Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland.
| | - Eva Smolka
- Department of Linguistics, University of Konstanz, Germany
| | - Jon A Duñabeitia
- Facultad de Lenguas y Educación, Universidad Nebrija, Madrid, Spain; Basque Center on Cognition, Brain and Language (BCBL), Donostia, Spain
| | - Christos Pliatsikas
- School of Psychology and Clinical Language Sciences, University of Reading, Reading, United Kingdom
| |
Collapse
|
13
|
Gansonre C, Højlund A, Leminen A, Bailey C, Shtyrov Y. Task-free auditory EEG paradigm for probing multiple levels of speech processing in the brain. Psychophysiology 2018; 55:e13216. [PMID: 30101984 DOI: 10.1111/psyp.13216] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2017] [Revised: 05/09/2018] [Accepted: 05/09/2018] [Indexed: 11/26/2022]
Abstract
While previous studies on language processing highlighted several ERP components in relation to specific stages of sound and speech processing, no study has yet combined them to obtain a comprehensive picture of language abilities in a single session. Here, we propose a novel task-free paradigm aimed at assessing multiple levels of speech processing by combining various speech and nonspeech sounds in an adaptation of a multifeature passive oddball design. We recorded EEG in healthy adult participants, who were presented with these sounds in the absence of sound-directed attention while being engaged in a primary visual task. This produced a range of responses indexing various levels of sound processing and language comprehension: (a) P1-N1 complex, indexing obligatory auditory processing; (b) P3-like dynamics associated with involuntary attention allocation for unusual sounds; (c) enhanced responses for native speech (as opposed to nonnative phonemes) from ∼50 ms from phoneme onset, indicating phonological processing; (d) amplitude advantage for familiar real words as opposed to meaningless pseudowords, indexing automatic lexical access; (e) topographic distribution differences in the cortical activation of action verbs versus concrete nouns, likely linked with the processing of lexical semantics. These multiple indices of speech-sound processing were acquired in a single attention-free setup that does not require any task or subject cooperation; subject to future research, the present protocol may potentially be developed into a useful tool for assessing the status of auditory and linguistic functions in uncooperative or unresponsive participants, including a range of clinical or developmental populations.
Collapse
Affiliation(s)
- Christelle Gansonre
- Center of Functionally Integrative Neuroscience (CFIN), Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Andreas Højlund
- Center of Functionally Integrative Neuroscience (CFIN), Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Alina Leminen
- Center of Functionally Integrative Neuroscience (CFIN), Department of Clinical Medicine, Aarhus University, Aarhus, Denmark.,Cognitive Brain Research Unit, Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Christopher Bailey
- Center of Functionally Integrative Neuroscience (CFIN), Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Yury Shtyrov
- Center of Functionally Integrative Neuroscience (CFIN), Department of Clinical Medicine, Aarhus University, Aarhus, Denmark.,Laboratory of Behavioural Neurodynamics, St. Petersburg State University, St. Petersburg, Russia
| |
Collapse
|
14
|
Hanna J, Pulvermüller F. Congruency of Separable Affix Verb Combinations Is Linearly Indexed by the N400. Front Hum Neurosci 2018; 12:219. [PMID: 29892220 PMCID: PMC5985318 DOI: 10.3389/fnhum.2018.00219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2017] [Accepted: 05/09/2018] [Indexed: 11/17/2022] Open
Abstract
Separable affix verbs consist of a stem and a derivational affix, which, in some languages can appear together or in discontinuous, distributed form, e.g., German “aufgreifen” and “greifen … auf” [“up-pick(ing)” and “pick … up”]. Certain stems can combine with only certain affixes. However, many such combinations are evaluated not as clearly correct or incorrect, but frequently take an intermediate status with participants rating them ambiguously. Here, we mapped brain responses to combinations of verb stems and affixes realized in short sentences, including more and less common particle verbs, borderline acceptable combinations and clear violations. Event-related potential responses to discontinuous particle verbs were obtained for five affixes re-combined with 10 verb stems, situated within short, German sentences, i.e., “sie <stem>en es <affix>,” English: “they <stem> it <affix>.” The congruity of combinations was assessed both with behavioral ratings of the stimuli and corpus-derived probability measures. The size of a frontal N400 correlated with the degree of incongruency between stem and affix, as assessed by both measures. Behavioral ratings performed better than corpus-derived measures in predicting N400 magnitudes, and a combined model performed best of all. No evidence for a discrete, right/wrong effect was found. We discuss methodological implications and integrate the results into past research on the N400 and neurophysiological studies on separable-affix verbs, generally.
Collapse
Affiliation(s)
- Jeff Hanna
- Department of Neurosurgery, University Hospital Erlangen, Erlangen, Germany.,Department of Philosophy and Humanities, Institute for German and Dutch Philology, Freie Universität Berlin, Berlin, Germany
| | - Friedemann Pulvermüller
- Department of Philosophy and Humanities, Institute for German and Dutch Philology, Freie Universität Berlin, Berlin, Germany.,Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
15
|
Hanna J, Cappelle B, Pulvermüller F. Spread the word: MMN brain response reveals whole-form access of discontinuous particle verbs. BRAIN AND LANGUAGE 2017; 175:86-98. [PMID: 29059543 DOI: 10.1016/j.bandl.2017.10.002] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2016] [Revised: 04/21/2017] [Accepted: 10/09/2017] [Indexed: 06/07/2023]
Abstract
The status of particle verbs such as rise (…) up as either lexically stored or combinatorially assembled is an issue which so far has not been settled decisively. In this study, we use the mismatch negativity (MMN) brain response to observe neurophysiological responses to discontinuous particle verbs. The MMN can be used to distinguish between whole-form storage and combinatorial processes, as it is enhanced to stored words compared to unknown pseudowords, whereas combinatorially legal strings elicit a reduced MMN relative to ungrammatical ones. Earlier work had found larger MMNs to congruent than to incongruent verb-particle combinations when particle and verb appeared as adjacent elements, thus suggesting whole-form storage at least in this case. However, it is still possible that particle verbs discontinuously spread out across a sentence would elicit the combinatorial, grammar-violation response pattern instead. Here, we tested the brain signatures of discontinuous verb-particle combinations, orthogonally varying congruence and semantic transparency. The results show for the first time brain indices of whole-form storage for discontinuous constituents, thus arguing in favour of access to whole-form-stored lexical elements in the processing of particle verbs, irrespective of their semantic opacity. Results are discussed in the context of linguistic debates about the status of particle verbs as words, lexical elements or syntactically generated combinations. The explanation of the pattern of results within a neurobiological language model is highlighted.
Collapse
Affiliation(s)
- Jeff Hanna
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität, Berlin, Germany.
| | - Bert Cappelle
- Univ. Lille, CNRS, UMR 8163 - STL - Savoirs Textes Langage, F-59000 Lille, France
| | - Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität, Berlin, Germany; Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany
| |
Collapse
|
16
|
Yu YH, Shafer VL, Sussman ES. Neurophysiological and Behavioral Responses of Mandarin Lexical Tone Processing. Front Neurosci 2017; 11:95. [PMID: 28321179 PMCID: PMC5338334 DOI: 10.3389/fnins.2017.00095] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2016] [Accepted: 02/13/2017] [Indexed: 11/22/2022] Open
Abstract
Language experience enhances discrimination of speech contrasts at a behavioral- perceptual level, as well as at a pre-attentive level, as indexed by event-related potential (ERP) mismatch negativity (MMN) responses. The enhanced sensitivity could be the result of changes in acoustic resolution and/or long-term memory representations of the relevant information in the auditory cortex. To examine these possibilities, we used a short (ca. 600 ms) vs. long (ca. 2,600 ms) interstimulus interval (ISI) in a passive, oddball discrimination task while obtaining ERPs. These ISI differences were used to test whether cross-linguistic differences in processing Mandarin lexical tone are a function of differences in acoustic resolution and/or differences in long-term memory representations. Bisyllabic nonword tokens that differed in lexical tone categories were presented using a passive listening multiple oddball paradigm. Behavioral discrimination and identification data were also collected. The ERP results revealed robust MMNs to both easy and difficult lexical tone differences for both groups at short ISIs. At long ISIs, there was either no change or an enhanced MMN amplitude for the Mandarin group, but reduced MMN amplitude for the English group. In addition, the Mandarin listeners showed a larger late negativity (LN) discriminative response than the English listeners for lexical tone contrasts in the long ISI condition. Mandarin speakers outperformed English speakers in the behavioral tasks, especially under the long ISI conditions with the more similar lexical tone pair. These results suggest that the acoustic correlates of lexical tone are fairly robust and easily discriminated at short ISIs, when the auditory sensory memory trace is strong. At longer ISIs beyond 2.5 s language-specific experience is necessary for robust discrimination.
Collapse
Affiliation(s)
- Yan H. Yu
- Department of Communication Sciences and Disorders, St. John's UniversityQueens, NY, USA
| | - Valerie L. Shafer
- Ph.D. Program of Speech-Language-Hearing Science, The Graduate Center, City University of New YorkNew York, NY, USA
| | - Elyse S. Sussman
- Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Center, Albert Einstein College of MedicineNew York, NY, USA
| |
Collapse
|
17
|
Lucchese G, Pulvermüller F, Stahl B, Dreyer FR, Mohr B. Therapy-Induced Neuroplasticity of Language in Chronic Post Stroke Aphasia: A Mismatch Negativity Study of (A)Grammatical and Meaningful/less Mini-Constructions. Front Hum Neurosci 2017; 10:669. [PMID: 28111545 PMCID: PMC5216683 DOI: 10.3389/fnhum.2016.00669] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2016] [Accepted: 12/15/2016] [Indexed: 12/20/2022] Open
Abstract
Clinical language performance and neurophysiological correlates of language processing were measured before and after intensive language therapy in patients with chronic (time post stroke >1 year) post stroke aphasia (PSA). As event-related potential (ERP) measure, the mismatch negativity (MMN) was recorded in a distracted oddball paradigm to short spoken sentences. Critical 'deviant' sentence stimuli where either well-formed and meaningful, or syntactically, or lexico-semantically incorrect. After 4 weeks of speech-language therapy (SLT) delivered with high intensity (10.5 h per week), clinical language assessment with the Aachen Aphasia Test battery demonstrated significant linguistic improvements, which were accompanied by enhanced MMN responses. More specifically, MMN amplitudes to grammatically correct and meaningful mini-constructions and to 'jabberwocky' sentences containing a pseudoword significantly increased after therapy. However, no therapy-related changes in MMN responses to syntactically incorrect strings including agreement violations were observed. While MMN increases to well-formed meaningful strings can be explained both at the word and construction levels, the neuroplastic change seen for 'jabberwocky' sentences suggests an explanation in terms of constructions. The results confirm previous reports that intensive SLT leads to improvements of linguistic skills in chronic aphasia patients and now demonstrate that this clinical improvement is associated with enhanced automatic brain indexes of construction processing, although no comparable change is present for ungrammatical strings. Furthermore, the data confirm that the language-induced MMN is a useful tool to map functional language recovery in PSA.
Collapse
Affiliation(s)
- Guglielmo Lucchese
- Brain Language Laboratory, Department of Philosophy and Humanities Freie Universität Berlin, Berlin Germany
| | - Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and HumanitiesFreie Universität Berlin, Berlin Germany; Berlin School of Mind and Brain, Humboldt-Universität zu BerlinBerlin, Germany
| | - Benjamin Stahl
- Brain Language Laboratory, Department of Philosophy and HumanitiesFreie Universität Berlin, Berlin Germany; Department of Neurology, Charité Universitätsmedizin Berlin, Campus MitteBerlin, Germany; Max Planck Institute for Human Cognitive and Brain SciencesLeipzig, Germany
| | - Felix R Dreyer
- Brain Language Laboratory, Department of Philosophy and Humanities Freie Universität Berlin, Berlin Germany
| | - Bettina Mohr
- Department of Psychiatry, Charité Universitätsmedizin Berlin Campus Benjamin Franklin, Berlin Germany
| |
Collapse
|
18
|
Lucchese G, Hanna J, Autenrieb A, Miller TM, Pulvermüller F. Electrophysiological Evidence for Early and Interactive Symbol Access and Rule Processing in Retrieving and Combining Language Constructions. J Cogn Neurosci 2016; 29:254-266. [PMID: 27626234 DOI: 10.1162/jocn_a_01038] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
The human brain stores an immense repertoire of linguistic symbols (morphemes, words) and combines them into a virtually unlimited set of well-formed strings (phrases, sentences) that serve as efficient communicative tools. Communication is hampered, however, if strings include meaningless items (e.g., "pseudomorphemes"), or if the rules for combining string elements are violated. Prior research suggests that, when participants attentively process sentences in a linguistic task, syntactic processing can occur quite early, but lexicosemantic processing, or any interaction involving this factor, is manifest later in time (ca. 400 msec or later). In contrast, recent evidence from passive speech perception paradigms suggests early processing of both combinatorial (morphosyntactic) and storage-related (lexicosemantic) properties. A crucial question is whether these parallel processes might also interact early in processing. Using ERPs in an orthogonal design, we presented spoken word strings to participants while they were distracted from incoming speech to obtain information about automatic language processing mechanisms unaffected by task-related strategies. Stimuli were either (1) well-formed miniconstructions (short pronoun-verb sentences), (2) "unstored" strings containing a pseudomorpheme, (3) "ill-combined" strings violating subject-verb agreement rules, or (4) double violations including both types of errors. We found that by 70-210 msec after the onset of the phrase-final syllable that disambiguated the strings, interactions of lexicosemantic and morphosyntactic deviance were evident in the ERPs. These results argue against serial processing of lexical storage, morphosyntactic combination and their interaction, and in favor of early, simultaneous, and interactive processing of symbols and their combinatorial structures.
Collapse
|
19
|
Grisoni L, Dreyer FR, Pulvermüller F. Somatotopic Semantic Priming and Prediction in the Motor System. Cereb Cortex 2016; 26:2353-66. [PMID: 26908635 PMCID: PMC4830302 DOI: 10.1093/cercor/bhw026] [Citation(s) in RCA: 44] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
The recognition of action-related sounds and words activates motor regions, reflecting the semantic grounding of these symbols in action information; in addition, motor cortex exerts causal influences on sound perception and language comprehension. However, proponents of classic symbolic theories still dispute the role of modality-preferential systems such as the motor cortex in the semantic processing of meaningful stimuli. To clarify whether the motor system carries semantic processes, we investigated neurophysiological indexes of semantic relationships between action-related sounds and words. Event-related potentials revealed that action-related words produced significantly larger stimulus-evoked (Mismatch Negativity-like) and predictive brain responses (Readiness Potentials) when presented in body-part-incongruent sound contexts (e.g., “kiss” in footstep sound context; “kick” in whistle context) than in body-part-congruent contexts, a pattern reminiscent of neurophysiological correlates of semantic priming. Cortical generators of the semantic relatedness effect were localized in areas traditionally associated with semantic memory, including left inferior frontal cortex and temporal pole, and, crucially, in motor areas, where body-part congruency of action sound–word relationships was indexed by a somatotopic pattern of activation. As our results show neurophysiological manifestations of action-semantic priming in the motor cortex, they prove semantic processing in the motor system and thus in a modality-preferential system of the human brain.
Collapse
Affiliation(s)
- Luigi Grisoni
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universtät Berlin, 14195 Berlin, Germany
| | - Felix R Dreyer
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universtät Berlin, 14195 Berlin, Germany
| | - Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universtät Berlin, 14195 Berlin, Germany Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10099 Berlin, Germany
| |
Collapse
|
20
|
Hanna J, Shtyrov Y, Williams J, Pulvermüller F. Early neurophysiological indices of second language morphosyntax learning. Neuropsychologia 2016; 82:18-30. [PMID: 26752451 PMCID: PMC4758809 DOI: 10.1016/j.neuropsychologia.2016.01.001] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2015] [Revised: 12/15/2015] [Accepted: 01/01/2016] [Indexed: 12/22/2022]
Abstract
Humans show variable degrees of success in acquiring a second language (L2). In many cases, morphological and syntactic knowledge remain deficient, although some learners succeed in reaching nativelike levels, even if they begin acquiring their L2 relatively late. In this study, we use psycholinguistic, online language proficiency tests and a neurophysiological index of syntactic processing, the syntactic mismatch negativity (sMMN) to local agreement violations, to compare behavioural and neurophysiological markers of grammar processing between native speakers (NS) of English and non-native speakers (NNS). Variable grammar proficiency was measured by psycholinguistic tests. When NS heard ungrammatical word sequences lacking agreement between subject and verb (e.g. *we kicks), the MMN was enhanced compared with syntactically legal sentences (e.g. he kicks). More proficient NNS also showed this difference, but less proficient NNS did not. The main cortical sources of the MMN responses were localised in bilateral superior temporal areas, where, crucially, source strength of grammar-related neuronal activity correlated significantly with grammatical proficiency of individual L2 speakers as revealed by the psycholinguistic tests. As our results show similar, early MMN indices to morpho-syntactic agreement violations among both native speakers and non-native speakers with high grammar proficiency, they appear consistent with the use of similar brain mechanisms for at least certain aspects of L1 and L2 grammars.
Collapse
Affiliation(s)
- Jeff Hanna
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität, Berlin, Germany; Medical Research Council - Cognition and Brain Sciences Unit, Cambridge, United Kingdom.
| | - Yury Shtyrov
- Center of Functionally Integrative Neuroscience, Aarhus University, Denmark; Medical Research Council - Cognition and Brain Sciences Unit, Cambridge, United Kingdom
| | - John Williams
- Department of Theoretical and Applied Linguistics, University of Cambridge, United Kingdom
| | - Friedemann Pulvermüller
- Brain Language Laboratory, Department of Philosophy and Humanities, Freie Universität, Berlin, Germany; Medical Research Council - Cognition and Brain Sciences Unit, Cambridge, United Kingdom
| |
Collapse
|