1
|
Wolpert M, Ao J, Zhang H, Baum S, Steinhauer K. The child the apple eats: processing of argument structure in Mandarin verb-final sentences. Sci Rep 2024; 14:20459. [PMID: 39227638 PMCID: PMC11372106 DOI: 10.1038/s41598-024-70318-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 08/14/2024] [Indexed: 09/05/2024] Open
Abstract
Mandarin Chinese is typologically unusual among the world's languages in having flexible word order despite a near absence of inflectional morphology. These features of Mandarin challenge conventional linguistic notions such as subject and object and the divide between syntax and semantics. In the present study, we tested monolingual processing of argument structure in Mandarin verb-final sentences, where word order alone is not a reliable cue. We collected participants' responses to a forced agent-assignment task while measuring their electroencephalography data to capture real-time processing throughout each sentence. We found that sentence interpretation was not informed by word order in the absence of other cues, and while the coverbs BA and BEI were strong signals for agent selection, comprehension was a result of multiple cues. These results challenge previous reports of a linear ranking of cue strength. Event-related potentials showed that BA and BEI impacted participants' processing even before the verb was read and that role reversal anomalies elicited an N400 effect without a subsequent semantic P600. This study demonstrates that Mandarin sentence comprehension requires online interaction among cues in a language-specific manner, consistent with models that predict crosslinguistic differences in core sentence processing mechanisms.
Collapse
Affiliation(s)
- Max Wolpert
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, 310027, China.
- Integrated Program in Neuroscience, McGill University, Room 302 Irving Ludmer Building, 1033 Pine Avenue West, Montreal, QC, H3A 1A1, Canada.
- Centre for Research on Brain, Language and Music, 2001 Av. McGill College #6, Montreal, QC, H3A 1G1, Canada.
| | - Jiarui Ao
- Integrated Program in Neuroscience, McGill University, Room 302 Irving Ludmer Building, 1033 Pine Avenue West, Montreal, QC, H3A 1A1, Canada
| | - Hui Zhang
- School of Foreign Languages and Cultures, Nanjing Normal University, Suiyuan Campus, Building 500, Nanjing, 210024, Jiangsu, China
| | - Shari Baum
- Centre for Research on Brain, Language and Music, 2001 Av. McGill College #6, Montreal, QC, H3A 1G1, Canada
- School of Communication Sciences and Disorders, McGill University, 2001 Av. McGill College #8, Montreal, QC, H3A 1G1, Canada
| | - Karsten Steinhauer
- Centre for Research on Brain, Language and Music, 2001 Av. McGill College #6, Montreal, QC, H3A 1G1, Canada
- School of Communication Sciences and Disorders, McGill University, 2001 Av. McGill College #8, Montreal, QC, H3A 1G1, Canada
| |
Collapse
|
2
|
Jano S, Cross ZR, Chatburn A, Schlesewsky M, Bornkessel-Schlesewsky I. Prior Context and Individual Alpha Frequency Influence Predictive Processing during Language Comprehension. J Cogn Neurosci 2024; 36:1898-1936. [PMID: 38820550 DOI: 10.1162/jocn_a_02196] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/02/2024]
Abstract
The extent to which the brain predicts upcoming information during language processing remains controversial. To shed light on this debate, the present study reanalyzed Nieuwland and colleagues' (2018) [Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., et al. Large-scale replication study reveals a limit on probabilistic prediction in language comprehension. eLife, 7, e33468, 2018] replication of DeLong and colleagues (2015) [DeLong, K. A., Urbach, T. P., & Kutas, M. Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience, 8, 1117-1121, 2005]. Participants (n = 356) viewed sentences containing articles and nouns of varying predictability, while their EEG was recorded. We measured ERPs preceding the critical words (namely, the semantic prediction potential), in conjunction with postword N400 patterns and individual neural metrics. ERP activity was compared with two measures of word predictability: cloze probability and lexical surprisal. In contrast to prior literature, semantic prediction potential amplitudes did not increase as cloze probability increased, suggesting that the component may not reflect prediction during natural language processing. Initial N400 results at the article provided evidence against phonological prediction in language, in line with Nieuwland and colleagues' findings. Strikingly, however, when the surprisal of the prior words in the sentence was included in the analysis, increases in article surprisal were associated with increased N400 amplitudes, consistent with prediction accounts. This relationship between surprisal and N400 amplitude was not observed when the surprisal of the two prior words was low, suggesting that expectation violations at the article may be overlooked under highly predictable conditions. Individual alpha frequency also modulated the relationship between article surprisal and the N400, emphasizing the importance of individual neural factors for prediction. The present study extends upon existing neurocognitive models of language and prediction more generally, by illuminating the flexible and subject-specific nature of predictive processing.
Collapse
|
3
|
Raz S. Event-related potentials and behavioral correlates of emotional recognition memory in late pregnancy. Arch Womens Ment Health 2024:10.1007/s00737-024-01503-8. [PMID: 39168912 DOI: 10.1007/s00737-024-01503-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 08/10/2024] [Indexed: 08/23/2024]
Abstract
PURPOSE Research on cognitive and emotional functions during pregnancy challenges the prevalent perception of cognitive decline in pregnant women. This study investigates the behavioral and neural dynamics of cognitive-affective processing in third-trimester pregnant women, comparing them with non-pregnant controls. METHODS Using a 64-channel EEG-ERP system, we recorded brain activity as participants engaged in an emotional word recognition task. This task involved initially viewing a sequence of emotional and neutral words, followed by a recognition test where participants identified each word as 'new' or 'previously seen'. RESULTS Contrary to widespread beliefs about diminished recognition ability during late pregnancy, our results revealed no significant differences in error rates between groups. However, pregnant participants demonstrated slower reaction times. In terms of neural responses, pregnant women exhibited increased amplitudes in the N1, P2, and N400 ERP components, suggesting that they may require additional brain resources compared with non-pregnant individuals to process perceptual information. A significant interaction was observed between pregnancy status and the emotional valence of stimuli. Pregnant women showed heightened N1 and N400 responses to negative words, indicating increased sensitivity to stimuli potentially representing threat. This enhanced response was not observed for positive or neutral words. Furthermore, there was an amplified N1 response to 'new' words, but not to 'old' words. CONCLUSION These findings suggest that late pregnancy is characterized by heightened responsiveness to new and particularly negative stimuli, potentially leading to a more cautious behavioral approach. Heightened vigilance and sensitivity could offer evolutionary advantages, optimizing fetal development and enhancing maternal well-being.
Collapse
Affiliation(s)
- Sivan Raz
- Department of Psychology, The Per Sternberg EEG-ERP Laboratory for the Study of Brain and Behavior, Tel Hai College, Upper Galilee, Israel.
- Department of Behavioral Sciences, The Center for Psychobiological Research, The Max Stern Yezreel Valley College, 19300, Emek Yezreel, Israel.
| |
Collapse
|
4
|
Contier F, Wartenburger I, Weymar M, Rabovsky M. Are the P600 and P3 ERP components linked to the task-evoked pupillary response as a correlate of norepinephrine activity? Psychophysiology 2024; 61:e14565. [PMID: 38469647 DOI: 10.1111/psyp.14565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 02/28/2024] [Accepted: 02/29/2024] [Indexed: 03/13/2024]
Abstract
During language comprehension, anomalies and ambiguities in the input typically elicit the P600 event-related potential component. Although traditionally interpreted as a specific signal of combinatorial operations in sentence processing, the component has alternatively been proposed to be a variant of the oddball-sensitive, domain-general P3 component. In particular, both components might reflect phasic norepinephrine release from the locus coeruleus (LC/NE) to motivationally significant stimuli. In this preregistered study, we tested this hypothesis by relating both components to the task-evoked pupillary response, a putative biomarker of LC/NE activity. 36 participants completed a sentence comprehension task (containing 25% morphosyntactic violations) and a non-linguistic oddball task (containing 20% oddballs), while the EEG and pupil size were co-registered. Our results showed that the task-evoked pupillary response and the ERP amplitudes of both components were similarly affected by both experimental tasks. In the oddball task, there was also a temporally specific relationship between the P3 and the pupillary response beyond the shared oddball effect, thereby further linking the P3 to NE. Because this link was less reliable in the linguistic context, we did not find conclusive evidence for or against a relationship between the P600 and the pupillary response. Still, our findings further stimulate the debate on whether language-related ERPs are indeed specific to linguistic processes or shared across cognitive domains. However, further research is required to verify a potential link between the two ERP positivities and the LC/NE system as the common neural generator.
Collapse
Affiliation(s)
- Friederike Contier
- Cognitive Sciences, Department of Psychology, University of Potsdam, Potsdam, Germany
| | - Isabell Wartenburger
- Cognitive Sciences, Department of Linguistics, University of Potsdam, Potsdam, Germany
| | - Mathias Weymar
- Cognitive Sciences, Department of Psychology, University of Potsdam, Potsdam, Germany
| | - Milena Rabovsky
- Cognitive Sciences, Department of Psychology, University of Potsdam, Potsdam, Germany
| |
Collapse
|
5
|
Brocard S, Wilson VAD, Berton C, Zuberbühler K, Bickel B. A universal preference for animate agents in hominids. iScience 2024; 27:109996. [PMID: 38883826 PMCID: PMC11177197 DOI: 10.1016/j.isci.2024.109996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 03/15/2024] [Accepted: 05/14/2024] [Indexed: 06/18/2024] Open
Abstract
When conversing, humans instantaneously predict meaning from fragmentary and ambiguous mspeech, long before utterance completion. They do this by integrating priors (initial assumptions about the world) with contextual evidence to rapidly decide on the most likely meaning. One powerful prior is attentional preference for agents, which biases sentence processing but universally so only if agents are animate. Here, we investigate the evolutionary origins of this preference, by allowing chimpanzees, gorillas, orangutans, human children, and adults to freely choose between agents and patients in still images, following video clips depicting their dyadic interaction. All participants preferred animate (and occasionally inanimate) agents, although the effect was attenuated if patients were also animate. The findings suggest that a preference for animate agents evolved before language and is not reducible to simple perceptual biases. To conclude, both humans and great apes prefer animate agents in decision tasks, echoing a universal prior in human language processing.
Collapse
Affiliation(s)
- Sarah Brocard
- Department of Comparative Cognition, Institute of Biology, University of Neuchatel, Neuchatel, Switzerland
| | - Vanessa A D Wilson
- Department of Comparative Cognition, Institute of Biology, University of Neuchatel, Neuchatel, Switzerland
- Department of Comparative Language Science, University of Zurich, Zurich, Switzerland
- Center for the Interdisciplinary Study of Language Evolution (ISLE), University of Zurich, Zurich, Switzerland
| | - Chloé Berton
- Department of Comparative Cognition, Institute of Biology, University of Neuchatel, Neuchatel, Switzerland
| | - Klaus Zuberbühler
- Department of Comparative Cognition, Institute of Biology, University of Neuchatel, Neuchatel, Switzerland
- Center for the Interdisciplinary Study of Language Evolution (ISLE), University of Zurich, Zurich, Switzerland
- School of Psychology & Neuroscience, University of St Andrews, St Andrews, Scotland (UK)
| | - Balthasar Bickel
- Department of Comparative Language Science, University of Zurich, Zurich, Switzerland
- Center for the Interdisciplinary Study of Language Evolution (ISLE), University of Zurich, Zurich, Switzerland
| |
Collapse
|
6
|
Hodapp A, Rabovsky M. Error-based Implicit Learning in Language: The Effect of Sentence Context and Constraint in a Repetition Paradigm. J Cogn Neurosci 2024; 36:1048-1070. [PMID: 38530326 DOI: 10.1162/jocn_a_02145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
Prediction errors drive implicit learning in language, but the specific mechanisms underlying these effects remain debated. This issue was addressed in an EEG study manipulating the context of a repeated unpredictable word (repetition of the complete sentence or repetition of the word in a new sentence context) and sentence constraint. For the manipulation of sentence constraint, unexpected words were presented either in high-constraint (eliciting a precise prediction) or low-constraint sentences (not eliciting any specific prediction). Repetition-induced reduction of N400 amplitudes and of power in the alpha/beta frequency band was larger for words repeated with their sentence context as compared with words repeated in a new low-constraint context, suggesting that implicit learning happens not only at the level of individual items but additionally improves sentence-based predictions. These processing benefits for repeated sentences did not differ between constraint conditions, suggesting that sentence-based prediction update might be proportional to the amount of unpredicted semantic information, rather than to the precision of the prediction that was violated. In addition, the consequences of high-constraint prediction violations, as reflected in a frontal positivity and increased theta band power, were reduced with repetition. Overall, our findings suggest a powerful and specific adaptation mechanism that allows the language system to quickly adapt its predictions when unexpected semantic information is processed, irrespective of sentence constraint, and to reduce potential costs of strong predictions that were violated.
Collapse
|
7
|
Nour Eddine S, Brothers T, Wang L, Spratling M, Kuperberg GR. A predictive coding model of the N400. Cognition 2024; 246:105755. [PMID: 38428168 PMCID: PMC10984641 DOI: 10.1016/j.cognition.2024.105755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 02/14/2024] [Accepted: 02/19/2024] [Indexed: 03/03/2024]
Abstract
The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.
Collapse
Affiliation(s)
- Samer Nour Eddine
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America.
| | - Trevor Brothers
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America; Department of Psychology, North Carolina A&T, United States of America
| | - Lin Wang
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America; Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, United States of America
| | | | - Gina R Kuperberg
- Department of Psychology and Center for Cognitive Science, Tufts University, United States of America; Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, United States of America
| |
Collapse
|
8
|
León-Cabrera P, Hjortdal A, Berthelsen SG, Rodríguez-Fornells A, Roll M. Neurophysiological signatures of prediction in language: A critical review of anticipatory negativities. Neurosci Biobehav Rev 2024; 160:105624. [PMID: 38492763 DOI: 10.1016/j.neubiorev.2024.105624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 02/09/2024] [Accepted: 03/13/2024] [Indexed: 03/18/2024]
Abstract
Recent event-related potential (ERP) studies in language comprehension converge in finding anticipatory negativities preceding words or word segments that can be pre-activated based on either sentence contexts or phonological cues. We review these findings from different paradigms in the light of evidence from other cognitive domains in which slow negative potentials have long been associated with anticipatory processes and discuss their potential underlying mechanisms. We propose that this family of anticipatory negativities captures common mechanisms associated with the pre-activation of linguistic information both within words and within sentences. Future studies could utilize these anticipatory negativities in combination with other, well-established ERPs, to simultaneously track prediction-related processes emerging at different time intervals (before and after the perception of pre-activated input) and with distinct time courses (shorter-lived and longer-lived cognitive operations).
Collapse
Affiliation(s)
- Patricia León-Cabrera
- Cognition and Brain Plasticity Unit, Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona, Spain; Department of Basic Sciences, Universitat Internacional de Catalunya, Sant Cugat del Vallès, Barcelona, Spain.
| | - Anna Hjortdal
- Centre for Languages and Literature (SOL), Lund University, Lund, Sweden
| | - Sabine Gosselke Berthelsen
- Centre for Languages and Literature (SOL), Lund University, Lund, Sweden; Department of Nordic Studies and Linguistics (NorS), University of Copenhagen, Copenhagen, Denmark
| | - Antoni Rodríguez-Fornells
- Cognition and Brain Plasticity Unit, Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona, Spain; Institut d'Investigació Biomèdica de Bellvitge (IDIBELL), Hospitalet de Llobregat, Barcelona, Spain; Catalan Institution for Research and Advanced Studies (ICREA), Barcelona, Spain
| | - Mikael Roll
- Centre for Languages and Literature (SOL), Lund University, Lund, Sweden.
| |
Collapse
|
9
|
Zárate-Rochín AM. Contemporary neurocognitive models of memory: A descriptive comparative analysis. Neuropsychologia 2024; 196:108846. [PMID: 38430963 DOI: 10.1016/j.neuropsychologia.2024.108846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 02/27/2024] [Accepted: 02/27/2024] [Indexed: 03/05/2024]
Abstract
The great complexity involved in the study of memory has given rise to numerous hypotheses and models associated with various phenomena at different levels of analysis. This has allowed us to delve deeper in our knowledge about memory but has also made it difficult to synthesize and integrate data from different lines of research. In this context, this work presents a descriptive comparative analysis of contemporary models that address the structure and function of multiple memory systems. The main goal is to outline a panoramic view of the key elements that constitute these models in order to visualize both the current state of research and possible future directions. The elements that stand out from different levels of analysis are distributed neural networks, hierarchical organization, predictive coding, homeostasis, and evolutionary perspective.
Collapse
Affiliation(s)
- Alba Marcela Zárate-Rochín
- Instituto de Investigaciones Cerebrales, Universidad Veracruzana, Dr. Castelazo Ayala s/n, Industrial Animas, 91190, Xalapa-Enríquez, Veracruz, Mexico.
| |
Collapse
|
10
|
Michaelov JA, Bardolph MD, Van Petten CK, Bergen BK, Coulson S. Strong Prediction: Language Model Surprisal Explains Multiple N400 Effects. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2024; 5:107-135. [PMID: 38645623 PMCID: PMC11025652 DOI: 10.1162/nol_a_00105] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Accepted: 03/24/2023] [Indexed: 04/23/2024]
Abstract
Theoretical accounts of the N400 are divided as to whether the amplitude of the N400 response to a stimulus reflects the extent to which the stimulus was predicted, the extent to which the stimulus is semantically similar to its preceding context, or both. We use state-of-the-art machine learning tools to investigate which of these three accounts is best supported by the evidence. GPT-3, a neural language model trained to compute the conditional probability of any word based on the words that precede it, was used to operationalize contextual predictability. In particular, we used an information-theoretic construct known as surprisal (the negative logarithm of the conditional probability). Contextual semantic similarity was operationalized by using two high-quality co-occurrence-derived vector-based meaning representations for words: GloVe and fastText. The cosine between the vector representation of the sentence frame and final word was used to derive contextual cosine similarity estimates. A series of regression models were constructed, where these variables, along with cloze probability and plausibility ratings, were used to predict single trial N400 amplitudes recorded from healthy adults as they read sentences whose final word varied in its predictability, plausibility, and semantic relationship to the likeliest sentence completion. Statistical model comparison indicated GPT-3 surprisal provided the best account of N400 amplitude and suggested that apparently disparate N400 effects of expectancy, plausibility, and contextual semantic similarity can be reduced to variation in the predictability of words. The results are argued to support predictive coding in the human language network.
Collapse
Affiliation(s)
- James A. Michaelov
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Megan D. Bardolph
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Cyma K. Van Petten
- Department of Psychology, Binghamton University, State University of New York, Binghamton, NY, USA
| | - Benjamin K. Bergen
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| | - Seana Coulson
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| |
Collapse
|
11
|
Huber E, Sauppe S, Isasi-Isasmendi A, Bornkessel-Schlesewsky I, Merlo P, Bickel B. Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle. NEUROBIOLOGY OF LANGUAGE (CAMBRIDGE, MASS.) 2024; 5:167-200. [PMID: 38645615 PMCID: PMC11025647 DOI: 10.1162/nol_a_00121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 08/30/2023] [Indexed: 04/23/2024]
Abstract
Language models based on artificial neural networks increasingly capture key aspects of how humans process sentences. Most notably, model-based surprisals predict event-related potentials such as N400 amplitudes during parsing. Assuming that these models represent realistic estimates of human linguistic experience, their success in modeling language processing raises the possibility that the human processing system relies on no other principles than the general architecture of language models and on sufficient linguistic input. Here, we test this hypothesis on N400 effects observed during the processing of verb-final sentences in German, Basque, and Hindi. By stacking Bayesian generalised additive models, we show that, in each language, N400 amplitudes and topographies in the region of the verb are best predicted when model-based surprisals are complemented by an Agent Preference principle that transiently interprets initial role-ambiguous noun phrases as agents, leading to reanalysis when this interpretation fails. Our findings demonstrate the need for this principle independently of usage frequencies and structural differences between languages. The principle has an unequal force, however. Compared to surprisal, its effect is weakest in German, stronger in Hindi, and still stronger in Basque. This gradient is correlated with the extent to which grammars allow unmarked NPs to be patients, a structural feature that boosts reanalysis effects. We conclude that language models gain more neurobiological plausibility by incorporating an Agent Preference. Conversely, theories of human processing profit from incorporating surprisal estimates in addition to principles like the Agent Preference, which arguably have distinct evolutionary roots.
Collapse
Affiliation(s)
- Eva Huber
- Department of Comparative Language Science, University of Zurich, Zurich, Switzerland
- Center for the Interdisciplinary Study of Language Evolution, University of Zurich, Zurich, Switzerland
| | - Sebastian Sauppe
- Department of Comparative Language Science, University of Zurich, Zurich, Switzerland
- Center for the Interdisciplinary Study of Language Evolution, University of Zurich, Zurich, Switzerland
- Department of Psychology, University of Zurich, Zurich, Switzerland
| | - Arrate Isasi-Isasmendi
- Department of Comparative Language Science, University of Zurich, Zurich, Switzerland
- Center for the Interdisciplinary Study of Language Evolution, University of Zurich, Zurich, Switzerland
| | - Ina Bornkessel-Schlesewsky
- Cognitive Neuroscience Laboratory, Australian Research Centre for Interactive and Virtual Environments, University of South Australia, Adelaide, Australia
| | - Paola Merlo
- Department of Linguistics, University of Geneva, Geneva, Switzerland
- University Center for Computer Science, University of Geneva, Geneva, Switzerland
| | - Balthasar Bickel
- Department of Comparative Language Science, University of Zurich, Zurich, Switzerland
- Center for the Interdisciplinary Study of Language Evolution, University of Zurich, Zurich, Switzerland
| |
Collapse
|
12
|
Rubianes M, Drijvers L, Muñoz F, Jiménez-Ortega L, Almeida-Rivera T, Sánchez-García J, Fondevila S, Casado P, Martín-Loeches M. The Self-reference Effect Can Modulate Language Syntactic Processing Even Without Explicit Awareness: An Electroencephalography Study. J Cogn Neurosci 2024; 36:460-474. [PMID: 38165746 DOI: 10.1162/jocn_a_02104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2024]
Abstract
Although it is well established that self-related information can rapidly capture our attention and bias cognitive functioning, whether this self-bias can affect language processing remains largely unknown. In addition, there is an ongoing debate as to the functional independence of language processes, notably regarding the syntactic domain. Hence, this study investigated the influence of self-related content on syntactic speech processing. Participants listened to sentences that could contain morphosyntactic anomalies while the masked face identity (self, friend, or unknown faces) was presented for 16 msec preceding the critical word. The language-related ERP components (left anterior negativity [LAN] and P600) appeared for all identity conditions. However, the largest LAN effect followed by a reduced P600 effect was observed for self-faces, whereas a larger LAN with no reduction of the P600 was found for friend faces compared with unknown faces. These data suggest that both early and late syntactic processes can be modulated by self-related content. In addition, alpha power was more suppressed over the left inferior frontal gyrus only when self-faces appeared before the critical word. This may reflect higher semantic demands concomitant to early syntactic operations (around 150-550 msec). Our data also provide further evidence of self-specific response, as reflected by the N250 component. Collectively, our results suggest that identity-related information is rapidly decoded from facial stimuli and may impact core linguistic processes, supporting an interactive view of syntactic processing. This study provides evidence that the self-reference effect can be extended to syntactic processing.
Collapse
Affiliation(s)
- Miguel Rubianes
- Complutense University of Madrid, Spain
- UCM-ISCIII Center for Human Evolution and Behavior, Madrid, Spain
| | - Linda Drijvers
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Francisco Muñoz
- Complutense University of Madrid, Spain
- UCM-ISCIII Center for Human Evolution and Behavior, Madrid, Spain
| | - Laura Jiménez-Ortega
- Complutense University of Madrid, Spain
- UCM-ISCIII Center for Human Evolution and Behavior, Madrid, Spain
| | | | | | - Sabela Fondevila
- Complutense University of Madrid, Spain
- UCM-ISCIII Center for Human Evolution and Behavior, Madrid, Spain
| | - Pilar Casado
- Complutense University of Madrid, Spain
- UCM-ISCIII Center for Human Evolution and Behavior, Madrid, Spain
| | - Manuel Martín-Loeches
- Complutense University of Madrid, Spain
- UCM-ISCIII Center for Human Evolution and Behavior, Madrid, Spain
| |
Collapse
|
13
|
Lindfors H, Hansson K, Pakulak E, Cohn N, Andersson A. Semantic processing of verbal narratives compared to semantic processing of visual narratives: an ERP study of school-aged children. Front Psychol 2024; 14:1253509. [PMID: 38282837 PMCID: PMC10812112 DOI: 10.3389/fpsyg.2023.1253509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 12/22/2023] [Indexed: 01/30/2024] Open
Abstract
There is a misconception that pictures are easy to comprehend, which is problematic in pedagogical practices that include pictures. For example, if a child has difficulties with verbal narration to picture sequences, it may be interpreted as specific to spoken language even though the child may have additional difficulties with comprehension of visual narratives in the form of picture sequences. The purpose of the present study was therefore to increase our understanding of semantic processing in the pictorial domain in relation to semantic processing in the verbal domain, focusing on 9-13 years-old children with typical language development. To this end, we measured electrical brain responses (event related potentials, ERPs) in 17 children to (i) pictures (panels) that were predicted versus unpredicted in sequences of panels that conveyed visual narratives and (ii) words that were predicted versus unpredicted in sentences that conveyed verbal narratives. Results demonstrated similarities as there were no significant difference in the magnitude of the N400 effect across domains. The only difference between domains was the predicted difference in distribution, that is, a more posterior N400 effect in the verbal domain than in the pictorial domain. The study contributes to an increased understanding of the complexity of processing of visual narratives and its shared features with processing of verbal narratives, which should be considered in pedagogical practices.
Collapse
Affiliation(s)
- Hanna Lindfors
- Linnaeus Language Processing Lab, Department of Swedish, Linnaeus University, Växjö, Sweden
| | - Kristina Hansson
- Logopedics, Phoniatrics and Audiology, Department of Clinical Sciences, Lund University, Lund, Sweden
| | - Eric Pakulak
- Department of Psychology, Stockholm University, Stockholm, Sweden
| | - Neil Cohn
- Visual Language Lab, Tilburg School of Humanities and Digital Sciences, Department of Communication and Cognition, Tilburg University, Tilburg, Netherlands
| | - Annika Andersson
- Linnaeus Language Processing Lab, Department of Swedish, Linnaeus University, Växjö, Sweden
| |
Collapse
|
14
|
Courteau É, Royle P, Steinhauer K. Number agreement processing in adolescents with and without developmental language disorder (DLD): evidence from event-related brain potentials. Sci Rep 2023; 13:22836. [PMID: 38129437 PMCID: PMC10739941 DOI: 10.1038/s41598-023-49121-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 12/04/2023] [Indexed: 12/23/2023] Open
Abstract
In morphologically richer languages, including French, one must learn the specific properties of number agreement in order to understand the language, and this learning process continues into adolescence. This study examined similarities and differences between French-speaking adolescents with and without developmental language disorder (DLD) when processing number agreement, and investigated how morpho-syntactic regularity affected language processing. Using event-related potentials (ERP) and only grammatical sentences with audio-visual mismatches, we studied ERP correlates to three types of number agreement: (1) regular determiner agreement in noun phrases, (2) regular subject-verb plural liaison, and (3) irregular subject-verb agreement. We also included a lexico-semantic mismatch condition to investigate lexico-semantic processing in our participants. 17 adolescents with DLD (M = 14.1 years) and 20 (pre)teens with typical language (TL, M = 12.2 years) participated in the study. Our results suggest three patterns. First, French-speaking teenagers without DLD are still consolidating their neurocognitive processing of morpho-syntactic number agreement and generally display ERP profiles typical of lower language proficiency than adult native speakers. Second, differences in morphosyntactic processing between teenagers with and without DLD seem to be limited to rule-based (regular) number agreement. Third, there is little evidence for corresponding differences in lexico-semantic processing.
Collapse
Affiliation(s)
- Émilie Courteau
- School of Speech Language Pathology and Audiology, Faculty of Medicine, University of Montreal, Montreal, QC, Canada.
- Department of Psychology and Neuroscience, Dalhousie University, Halifax, NS, Canada.
- Centre for Research On Brain, Language and Music (CRBLM), Montreal, QC, Canada.
| | - Phaedra Royle
- School of Speech Language Pathology and Audiology, Faculty of Medicine, University of Montreal, Montreal, QC, Canada
- Centre for Research On Brain, Language and Music (CRBLM), Montreal, QC, Canada
| | - Karsten Steinhauer
- Centre for Research On Brain, Language and Music (CRBLM), Montreal, QC, Canada
- School of Communication Sciences and Disorders, Faculty of Medicine, McGill University, Montreal, QC, Canada
| |
Collapse
|
15
|
Stone K, Khaleghi N, Rabovsky M. The N400 is Elicited by Meaning Changes but not Synonym Substitutions: Evidence From Persian Phrasal Verbs. Cogn Sci 2023; 47:e13394. [PMID: 38088460 DOI: 10.1111/cogs.13394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2023] [Revised: 10/20/2023] [Accepted: 11/22/2023] [Indexed: 12/18/2023]
Abstract
We tested two accounts of the cognitive process underlying the N400 event-related potential component: one that it reflects meaning-based processing and one that it reflects the processing of specific words. The experimental design utilized separable Persian phrasal verbs, which form a strongly probabilistic, long-distance dependency, ideal for the study of probabilistic processing. In sentences strongly constraining for a particular continuation, we show evidence that between two low-probability words, only the word that changed the expected meaning of the sentence increased N400 amplitude, while a synonym of the most probable target word did not. The findings support an account of the N400 in which its underlying process is driven by the processing of meaning rather than of specific words.
Collapse
Affiliation(s)
- Kate Stone
- Department of Psychology, University of Potsdam
| | | | | |
Collapse
|
16
|
Lago S, Pezzetta R, Gastaldon S, Peressotti F, Arcara G. Trial-by-trial fluctuations of pre-stimulus alpha power predict language ERPs. Psychophysiology 2023; 60:e14388. [PMID: 37477167 DOI: 10.1111/psyp.14388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 06/12/2023] [Accepted: 06/20/2023] [Indexed: 07/22/2023]
Abstract
Anticipatory mechanisms are known to play a key role in language, but they have been mostly investigated with violation paradigms, which only consider what happens after predictions have been (dis)confirmed. Relatively few studies focused on the pre-stimulus interval and found that stronger expectations are associated with lower pre-stimulus alpha power. However, alpha power also fluctuates spontaneously, in the absence of experimental manipulations; and in the attention and perception domains, spontaneously low pre-stimulus power is associated with better behavioral performance and with event-related potential (ERPs) with shorter latencies and higher amplitudes. Importantly, little is known about the role of alpha fluctuations in other domains, as it is in language. To this aim, we investigated whether spontaneous fluctuations in pre-stimulus alpha power modulate language-related ERPs in a semantic congruence task. Electrophysiology data were analyzed using Generalized Additive Mixed Models to model nonlinear interactions between pre-stimulus alpha power and EEG amplitude, at the single-trial level. We found that the N400 and the late posterior positivity/P600 were larger in the case of lower pre-stimulus alpha power. Still, while the N400 was observable regardless of the level of pre-stimulus power, a late posterior positivity/P600 effect was only observable for low pre-stimulus alpha power. We discuss these findings in light of the different, albeit connected, functional interpretations of pre-stimulus alpha and the ERPs according to both a nonpredictive interpretation focused on attentional mechanisms and under a predictive processing framework.
Collapse
Affiliation(s)
- Sara Lago
- IRCCS San Camillo Hospital, Venice, Italy
- Padova Neuroscience Centre (PNC), University of Padova, Padova, Italy
| | | | - Simone Gastaldon
- Padova Neuroscience Centre (PNC), University of Padova, Padova, Italy
- Department of Developmental and Social Psychology (DPSS), University of Padova, Padova, Italy
| | - Francesca Peressotti
- Padova Neuroscience Centre (PNC), University of Padova, Padova, Italy
- Department of Developmental and Social Psychology (DPSS), University of Padova, Padova, Italy
- Centro Interdipartimentale di Ricerca "I-APPROVE - International Auditory Processing Project in Venice", Venice, Italy
| | | |
Collapse
|
17
|
Zora H, Wester J, Csépe V. Predictions about prosody facilitate lexical access: Evidence from P50/N100 and MMN components. Int J Psychophysiol 2023; 194:112262. [PMID: 37924955 DOI: 10.1016/j.ijpsycho.2023.112262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Revised: 08/23/2023] [Accepted: 10/28/2023] [Indexed: 11/06/2023]
Abstract
Research into the neural foundation of perception asserts a model where top-down predictions modulate the bottom-up processing of sensory input. Despite becoming increasingly influential in cognitive neuroscience, the precise account of this predictive coding framework remains debated. In this study, we aim to contribute to this debate by investigating how predictions about prosody facilitate speech perception, and to shed light especially on lexical access influenced by simultaneous predictions in different domains, inter alia, prosodic and semantic. Using a passive auditory oddball paradigm, we examined neural responses to prosodic changes, leading to a semantic change as in Dutch nouns canon ['kaːnɔn] 'canon' vs kanon [kaː'nɔn] 'cannon', and used acoustically identical pseudowords as controls. Results from twenty-eight native speakers of Dutch (age range 18-32 years) indicated an enhanced P50/N100 complex to prosodic change in pseudowords as well as an MMN response to both words and pseudowords. The enhanced P50/N100 response to pseudowords is claimed to indicate that all relevant auditory information is still processed by the brain, whereas the reduced response to words might reflect the suppression of information that has already been encoded. The MMN response to pseudowords and words, on the other hand, is best justified by the unification of previously established prosodic representations with sensory and semantic input respectively. This pattern of results is in line with the predictive coding framework acting on multiple levels and is of crucial importance to indicate that predictions about linguistic prosodic information are utilized by the brain as early as 50 ms.
Collapse
Affiliation(s)
- Hatice Zora
- Max Planck Institute for Psycholinguistics, P.O. Box 310 6500, AH, Nijmegen, the Netherlands.
| | - Janniek Wester
- Max Planck Institute for Psycholinguistics, P.O. Box 310 6500, AH, Nijmegen, the Netherlands
| | - Valéria Csépe
- HUN-REN Research Centre of Natural Sciences, Brain Imaging Centre, P.O. Box 286 1519, Budapest, Hungary
| |
Collapse
|
18
|
Ryskin R, Nieuwland MS. Prediction during language comprehension: what is next? Trends Cogn Sci 2023; 27:1032-1052. [PMID: 37704456 DOI: 10.1016/j.tics.2023.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 08/03/2023] [Accepted: 08/04/2023] [Indexed: 09/15/2023]
Abstract
Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.
Collapse
Affiliation(s)
- Rachel Ryskin
- Department of Cognitive and Information Sciences, University of California Merced, 5200 Lake Road, Merced, CA 95343, USA.
| | - Mante S Nieuwland
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands; Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
19
|
Michaelov JA, Bergen BK. Ignoring the alternatives: The N400 is sensitive to stimulus preactivation alone. Cortex 2023; 168:82-101. [PMID: 37678069 DOI: 10.1016/j.cortex.2023.08.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Revised: 05/12/2023] [Accepted: 08/03/2023] [Indexed: 09/09/2023]
Abstract
The N400 component of the event-related brain potential is a neural signal of processing difficulty. In the language domain, it is widely believed to be sensitive to the degree to which a given word or its semantic features have been preactivated in the brain based on the preceding context. However, it has also been shown that the brain often preactivates many words in parallel. It is currently unknown whether the N400 is also affected by the preactivations of alternative words other than the stimulus that is actually presented. This leaves a weak link in the derivation chain-how can we use the N400 to understand the mechanisms of preactivation if we do not know what it indexes? This study directly addresses this gap. We estimate the extent to which all words in a lexicon are preactivated in a given context using the predictions of contemporary large language models. We then directly compare two competing possibilities: that the amplitude of the N400 is sensitive only to the extent to which the stimulus is preactivated, and that it is also sensitive to the preactivation states of the alternatives. We find evidence of the former. This result allows for better grounded inferences about the mechanisms underlying the N400, lexical preactivation in the brain, and language processing more generally.
Collapse
Affiliation(s)
- James A Michaelov
- Department of Cognitive Science, University of California San Diego, La Jolla, CA, USA.
| | - Benjamin K Bergen
- Department of Cognitive Science, University of California San Diego, La Jolla, CA, USA.
| |
Collapse
|
20
|
Ivanova M, Neubert CR, Schmied J, Bendixen A. ERP evidence for Slavic and German word stress cue sensitivity in English. Front Psychol 2023; 14:1193822. [PMID: 37425183 PMCID: PMC10328821 DOI: 10.3389/fpsyg.2023.1193822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Accepted: 05/31/2023] [Indexed: 07/11/2023] Open
Abstract
Word stress is demanding for non-native learners of English, partly because speakers from different backgrounds weight perceptual cues to stress like pitch, intensity, and duration differently. Slavic learners of English and particularly those with a fixed stress language background like Czech and Polish have been shown to be less sensitive to stress in their native and non-native languages. In contrast, German English learners are rarely discussed in a word stress context. A comparison of these varieties can reveal differences in the foreign language processing of speakers from two language families. We use electroencephalography (EEG) to explore group differences in word stress cue perception between Slavic and German learners of English. Slavic and German advanced English speakers were examined in passive multi-feature oddball experiments, where they were exposed to the word impact as an unstressed standard and as deviants stressed on the first or second syllable through higher pitch, intensity, or duration. The results revealed a robust Mismatch Negativity (MMN) component of the event-related potential (ERP) in both language groups in response to all conditions, demonstrating sensitivity to stress changes in a non-native language. While both groups showed higher MMN responses to stress changes to the second than the first syllable, this effect was more pronounced for German than for Slavic participants. Such group differences in non-native English word stress perception from the current and previous studies are argued to speak in favor of customizable language technologies and diversified English curricula compensating for non-native perceptual variation.
Collapse
Affiliation(s)
- Marina Ivanova
- Faculty of Humanities, English and Digital Linguistics, Institute of English and American Studies, Chemnitz University of Technology, Chemnitz, Germany
| | - Christiane R. Neubert
- Faculty of Natural Sciences, Cognitive Systems Lab, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany
| | - Josef Schmied
- Faculty of Humanities, English and Digital Linguistics, Institute of English and American Studies, Chemnitz University of Technology, Chemnitz, Germany
| | - Alexandra Bendixen
- Faculty of Natural Sciences, Cognitive Systems Lab, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany
| |
Collapse
|
21
|
Daria Dołżycka J, Nikadon J, Peter Weis P, Herbert C, Formanowicz M. Linguistic and emotional responses evoked by pseudoword presentation: An EEG and behavioral study. Brain Cogn 2023; 168:105973. [PMID: 37060645 DOI: 10.1016/j.bandc.2023.105973] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 03/15/2023] [Accepted: 03/27/2023] [Indexed: 04/17/2023]
Abstract
When the semantic properties of words are turned off, such as in pseudowords, the grammatical properties of the stimuli indicated through suffixes may provide cues to the meaning. The application of electroencephalography (EEG), combined with the pseudoword paradigm, allows for evaluating the effects of verbs and nouns as linguistic categories within the time course of processing. To contribute to the ongoing discussion regarding the functional processing of words from different grammatical classes, we conducted an EEG experiment, followed by a behavioral lexical decision task (LDT). The EEG and LDT indicated different neural and behavioral reactions to the presented grammar classes, allowing for a deeper understanding of the neuro- and psycholinguistic dimensions of grammar.
Collapse
Affiliation(s)
- Joanna Daria Dołżycka
- Nicolaus Copernicus University, Toruń, Poland; Applied Emotion and Motivation Psychology, Ulm University, Ulm, Germany.
| | - Jan Nikadon
- Center for Research on Social Relations, SWPS University of Social Sciences and Humanities in Warsaw, Poland
| | - Patrick Peter Weis
- Applied Emotion and Motivation Psychology, Ulm University, Ulm, Germany; Center for Research on Social Relations, SWPS University of Social Sciences and Humanities in Warsaw, Poland
| | - Cornelia Herbert
- Applied Emotion and Motivation Psychology, Ulm University, Ulm, Germany
| | - Magdalena Formanowicz
- Nicolaus Copernicus University, Toruń, Poland; Center for Research on Social Relations, SWPS University of Social Sciences and Humanities in Warsaw, Poland
| |
Collapse
|
22
|
Wang L, Schoot L, Brothers T, Alexander E, Warnke L, Kim M, Khan S, Hämäläinen M, Kuperberg GR. Predictive coding across the left fronto-temporal hierarchy during language comprehension. Cereb Cortex 2023; 33:4478-4497. [PMID: 36130089 PMCID: PMC10110445 DOI: 10.1093/cercor/bhac356] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 08/08/2022] [Accepted: 08/15/2022] [Indexed: 11/12/2022] Open
Abstract
We used magnetoencephalography (MEG) and event-related potentials (ERPs) to track the time-course and localization of evoked activity produced by expected, unexpected plausible, and implausible words during incremental language comprehension. We suggest that the full pattern of results can be explained within a hierarchical predictive coding framework in which increased evoked activity reflects the activation of residual information that was not already represented at a given level of the fronto-temporal hierarchy ("error" activity). Between 300 and 500 ms, the three conditions produced progressively larger responses within left temporal cortex (lexico-semantic prediction error), whereas implausible inputs produced a selectively enhanced response within inferior frontal cortex (prediction error at the level of the event model). Between 600 and 1,000 ms, unexpected plausible words activated left inferior frontal and middle temporal cortices (feedback activity that produced top-down error), whereas highly implausible inputs activated left inferior frontal cortex, posterior fusiform (unsuppressed orthographic prediction error/reprocessing), and medial temporal cortex (possibly supporting new learning). Therefore, predictive coding may provide a unifying theory that links language comprehension to other domains of cognition.
Collapse
Affiliation(s)
- Lin Wang
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| | - Lotte Schoot
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| | - Trevor Brothers
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| | - Edward Alexander
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| | - Lena Warnke
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
| | - Minjae Kim
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| | - Sheraz Khan
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
| | - Matti Hämäläinen
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
| | - Gina R Kuperberg
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA 02129, United States
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| |
Collapse
|
23
|
Semantic surprise predicts the N400 brain potential. NEUROIMAGE: REPORTS 2023. [DOI: 10.1016/j.ynirp.2023.100161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/07/2023]
|
24
|
Gastaldon S, Busan P, Arcara G, Peressotti F. Inefficient speech-motor control affects predictive speech comprehension: atypical electrophysiological correlates in stuttering. Cereb Cortex 2023:6995383. [PMID: 36682885 DOI: 10.1093/cercor/bhad004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 12/29/2022] [Accepted: 12/30/2022] [Indexed: 01/24/2023] Open
Abstract
Listeners predict upcoming information during language comprehension. However, how this ability is implemented is still largely unknown. Here, we tested the hypothesis proposing that language production mechanisms have a role in prediction. We studied 2 electroencephalographic correlates of predictability during speech comprehension-pre-target alpha-beta (8-30 Hz) power decrease and the post-target N400 event-related potential effect-in a population with impaired speech-motor control, i.e. adults who stutter (AWS), compared to typically fluent adults (TFA). Participants listened to sentences that could either constrain towards a target word or not, modulating its predictability. As a complementary task, participants also performed context-driven word production. Compared to TFA, AWS not only displayed atypical neural responses in production, but, critically, they showed a different pattern also in comprehension. Specifically, while TFA showed the expected pre-target power decrease, AWS showed a power increase in frontal regions, associated with speech-motor control. In addition, the post-target N400 effect was reduced for AWS with respect to TFA. Finally, we found that production and comprehension power changes were positively correlated in TFA, but not in AWS. Overall, the results support the idea that processes and neural structures prominently devoted to speech planning also support prediction during speech comprehension.
Collapse
Affiliation(s)
- Simone Gastaldon
- Dipartimento di Psicologia dello Sviluppo e della Socializzazione (DPSS), University of Padova, Via Venezia 8, Padova (PD) 35131, Italy.,Padova Neuroscience Center (PNC), University of Padova, Via Giuseppe Orus 2/B, Padova (PD) 35131, Italy
| | - Pierpaolo Busan
- IRCCS Ospedale San Camillo, Via Alberoni 70, Lido (VE) 30126, Italy
| | - Giorgio Arcara
- IRCCS Ospedale San Camillo, Via Alberoni 70, Lido (VE) 30126, Italy
| | - Francesca Peressotti
- Dipartimento di Psicologia dello Sviluppo e della Socializzazione (DPSS), University of Padova, Via Venezia 8, Padova (PD) 35131, Italy.,Padova Neuroscience Center (PNC), University of Padova, Via Giuseppe Orus 2/B, Padova (PD) 35131, Italy.,Centro Interdipartimentale di Ricerca "I-APPROVE-International Auditory Processing Project in Venice", University of Padova, Via Belzoni 160, Padova (PD) 35121, Italy
| |
Collapse
|
25
|
Seghier ML. Multiple functions of the angular gyrus at high temporal resolution. Brain Struct Funct 2023; 228:7-46. [PMID: 35674917 DOI: 10.1007/s00429-022-02512-y] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 05/22/2022] [Indexed: 02/07/2023]
Abstract
Here, the functions of the angular gyrus (AG) are evaluated in the light of current evidence from transcranial magnetic/electric stimulation (TMS/TES) and EEG/MEG studies. 65 TMS/TES and 52 EEG/MEG studies were examined in this review. TMS/TES literature points to a causal role in semantic processing, word and number processing, attention and visual search, self-guided movement, memory, and self-processing. EEG/MEG studies reported AG effects at latencies varying between 32 and 800 ms in a wide range of domains, with a high probability to detect an effect at 300-350 ms post-stimulus onset. A three-phase unifying model revolving around the process of sensemaking is then suggested: (1) early AG involvement in defining the current context, within the first 200 ms, with a bias toward the right hemisphere; (2) attention re-orientation and retrieval of relevant information within 200-500 ms; and (3) cross-modal integration at late latencies with a bias toward the left hemisphere. This sensemaking process can favour accuracy (e.g. for word and number processing) or plausibility (e.g. for comprehension and social cognition). Such functions of the AG depend on the status of other connected regions. The much-debated semantic role is also discussed as follows: (1) there is a strong TMS/TES evidence for a causal semantic role, (2) current EEG/MEG evidence is however weak, but (3) the existing arguments against a semantic role for the AG are not strong. Some outstanding questions for future research are proposed. This review recognizes that cracking the role(s) of the AG in cognition is possible only when its exact contributions within the default mode network are teased apart.
Collapse
Affiliation(s)
- Mohamed L Seghier
- Department of Biomedical Engineering, Khalifa University of Science and Technology, Abu Dhabi, UAE. .,Healthcare Engineering Innovation Center (HEIC), Khalifa University of Science and Technology, Abu Dhabi, UAE.
| |
Collapse
|
26
|
Li J, Bhattasali S, Zhang S, Franzluebbers B, Luh WM, Spreng RN, Brennan JR, Yang Y, Pallier C, Hale J. Le Petit Prince multilingual naturalistic fMRI corpus. Sci Data 2022; 9:530. [PMID: 36038567 PMCID: PMC9424229 DOI: 10.1038/s41597-022-01625-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Accepted: 08/10/2022] [Indexed: 12/24/2022] Open
Abstract
Neuroimaging using more ecologically valid stimuli such as audiobooks has advanced our understanding of natural language comprehension in the brain. However, prior naturalistic stimuli have typically been restricted to a single language, which limited generalizability beyond small typological domains. Here we present the Le Petit Prince fMRI Corpus (LPPC-fMRI), a multilingual resource for research in the cognitive neuroscience of speech and language during naturalistic listening (OpenNeuro: ds003643). 49 English speakers, 35 Chinese speakers and 28 French speakers listened to the same audiobook The Little Prince in their native language while multi-echo functional magnetic resonance imaging was acquired. We also provide time-aligned speech annotation and word-by-word predictors obtained using natural language processing tools. The resulting timeseries data are shown to be of high quality with good temporal signal-to-noise ratio and high inter-subject correlation. Data-driven functional analyses provide further evidence of data quality. This annotated, multilingual fMRI dataset facilitates future re-analysis that addresses cross-linguistic commonalities and differences in the neural substrate of language processing on multiple perceptual and linguistic levels.
Collapse
Affiliation(s)
- Jixing Li
- New York University Abu Dhabi, Neuroscience of Language Lab, Abu Dhabi, UAE.
- Department of Linguistics and Translation, City University of Hong Kong, Hong Kong, Hong Kong.
| | - Shohini Bhattasali
- University of Maryland, Department of Linguistics & Institute of Advanced Computer Studies, College Park, MD, 20742, USA
| | - Shulin Zhang
- University of Georgia, Department of Linguistics, Athens, GA, 30602, USA
| | | | - Wen-Ming Luh
- National Institute on Aging, Baltimore, MD, 21225, USA
| | - R Nathan Spreng
- Laboratory of Brain and Cognition, Montreal Neurological Institute, Department of Neurology and Neurosurgery, Faculty of Medicine, McGill University, Montreal, QC, H3A 2B4, Canada
| | - Jonathan R Brennan
- Department of Linguistics, University of Michigan, Ann Arbor, MI48109, USA
| | - Yiming Yang
- Jiangsu Key Laboratory of Language and Cognitive Neuroscience, Jiangsu Normal University, Xuzhou, 221116, China
| | - Christophe Pallier
- Cognitive Neuroimaging Unit, INSERM, CEA, CNRS, Universit Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - John Hale
- University of Georgia, Department of Linguistics, Athens, GA, 30602, USA.
| |
Collapse
|
27
|
Bornkessel-Schlesewsky I, Sharrad I, Howlett CA, Alday PM, Corcoran AW, Bellan V, Wilkinson E, Kliegl R, Lewis RL, Small SL, Schlesewsky M. Rapid adaptation of predictive models during language comprehension: Aperiodic EEG slope, individual alpha frequency and idea density modulate individual differences in real-time model updating. Front Psychol 2022; 13:817516. [PMID: 36092106 PMCID: PMC9461998 DOI: 10.3389/fpsyg.2022.817516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 07/22/2022] [Indexed: 11/13/2022] Open
Abstract
Predictive coding provides a compelling, unified theory of neural information processing, including for language. However, there is insufficient understanding of how predictive models adapt to changing contextual and environmental demands and the extent to which such adaptive processes differ between individuals. Here, we used electroencephalography (EEG) to track prediction error responses during a naturalistic language processing paradigm. In Experiment 1, 45 native speakers of English listened to a series of short passages. Via a speaker manipulation, we introduced changing intra-experimental adjective order probabilities for two-adjective noun phrases embedded within the passages and investigated whether prediction error responses adapt to reflect these intra-experimental predictive contingencies. To this end, we calculated a novel measure of speaker-based, intra-experimental surprisal (“speaker-based surprisal”) as defined on a trial-by-trial basis and by clustering together adjectives with a similar meaning. N400 amplitude at the position of the critical second adjective was used as an outcome measure of prediction error. Results showed that N400 responses attuned to speaker-based surprisal over the course of the experiment, thus indicating that listeners rapidly adapt their predictive models to reflect local environmental contingencies (here: the probability of one type of adjective following another when uttered by a particular speaker). Strikingly, this occurs in spite of the wealth of prior linguistic experience that participants bring to the laboratory. Model adaptation effects were strongest for participants with a steep aperiodic (1/f) slope in resting EEG and low individual alpha frequency (IAF), with idea density (ID) showing a more complex pattern. These results were replicated in a separate sample of 40 participants in Experiment 2, which employed a highly similar design to Experiment 1. Overall, our results suggest that individuals with a steep aperiodic slope adapt their predictive models most strongly to context-specific probabilistic information. Steep aperiodic slope is thought to reflect low neural noise, which in turn may be associated with higher neural gain control and better cognitive control. Individuals with a steep aperiodic slope may thus be able to more effectively and dynamically reconfigure their prediction-related neural networks to meet current task demands. We conclude that predictive mechanisms in language are highly malleable and dynamic, reflecting both the affordances of the present environment as well as intrinsic information processing capabilities of the individual.
Collapse
Affiliation(s)
- Ina Bornkessel-Schlesewsky
- Cognitive Neuroscience Laboratory, Australian Research Centre for Interactive and Virtual Environments, University of South Australia, Adelaide, SA, Australia
- *Correspondence: Ina Bornkessel-Schlesewsky
| | - Isabella Sharrad
- Cognitive Neuroscience Laboratory, Australian Research Centre for Interactive and Virtual Environments, University of South Australia, Adelaide, SA, Australia
| | - Caitlin A. Howlett
- Innovation, Implementation and Clinical Translation (IIMPACT) in Health, University of South Australia, Adelaide, SA, Australia
| | | | - Andrew W. Corcoran
- Cognition and Philosophy Laboratory, Monash University, Melbourne, VIC, Australia
- Monash Centre for Consciousness and Contemplative Studies, Monash University, Melbourne, VIC, Australia
| | - Valeria Bellan
- Cognitive Neuroscience Laboratory, Australian Research Centre for Interactive and Virtual Environments, University of South Australia, Adelaide, SA, Australia
- Innovation, Implementation and Clinical Translation (IIMPACT) in Health, University of South Australia, Adelaide, SA, Australia
| | - Erica Wilkinson
- Innovation, Implementation and Clinical Translation (IIMPACT) in Health, University of South Australia, Adelaide, SA, Australia
| | - Reinhold Kliegl
- Division of Training and Movement Science, University of Potsdam, Potsdam, Germany
| | - Richard L. Lewis
- Department of Psychology, University of Michigan, Ann Arbor, MI, United States
- Weinberg Institute for Cognitive Science, University of Michigan, Ann Arbor, MI, United States
| | - Steven L. Small
- School of Behavioral and Brain Sciences, University of Texas at Dallas, Dallas, TX, United States
| | - Matthias Schlesewsky
- Cognitive Neuroscience Laboratory, Australian Research Centre for Interactive and Virtual Environments, University of South Australia, Adelaide, SA, Australia
| |
Collapse
|
28
|
Politzer-Ahles S, Lin J, Pan L, Lee KK. N400 Evidence That the Early Stages of Lexical Access Ignore Knowledge About Phonological Alternations. LANGUAGE AND SPEECH 2022; 65:354-376. [PMID: 34080466 PMCID: PMC9014668 DOI: 10.1177/00238309211020026] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
Hearing a word that was already expected often facilitates comprehension, attenuating the amplitude of the N400 event-related brain potential component. On the other hand, hearing a word that was not expected elicits a larger N400. In the present study, we examined whether the N400 would be attenuated when a person hears something that is not exactly what they expected but is a viable alternative pronunciation of the morpheme they expected. This was done using Mandarin syllables, some of which can be pronounced with different lexical tones depending on the context. In two large-sample experiments (total n = 160) testing syllables in isolation and in phonologically viable contexts, we found little evidence that hearing an alternative pronunciation of the expected word attenuates the N400. These results suggest that comprehenders do not take advantage of their knowledge about systematic phonological alternations during the early stages of prediction or discrimination.
Collapse
Affiliation(s)
- Stephen Politzer-Ahles
- Stephen Politzer-Ahles, Department of
Chinese and Bilingual Studies, Hong Kong Polytechnic University, Hong Kong.
| | - Jueyao Lin
- Hong Kong Polytechnic University, Hong
Kong
| | - Lei Pan
- Hong Kong Polytechnic University, Hong
Kong
- Nanyang Technological University,
Singapore
| | | |
Collapse
|
29
|
Weyers I, Mueller J. A Special Role of Syllables, But Not Vowels or Consonants, for Nonadjacent Dependency Learning. J Cogn Neurosci 2022; 34:1467-1487. [PMID: 35604359 DOI: 10.1162/jocn_a_01874] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Successful language processing entails tracking (morpho)syntactic relationships between distant units of speech, so-called nonadjacent dependencies (NADs). Many cues to such dependency relations have been identified, yet the linguistic elements encoding them have received little attention. In the present investigation, we tested whether and how these elements, here syllables, consonants, and vowels, affect behavioral learning success as well as learning-related changes in neural activity in relation to item-specific NAD learning. In a set of two EEG studies with adults, we compared learning under conditions where either all segment types (Experiment 1) or only one segment type (Experiment 2) was informative. The collected behavioral and ERP data indicate that, when all three segment types are available, participants mainly rely on the syllable for NAD learning. With only one segment type available for learning, adults also perform most successfully with syllable-based dependencies. Although we find no evidence for successful learning across vowels in Experiment 2, dependencies between consonants seem to be identified at least passively at the phonetic-feature level. Together, these results suggest that successful item-specific NAD learning may depend on the availability of syllabic information. Furthermore, they highlight consonants' distinctive power to support lexical processes. Although syllables show a clear facilitatory function for NAD learning, the underlying mechanisms of this advantage require further research.
Collapse
|
30
|
Radošević T, Malaia EA, Milković M. Predictive Processing in Sign Languages: A Systematic Review. Front Psychol 2022; 13:805792. [PMID: 35496220 PMCID: PMC9047358 DOI: 10.3389/fpsyg.2022.805792] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Accepted: 03/03/2022] [Indexed: 01/12/2023] Open
Abstract
The objective of this article was to review existing research to assess the evidence for predictive processing (PP) in sign language, the conditions under which it occurs, and the effects of language mastery (sign language as a first language, sign language as a second language, bimodal bilingualism) on the neural bases of PP. This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework. We searched peer-reviewed electronic databases (SCOPUS, Web of Science, PubMed, ScienceDirect, and EBSCO host) and gray literature (dissertations in ProQuest). We also searched the reference lists of records selected for the review and forward citations to identify all relevant publications. We searched for records based on five criteria (original work, peer-reviewed, published in English, research topic related to PP or neural entrainment, and human sign language processing). To reduce the risk of bias, the remaining two authors with expertise in sign language processing and a variety of research methods reviewed the results. Disagreements were resolved through extensive discussion. In the final review, 7 records were included, of which 5 were published articles and 2 were dissertations. The reviewed records provide evidence for PP in signing populations, although the underlying mechanism in the visual modality is not clear. The reviewed studies addressed the motor simulation proposals, neural basis of PP, as well as the development of PP. All studies used dynamic sign stimuli. Most of the studies focused on semantic prediction. The question of the mechanism for the interaction between one’s sign language competence (L1 vs. L2 vs. bimodal bilingual) and PP in the manual-visual modality remains unclear, primarily due to the scarcity of participants with varying degrees of language dominance. There is a paucity of evidence for PP in sign languages, especially for frequency-based, phonetic (articulatory), and syntactic prediction. However, studies published to date indicate that Deaf native/native-like L1 signers predict linguistic information during sign language processing, suggesting that PP is an amodal property of language processing.
Collapse
Affiliation(s)
- Tomislav Radošević
- Laboratory for Sign Language and Deaf Culture Research, Faculty of Education and Rehabilitation Sciences, University of Zagreb, Zagreb, Croatia
| | - Evie A Malaia
- Laboratory for Neuroscience of Dynamic Cognition, Department of Communicative Disorders, College of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States
| | - Marina Milković
- Laboratory for Sign Language and Deaf Culture Research, Faculty of Education and Rehabilitation Sciences, University of Zagreb, Zagreb, Croatia
| |
Collapse
|
31
|
Mechanisms of associative word learning: Benefits from the visual modality and synchrony of labeled objects. Cortex 2022; 152:36-52. [DOI: 10.1016/j.cortex.2022.03.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Revised: 12/05/2021] [Accepted: 03/30/2022] [Indexed: 11/21/2022]
|
32
|
Corcoran AW, Perera R, Koroma M, Kouider S, Hohwy J, Andrillon T. Expectations boost the reconstruction of auditory features from electrophysiological responses to noisy speech. Cereb Cortex 2022; 33:691-708. [PMID: 35253871 PMCID: PMC9890472 DOI: 10.1093/cercor/bhac094] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 02/11/2022] [Accepted: 02/12/2022] [Indexed: 02/04/2023] Open
Abstract
Online speech processing imposes significant computational demands on the listening brain, the underlying mechanisms of which remain poorly understood. Here, we exploit the perceptual "pop-out" phenomenon (i.e. the dramatic improvement of speech intelligibility after receiving information about speech content) to investigate the neurophysiological effects of prior expectations on degraded speech comprehension. We recorded electroencephalography (EEG) and pupillometry from 21 adults while they rated the clarity of noise-vocoded and sine-wave synthesized sentences. Pop-out was reliably elicited following visual presentation of the corresponding written sentence, but not following incongruent or neutral text. Pop-out was associated with improved reconstruction of the acoustic stimulus envelope from low-frequency EEG activity, implying that improvements in perceptual clarity were mediated via top-down signals that enhanced the quality of cortical speech representations. Spectral analysis further revealed that pop-out was accompanied by a reduction in theta-band power, consistent with predictive coding accounts of acoustic filling-in and incremental sentence processing. Moreover, delta-band power, alpha-band power, and pupil diameter were all increased following the provision of any written sentence information, irrespective of content. Together, these findings reveal distinctive profiles of neurophysiological activity that differentiate the content-specific processes associated with degraded speech comprehension from the context-specific processes invoked under adverse listening conditions.
Collapse
Affiliation(s)
- Andrew W Corcoran
- Corresponding author: Room E672, 20 Chancellors Walk, Clayton, VIC 3800, Australia.
| | - Ricardo Perera
- Cognition & Philosophy Laboratory, School of Philosophical, Historical, and International Studies, Monash University, Melbourne, VIC 3800 Australia
| | - Matthieu Koroma
- Brain and Consciousness Group (ENS, EHESS, CNRS), Département d’Études Cognitives, École Normale Supérieure-PSL Research University, Paris 75005, France
| | - Sid Kouider
- Brain and Consciousness Group (ENS, EHESS, CNRS), Département d’Études Cognitives, École Normale Supérieure-PSL Research University, Paris 75005, France
| | - Jakob Hohwy
- Cognition & Philosophy Laboratory, School of Philosophical, Historical, and International Studies, Monash University, Melbourne, VIC 3800 Australia,Monash Centre for Consciousness & Contemplative Studies, Monash University, Melbourne, VIC 3800 Australia
| | - Thomas Andrillon
- Monash Centre for Consciousness & Contemplative Studies, Monash University, Melbourne, VIC 3800 Australia,Paris Brain Institute, Sorbonne Université, Inserm-CNRS, Paris 75013, France
| |
Collapse
|
33
|
Liu M, Chen Y, Schiller NO. Context Matters for Tone and Intonation Processing in Mandarin. LANGUAGE AND SPEECH 2022; 65:52-72. [PMID: 33482696 DOI: 10.1177/0023830920986174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
In tonal languages such as Mandarin, both lexical tone and sentence intonation are primarily signaled by F0. Their F0 encodings are sometimes in conflict and sometimes in congruency. The present study investigated how tone and intonation, with F0 encodings in conflict or in congruency, are processed and how semantic context may affect their processing. To this end, tone and intonation identification experiments were conducted in both semantically neutral and constraining contexts. Results showed that the overall performance of tone identification was better than that of intonation. Specifically, tone identification was seldom affected by intonation information irrespective of semantic contexts. However, intonation identification, particularly question intonation, was susceptible to the final lexical tone identity and affected by the semantic context. In the semantically neutral context, questions ending with a rising tone and a falling tone were equally difficult to identify. In the semantically constraining context, questions ending with a falling tone were much better identified than those ending with a rising tone. This perceptual asymmetry suggests that top-down information provided by the semantically constraining context can play a facilitating role for listeners to disentangle intonational information from tonal information, but mainly in sentences with the lexical falling tone in the final position.
Collapse
Affiliation(s)
- Min Liu
- College of Chinese Language and Culture & Institute of Applied Linguistics, Jinan University, China
| | | | - Niels O Schiller
- Leiden University Centre for Linguistics & Leiden Institute for Brain and Cognition, Leiden University, the Netherlands
| |
Collapse
|
34
|
Goto SG, Cho HJ, Park G, Coyiuto SM, Lewis RS. The Neural Processing of Social Norms in Biculturals: The Relation Between Cultural Tightness and Semantic Processing. Biol Psychol 2022; 170:108321. [DOI: 10.1016/j.biopsycho.2022.108321] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2021] [Revised: 03/08/2022] [Accepted: 03/18/2022] [Indexed: 11/25/2022]
|
35
|
Hidalgo C, Mohamed I, Zielinski C, Schön D. The effect of speech degradation on the ability to track and predict turn structure in conversation. Cortex 2022; 151:105-115. [DOI: 10.1016/j.cortex.2022.01.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 11/15/2021] [Accepted: 01/20/2022] [Indexed: 11/03/2022]
|
36
|
Pell MD, Sethi S, Rigoulot S, Rothermich K, Liu P, Jiang X. Emotional voices modulate perception and predictions about an upcoming face. Cortex 2022; 149:148-164. [DOI: 10.1016/j.cortex.2021.12.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 09/15/2021] [Accepted: 01/05/2022] [Indexed: 11/26/2022]
|
37
|
Federmeier KD. Connecting and considering: Electrophysiology provides insights into comprehension. Psychophysiology 2022; 59:e13940. [PMID: 34520568 PMCID: PMC9009268 DOI: 10.1111/psyp.13940] [Citation(s) in RCA: 31] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2021] [Revised: 08/27/2021] [Accepted: 08/30/2021] [Indexed: 11/29/2022]
Abstract
The ability to rapidly and systematically access knowledge stored in long-term memory in response to incoming sensory information-that is, to derive meaning from the world-lies at the core of human cognition. Research using methods that can precisely track brain activity over time has begun to reveal the multiple cognitive and neural mechanisms that make this possible. In this article, I delineate how a process of connecting affords an effortless, continuous infusion of meaning into human perception. In a relatively invariant time window, uncovered through studies using the N400 component of the event-related potential, incoming sensory information naturally induces a graded landscape of activation across long-term semantic memory, creating what might be called "proto-concepts". Connecting can be (but is not always) followed by a process of further considering those activations, wherein a set of more attentionally demanding "active comprehension" mechanisms mediate the selection, augmentation, and transformation of the initial semantic representations. The result is a limited set of more stable bindings that can be arranged in time or space, revised as needed, and brought to awareness. With this research, we are coming closer to understanding how the human brain is able to fluidly link sensation to experience, to appreciate language sequences and event structures, and, sometimes, to even predict what might be coming up next.
Collapse
Affiliation(s)
- Kara D Federmeier
- Department of Psychology, Program in Neuroscience, and the Beckman Institute for Advanced Science and Technology, University of Illinois, Champaign, Illinois, USA
| |
Collapse
|
38
|
Nour Eddine S, Brothers T, Kuperberg GR. The N400 in silico: A review of computational models. PSYCHOLOGY OF LEARNING AND MOTIVATION 2022. [DOI: 10.1016/bs.plm.2022.03.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
39
|
Implicit auditory perception of local and global irregularities in passive listening condition. Neuropsychologia 2021; 165:108129. [PMID: 34929262 DOI: 10.1016/j.neuropsychologia.2021.108129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2020] [Revised: 12/14/2021] [Accepted: 12/15/2021] [Indexed: 11/19/2022]
Abstract
The auditory system detects differences in sounds at an implicit level, but data on this difference might not be sufficient to make explicit discrimination. The biomarkers of implicit auditory memory of ambiguous stimuli could shed light on unconscious auditory processing and implicit auditory learning. Mismatch negativity (MMN) and P3a, components of event-related potentials (ERPs) reflecting stimuli discrimination without direct attention, were previously detected in response to the local (short-term) irregularity in the auditory sequence even in an unconscious state. At the same time, P3b was elicited only in case of direct attention in response to the global (long-term) irregularity. In this study, we applied the local-global auditory paradigm to obtain possible electrophysiological signatures of implicit detection of hardly distinguishable auditory stimuli. ERPs were recorded from 20 healthy volunteers during active discrimination of deviant sounds in the old-ball sequence and passive listening of the same sounds in the sequence with local-global irregularity. The discrimination task consisted of two blocks with different deviant sounds targeted to respond. The sound discrimination accuracy was at an average of 40%, implying the difficulty of explicit sound recognition. Comparing ERPs to standard and deviant sounds, we found posterior negativity in ERP around 450-600 ms in response to targeted deviant sounds. MMN was significant only in response to non-target deviants. In the passive local-global paradigm, we observed an anterior positivity (284-412 ms), compatible with P3a, in response to a violation of local regularity. Violation of global regularity elicited an anterior negative response (228-586 ms), resembling the N400 component of ERPs. Importantly, the other indexes of auditory discrimination, such as MMN and P3b, were insignificant in ERPs to both regularity violations. The observed P3a and N400 components of ERPs may reflect prediction error signals in the implicit perception of sound patterns even if behavioral recognition was poor.
Collapse
|
40
|
Hörberg T, Jaeger TF. A Rational Model of Incremental Argument Interpretation: The Comprehension of Swedish Transitive Clauses. Front Psychol 2021; 12:674202. [PMID: 34721134 PMCID: PMC8554243 DOI: 10.3389/fpsyg.2021.674202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Accepted: 09/21/2021] [Indexed: 11/16/2022] Open
Abstract
A central component of sentence understanding is verb-argument interpretation, determining how the referents in the sentence are related to the events or states expressed by the verb. Previous work has found that comprehenders change their argument interpretations incrementally as the sentence unfolds, based on morphosyntactic (e.g., case, agreement), lexico-semantic (e.g., animacy, verb-argument fit), and discourse cues (e.g., givenness). However, it is still unknown whether these cues have a privileged role in language processing, or whether their effects on argument interpretation originate in implicit expectations based on the joint distribution of these cues with argument assignments experienced in previous language input. We compare the former, linguistic account against the latter, expectation-based account, using data from production and comprehension of transitive clauses in Swedish. Based on a large corpus of Swedish, we develop a rational (Bayesian) model of incremental argument interpretation. This model predicts the processing difficulty experienced at different points in the sentence as a function of the Bayesian surprise associated with changes in expectations over possible argument interpretations. We then test the model against reading times from a self-paced reading experiment on Swedish. We find Bayesian surprise to be a significant predictor of reading times, complementing effects of word surprisal. Bayesian surprise also captures the qualitative effects of morpho-syntactic and lexico-semantic cues. Additional model comparisons find that it—with a single degree of freedom—captures much, if not all, of the effects associated with these cues. This suggests that the effects of form- and meaning-based cues to argument interpretation are mediated through expectation-based processing.
Collapse
Affiliation(s)
- Thomas Hörberg
- Department of Linguistics, Stockholm University, Stockholm, Sweden.,Department of Computational Science and Technology, KTH Royal Institute of Technology, Stockholm, Sweden
| | - T Florian Jaeger
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, United States.,Department of Computer Science, University of Rochester, Rochester, NY, United States
| |
Collapse
|
41
|
Almeida VN, Radanovic M. Semantic priming and neurobiology in schizophrenia: A theoretical review. Neuropsychologia 2021; 163:108058. [PMID: 34655651 DOI: 10.1016/j.neuropsychologia.2021.108058] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Revised: 10/06/2021] [Accepted: 10/07/2021] [Indexed: 12/11/2022]
Abstract
In this theoretical review we bridge the cognitive and neurobiological sciences to shed light on the neurocognitive foundations of the semantic priming effect in schizophrenia. We review and theoretically evaluate the neurotransmitter systems (dopaminergic, GABAergic and glutamatergic) and neurobiological underpinnings of behavioural and electrophysiological (N400) semantic priming in the pathology, and the main hypotheses on their geneses: a disinhibition of the semantic spread of activation, a disorganised semantic storage or noisy lexical-semantic associations, a psychomotor artefact, an artefact of relatedness proportions, or an inability to mobilise contextual information. We further assess the literature on the endophenotype of Formal Thought Disorder from multiple standpoints, ranging from neurophysiology to cognition: considerations are weaved on neuronal (PV basket cell, SST, VIP) and receptor deficits (DRD1, NMDA), neurotransmitter imbalances (dopamine), cortical and dopaminergic lateralisation, inter alia. In conclusion, we put forth novel postulates on the underlying causes of controlled hypopriming, automatic hyperpriming, N400 reversals (larger amplitudes for close associations), indirect versus direct hyperpriming, and the endophenotype of lexical-semantic disturbances in schizophrenia.
Collapse
Affiliation(s)
- Victor N Almeida
- Faculdade de Letras, Universidade Federal de Minas Gerais (UFMG), Av. Pres. Antônio Carlos, 6627 - Pampulha, Belo Horizonte, MG, 31270-901, Brazil.
| | - Marcia Radanovic
- Laboratório de Neurociências (LIM-27), Faculdade de Medicina, Departamento e Instituto de Psiquiatria, Hospital das Clínicas HCFMUSP, Universidade de São Paulo, Brazil
| |
Collapse
|
42
|
Hodapp A, Rabovsky M. The N400 ERP component reflects an error-based implicit learning signal during language comprehension. Eur J Neurosci 2021; 54:7125-7140. [PMID: 34535935 DOI: 10.1111/ejn.15462] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Accepted: 09/10/2021] [Indexed: 11/26/2022]
Abstract
The functional significance of the N400 evoked-response component is still actively debated. An increasing amount of theoretical and computational modelling work is built on the interpretation of the N400 as a prediction error. In neural network modelling work, it was proposed that the N400 component can be interpreted as the change in a probabilistic representation of meaning that drives the continuous adaptation of an internal model of the statistics of the environment. These results imply that increased N400 amplitudes should correspond to greater adaptation, which can be measured via implicit memory. To investigate this model derived hypothesis, the current study manipulated expectancy in a sentence reading task to influence N400 amplitudes and subsequently presented the previously expected vs. unexpected words in a perceptual identification task to measure implicit memory. As predicted, reaction times in the perceptual identification task were significantly faster for previously unexpected words that induced larger N400 amplitudes in the previous sentence reading task. Additionally, it could be demonstrated that this adaptation seems to specifically depend on the process underlying N400 amplitudes, as participants with larger N400 differences during sentence reading also exhibited a larger implicit memory benefit in the perceptual identification task. These findings support the interpretation of the N400 as an implicit learning signal driving adaptation in language processing.
Collapse
Affiliation(s)
- Alice Hodapp
- Department of Psychology, University of Potsdam, Potsdam, Germany
| | - Milena Rabovsky
- Department of Psychology, University of Potsdam, Potsdam, Germany
| |
Collapse
|
43
|
Brilmayer I, Schumacher PB. Referential Chains Reveal Predictive Processes and Form-to-Function Mapping: An Electroencephalographic Study Using Naturalistic Story Stimuli. Front Psychol 2021; 12:623648. [PMID: 34489770 PMCID: PMC8417467 DOI: 10.3389/fpsyg.2021.623648] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2020] [Accepted: 07/12/2021] [Indexed: 11/13/2022] Open
Abstract
In discourse pragmatics, different referential forms are claimed to be indicative of the cognitive status of a referent in the current discourse. Referential expressions thereby possess a double function: They point back to an (existing) referent (form-to-function mapping), and they are used to derive predictions about a referent’s subsequent recurrence in discourse. Existing event-related potential (ERP) research has mainly focused on the form-to-function mapping of referential expression. In the present ERP study, we explore the relationship of form-to-function mapping and prediction derived from the antecedent of referential expressions in naturalistic auditory language comprehension. Specifically, the study investigates the relationship between the form of a referential expression (pronoun vs. noun) and the form of its antecedent (pronoun vs. noun); i.e., it examines the influence of the interplay of predictions derived from an antecedent (forward-looking function) and the form-to-function mapping of an anaphor (backward-looking function) on the ERPs time-locked to anaphoric expressions. The results in the time range of the P300 and N400 allow for a dissociation of these two functions during online language comprehension.
Collapse
Affiliation(s)
- Ingmar Brilmayer
- Department of German Language and Literature I-Linguistics, University of Cologne, Cologne, Germany
| | - Petra B Schumacher
- Department of German Language and Literature I-Linguistics, University of Cologne, Cologne, Germany
| |
Collapse
|
44
|
Yacovone A, Moya E, Snedeker J. Unexpected words or unexpected languages? Two ERP effects of code-switching in naturalistic discourse. Cognition 2021; 215:104814. [PMID: 34303181 DOI: 10.1016/j.cognition.2021.104814] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2019] [Revised: 06/10/2021] [Accepted: 06/13/2021] [Indexed: 11/19/2022]
Abstract
Bilingual speakers often switch between languages in conversation without any advance notice. Psycholinguistic research has found that these language shifts (or code-switches) can be costly for comprehenders in certain situations. The present study explores the nature of these costs by comparing code-switches to other types of unexpected linguistic material. To do this, we used a novel EEG paradigm, the Storytime task, in which we record readings of natural texts, and then experimentally manipulate their properties by splicing in words. In this study, we manipulated the language of our target words (English, Spanish) and their fit with the preceding context (strong-fit, weak-fit). If code-switching incurs a unique cost beyond that incurred by an unexpected word, then we should see an additive pattern in our ERP indices. If an effect is driven by lexical expectation alone, then there should be a non-additive interaction such that all unexpected forms incur a similar cost. We found three effects: a general prediction effect (a non-additive N400), a post-lexical recognition of the switch in languages (an LPC for code-switched words), and a prolonged integration difficulty associated with weak-fitting words regardless of language (a sustained negativity). We interpret these findings as suggesting that the processing difficulties experienced by bilinguals can largely be understood within more general frameworks for understanding language comprehension. Our findings are consistent with the broader literature demonstrating that bilinguals do not have two wholly separate language systems but rather a single language system capable of using two coding systems.
Collapse
|
45
|
Neurophysiological basis of the N400 deflection, from Mismatch Negativity to Semantic Prediction Potentials and late positive components. Int J Psychophysiol 2021; 166:134-150. [PMID: 34097935 DOI: 10.1016/j.ijpsycho.2021.06.001] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 05/20/2021] [Accepted: 06/02/2021] [Indexed: 11/21/2022]
Abstract
The first theoretical model on the neurophysiological basis of the N400: the deflection reflects layer I dendritic plateaus on a preparatory state of synaptic integration that precedes layer V somatic burst firing for conscious identification of the higher-order features of the stimulus (a late positive shift). Plateaus ensue from apical disinhibition by vasoactive intestinal polypeptide-positive interneurons (VIPs) through suppression of Martinotti cells, opening the gates for glutamatergic feedback to trigger dendritic regenerative potentials. Cholinergic transients contribute to these dynamics directly, holding a central role in the N400 deflection. The stereotypical timing of the (frontal) glutamatergic feedback and the accompanying cholinergic transients account for the enigmatic "invariability" of the peak latency in the face of a gamut of different stimuli and paradigms. The theoretical postulations presented here may bring about unprecedented level of detail for the N400 deflection to be used in the study of schizophrenia, Alzheimer's disease and other higher-order pathologies. The substrates of a late positive component, the Mismatch Negativity and the Semantic Prediction Potentials are also surveyed.
Collapse
|
46
|
Muralikrishnan R, Idrissi A. Salience-weighted agreement feature hierarchy modulates language comprehension. Cortex 2021; 141:168-189. [PMID: 34058618 DOI: 10.1016/j.cortex.2021.03.029] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Revised: 03/01/2020] [Accepted: 03/21/2021] [Indexed: 10/21/2022]
Abstract
The brain establishes relations between elements of an unfolding sentence in order to incrementally build a representation of who is doing what based on various linguistic cues. Many languages systematically mark the verb and/or its arguments to imply the manner in which they are related. A common mechanism to this end is subject-verb agreement, whereby the marking on the verb covaries with one or more of the features such as person, number and gender of the subject argument in a sentence. The cross-linguistic variability of these features would suggest that they may modulate language comprehension differentially based on their relative weightings in a given language. To test this, we investigated the processing of subject-verb agreement in simple intransitive Arabic sentences in a visual event-related brain potential (ERP) study. Specifically, we examined the differences, if any, that ensue in the processing of person, number and gender features during online comprehension, employing sentences in which the verb either showed full agreement with the subject noun (singular or plural) or did not agree in one of the features. ERP responses were measured at the post-nominal verb. Results showed a biphasic negativity-late-positivity effect when the verb did not agree with its subject noun in either of the features, in line with similar findings from other languages. Crucially however, the biphasic effect for agreement violations was systematically graded based on the feature that was violated, which is a novel finding in view of results from other languages. Furthermore, this graded effect was qualitatively different for singular and plural subjects based on the differing salience of the features for each subject-type. These results suggest that agreement features, varying in their salience due to their language-specific weightings, differentially modulate language comprehension. We postulate a Salience-weighted Feature Hierarchy based on our findings and argue that this parsimoniously accounts for the diversity of existing cross-linguistic neurophysiological results on verb agreement processing.
Collapse
Affiliation(s)
- R Muralikrishnan
- Max Planck Institute for Empirical Aesthetics, Frankfurt, Germany.
| | | |
Collapse
|
47
|
Hernández-Gutiérrez D, Muñoz F, Sánchez-García J, Sommer W, Abdel Rahman R, Casado P, Jiménez-Ortega L, Espuny J, Fondevila S, Martín-Loeches M. Situating language in a minimal social context: how seeing a picture of the speaker's face affects language comprehension. Soc Cogn Affect Neurosci 2021; 16:502-511. [PMID: 33470410 PMCID: PMC8094999 DOI: 10.1093/scan/nsab009] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2020] [Revised: 11/29/2020] [Accepted: 01/19/2021] [Indexed: 11/14/2022] Open
Abstract
Natural use of language involves at least two individuals. Some studies have focused on the interaction between senders in communicative situations and how the knowledge about the speaker can bias language comprehension. However, the mere effect of a face as a social context on language processing remains unknown. In the present study, we used event-related potentials to investigate the semantic and morphosyntactic processing of speech in the presence of a photographic portrait of the speaker. In Experiment 1, we show that the N400, a component related to semantic comprehension, increased its amplitude when processed within this minimal social context compared to a scrambled face control condition. Hence, the semantic neural processing of speech is sensitive to the concomitant perception of a picture of the speaker's face, even if irrelevant to the content of the sentences. Moreover, a late posterior negativity effect was found to the presentation of the speaker's face compared to control stimuli. In contrast, in Experiment 2, we found that morphosyntactic processing, as reflected in left anterior negativity and P600 effects, is not notably affected by the presence of the speaker's portrait. Overall, the present findings suggest that the mere presence of the speaker's image seems to trigger a minimal communicative context, increasing processing resources for language comprehension at the semantic level.
Collapse
Affiliation(s)
- David Hernández-Gutiérrez
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
| | - Francisco Muñoz
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
- Department of Psychobiology & Methods for the Behavioural Sciences, Complutense University of Madrid, Madrid 28040, Spain
| | - Jose Sánchez-García
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
| | - Werner Sommer
- Department of Psychology, Humboldt Universität zu Berlin, Berlin 10117, Germany
| | - Rasha Abdel Rahman
- Department of Psychology, Humboldt Universität zu Berlin, Berlin 10117, Germany
| | - Pilar Casado
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
- Department of Psychobiology & Methods for the Behavioural Sciences, Complutense University of Madrid, Madrid 28040, Spain
| | - Laura Jiménez-Ortega
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
- Department of Psychobiology & Methods for the Behavioural Sciences, Complutense University of Madrid, Madrid 28040, Spain
| | - Javier Espuny
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
| | - Sabela Fondevila
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
- Department of Psychobiology & Methods for the Behavioural Sciences, Complutense University of Madrid, Madrid 28040, Spain
| | - Manuel Martín-Loeches
- Cognitive Neuroscience Section, Center UCM-ISCIII for Human Evolution and Behaviour, Madrid 28029, Spain
- Department of Psychobiology & Methods for the Behavioural Sciences, Complutense University of Madrid, Madrid 28040, Spain
| |
Collapse
|
48
|
Fu Z, Monahan PJ. Extracting Phonetic Features From Natural Classes: A Mismatch Negativity Study of Mandarin Chinese Retroflex Consonants. Front Hum Neurosci 2021; 15:609898. [PMID: 33841113 PMCID: PMC8029992 DOI: 10.3389/fnhum.2021.609898] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2020] [Accepted: 02/23/2021] [Indexed: 11/13/2022] Open
Abstract
How speech sounds are represented in the brain is not fully understood. The mismatch negativity (MMN) has proven to be a powerful tool in this regard. The MMN event-related potential is elicited by a deviant stimulus embedded within a series of repeating standard stimuli. Listeners construct auditory memory representations of these standards despite acoustic variability. In most designs that test speech sounds, however, this variation is typically intra-category: All standards belong to the same phonetic category. In the current paper, inter-category variation is presented in the standards. These standards vary in manner of articulation but share a common phonetic feature. In the standard retroflex experimental block, Mandarin Chinese speaking participants are presented with a series of "standard" consonants that share the feature [retroflex], interrupted by infrequent non-retroflex deviants. In the non-retroflex standard experimental block, non-retroflex standards are interrupted by infrequent retroflex deviants. The within-block MMN was calculated, as was the identity MMN (iMMN) to account for intrinsic differences in responses to the stimuli. We only observed a within-block MMN to the non-retroflex deviant embedded in the standard retroflex block. This suggests that listeners extract [retroflex] despite significant inter-category variation. In the non-retroflex standard block, because there is little on which to base a coherent auditory memory representation, no within-block MMN was observed. The iMMN to the retroflex was observed in a late time-window at centro-parieto-occipital electrode sites instead of fronto-central electrodes, where the MMN is typically observed, potentially reflecting the increased difficulty posed by the added variation in the standards. In short, participants can construct auditory memory representations despite significant acoustic and inter-category phonological variation so long as a shared phonetic feature binds them together.
Collapse
Affiliation(s)
- Zhanao Fu
- Department of Linguistics, University of Toronto, Toronto, ON, Canada
| | - Philip J. Monahan
- Department of Linguistics, University of Toronto, Toronto, ON, Canada
- Department of Language Studies, University of Toronto Scarborough, Toronto, ON, Canada
- Department of Psychology, University of Toronto Scarborough, Toronto, ON, Canada
| |
Collapse
|
49
|
Meyer L, Lakatos P, He Y. Language Dysfunction in Schizophrenia: Assessing Neural Tracking to Characterize the Underlying Disorder(s)? Front Neurosci 2021; 15:640502. [PMID: 33692672 PMCID: PMC7937925 DOI: 10.3389/fnins.2021.640502] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Accepted: 02/03/2021] [Indexed: 12/19/2022] Open
Abstract
Deficits in language production and comprehension are characteristic of schizophrenia. To date, it remains unclear whether these deficits arise from dysfunctional linguistic knowledge, or dysfunctional predictions derived from the linguistic context. Alternatively, the deficits could be a result of dysfunctional neural tracking of auditory information resulting in decreased auditory information fidelity and even distorted information. Here, we discuss possible ways for clinical neuroscientists to employ neural tracking methodology to independently characterize deficiencies on the auditory-sensory and abstract linguistic levels. This might lead to a mechanistic understanding of the deficits underlying language related disorder(s) in schizophrenia. We propose to combine naturalistic stimulation, measures of speech-brain synchronization, and computational modeling of abstract linguistic knowledge and predictions. These independent but likely interacting assessments may be exploited for an objective and differential diagnosis of schizophrenia, as well as a better understanding of the disorder on the functional level-illustrating the potential of neural tracking methodology as translational tool in a range of psychotic populations.
Collapse
Affiliation(s)
- Lars Meyer
- Research Group Language Cycles, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Clinic for Phoniatrics and Pedaudiology, University Hospital Münster, Münster, Germany
| | - Peter Lakatos
- Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, United States
| | - Yifei He
- Department of Psychiatry and Psychotherapy, Philipps-University Marburg, Marburg, Germany
| |
Collapse
|
50
|
Cohn N. A starring role for inference in the neurocognition of visual narratives. COGNITIVE RESEARCH-PRINCIPLES AND IMPLICATIONS 2021; 6:8. [PMID: 33587244 PMCID: PMC7884514 DOI: 10.1186/s41235-021-00270-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Accepted: 01/06/2021] [Indexed: 11/10/2022]
Abstract
Research in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped “action stars” where a reader knows events are undepicted at that moment, rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a “noise” panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300–500 ms), but action stars and noise panels then differed in their later effects (500–900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.
Collapse
Affiliation(s)
- Neil Cohn
- Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, P.O. Box 90153, 5000 LE, Tilburg, The Netherlands.
| |
Collapse
|