Early activation of Broca’s area in grammar processing as revealed by the syntactic mismatch negativity and distributed source analysis

Morphosyntactic prediction in automatic neural processing of spoken language: EEG evidence

Automatic parsing of syntactic information by the human brain is a well-established phenomenon, but its mechanisms remain poorly understood. Its best-known neurophysiological reflection is the so-called early left-anterior negativity (ELAN) component of event-related potentials (ERPs), with two alternative hypotheses for its origin: (1) error detection, or (2) morphosyntactic prediction/priming. To test these alternatives, we conducted two experiments using a non-attend passive design with visual distraction and recorded ERPs to spoken pronoun-verb phrases with/without agreement violations and to the same critical verbs presented in isolation without preceding pronouns. The results revealed an ELAN at ∼130-220 ms for pronoun-verb gender agreement violations, confirming a high degree of automaticity in early morphosyntactic parsing. Critically, the strongest ELAN was elicited by verbs outside phrasal context, which suggests that the typical ELAN pattern is underpinned by a reduction of ERP amplitudes for felicitous combinations, reflecting syntactic priming/predictability between related words/morphemes (potentially mediated by associative links formed during previous linguistic experience) rather than specialised error-detection processes.

Brain oscillatory processes related to sequence memory in healthy older adults

Sequence memory is subject to age-related decline, but the underlying processes are not yet fully understood. We analyzed electroencephalography (EEG) in 21 healthy older (60-80 years) and 26 young participants (20-30 years) and compared time-frequency spectra and theta-gamma phase-amplitude-coupling (PAC) during encoding of the order of visually presented items. In older adults, desynchronization in theta (4-8 Hz) and synchronization in gamma (30-45 Hz) power did not distinguish between subsequently correctly and incorrectly remembered trials, while there was a subsequent memory effect for young adults. Theta-gamma PAC was modulated by item position within a sequence for older but not young adults. Specifically, position within a sequence was coded by higher gamma amplitude for successive theta phases for later correctly remembered trials. Thus, deficient differentiation in theta desynchronization and gamma oscillations during sequence encoding in older adults may reflect neurophysiological correlates of age-related memory decline. Furthermore, our results indicate that sequences are coded by theta-gamma PAC in older adults, but that this mechanism might lose precision in aging.

Speech comprehension across time, space, frequency, and age: MEG-MVPA classification of intertrial phase coherence

Language is a key part of human cognition, essential for our well-being at all stages of our lives. Whereas many neurocognitive abilities decline with age, for language the picture is much less clear, and how exactly speech comprehension changes with ageing is still unknown. To investigate this, we employed magnetoencephalography (MEG) and recorded neuromagnetic brain responses to auditory linguistic stimuli in healthy participants of younger and older age using a passive task-free paradigm and a range of different linguistic stimulus contrasts, which enabled us to assess neural processing of spoken language at multiple levels (lexical, semantic, morphosyntactic). Using machine learning-based classification algorithms to scrutinise intertrial phase coherence of MEG responses in cortical source space, we found that patterns of oscillatory neural activity diverged between younger and older participants across several frequency bands (alpha, beta, gamma) for all tested linguistic information types. The results suggest multiple age-related changes in the brain's neurolinguistic circuits, which may be due to both healthy ageing in general and compensatory processes in particular.

Online neurostimulation of Broca's area does not interfere with syntactic predictions: A combined TMS-EEG approach to basic linguistic combination

Categorical predictions have been proposed as the key mechanism supporting the fast pace of syntactic composition in language. Accordingly, grammar-based expectations are formed-e.g., the determiner "a" triggers the prediction for a noun-and facilitate the analysis of incoming syntactic information, which is then checked against a single or few other word categories. Previous functional neuroimaging studies point towards Broca's area in the left inferior frontal gyrus (IFG) as one fundamental cortical region involved in categorical prediction during incremental language processing. Causal evidence for this hypothesis is however still missing. In this study, we combined Electroencephalography (EEG) and Transcranial Magnetic Stimulation (TMS) to test whether Broca's area is functionally relevant in predictive mechanisms for language. We transiently perturbed Broca's area during the first word in a two-word construction, while simultaneously measuring the Event-Related Potential (ERP) correlates of syntactic composition. We reasoned that if Broca's area is involved in predictive mechanisms for syntax, disruptive TMS during the first word would mitigate the difference in the ERP responses for predicted and unpredicted categories in basic two-word constructions. Contrary to this hypothesis, perturbation of Broca's area at the predictive stage did not affect the ERP correlates of basic composition. The correlation strength between the electrical field induced by TMS and the ERP responses further confirmed this pattern. We discuss the present results considering an alternative account of the role of Broca's area in syntactic composition, namely the bottom-up integration of words into constituents, and of compensatory mechanisms within the language predictive network.

The cognitive hearing science perspective on perceiving, understanding, and remembering language: The ELU model

The review gives an introductory description of the successive development of data patterns based on comparisons between hearing-impaired and normal hearing participants' speech understanding skills, later prompting the formulation of the Ease of Language Understanding (ELU) model. The model builds on the interaction between an input buffer (RAMBPHO, Rapid Automatic Multimodal Binding of PHOnology) and three memory systems: working memory (WM), semantic long-term memory (SLTM), and episodic long-term memory (ELTM). RAMBPHO input may either match or mismatch multimodal SLTM representations. Given a match, lexical access is accomplished rapidly and implicitly within approximately 100-400 ms. Given a mismatch, the prediction is that WM is engaged explicitly to repair the meaning of the input - in interaction with SLTM and ELTM - taking seconds rather than milliseconds. The multimodal and multilevel nature of representations held in WM and LTM are at the center of the review, being integral parts of the prediction and postdiction components of language understanding. Finally, some hypotheses based on a selective use-disuse of memory systems mechanism are described in relation to mild cognitive impairment and dementia. Alternative speech perception and WM models are evaluated, and recent developments and generalisations, ELU model tests, and boundaries are discussed.

Neurophysiological correlates of automatic integration of voice and gender information during grammatical processing

During verbal communication, interlocutors rely on both linguistic (e.g., words, syntax) and extralinguistic (e.g., voice quality) information. The neural mechanisms of extralinguistic information processing are particularly poorly understood. To address this, we used EEG and recorded event-related brain potentials while participants listened to Russian pronoun-verb phrases presented in either male or female voice. Crucially, we manipulated congruency between the grammatical gender signaled by the verbs' ending and the speakers' apparent gender. To focus on putative automatic integration of extralinguistic information into syntactic processing and avoid confounds arising from secondary top-down processes, we used passive non-attend auditory presentation with visual distraction and no stimulus-related task. Most expressed neural responses were found at both early (150 ms, ELAN-like) and late (400 ms, N400-like) phrase processing stages. Crucially, both of these brain responses exhibited sensitivity to extralinguistic information and were significantly enhanced for phrases whose voice and grammatical gender were incongruent, similar to what is known for ERPs effects related to overt grammatical violations. Our data suggest a high degree of automaticity in processing extralinguistic information during spoken language comprehension which indicates existence of a rapid automatic syntactic integration mechanism sensitive to both linguistic and extralinguistic information.

OUP accepted manuscript

During conversations, speech prosody provides important clues about the speaker’s communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. Here, the neurophysiological basis of intonation and speech act understanding were investigated with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level. Already approximately 100 ms after the sentence-final word differing in prosody, questions, and statements expressed with the same sentences led to different neurophysiological activity recorded in the event-related potential. Interestingly, low-pass filtered sentences and acoustically matched nonvocal musical signals failed to show any neurophysiological dissociations, thus suggesting that the physical intonation alone cannot explain this modulation. Our results show rapid neurophysiological indexes of prosodic communicative information processing that emerge only when pragmatic and lexico-semantic information are fully expressed. The early enhancement of question-related activity compared with statements was due to sources in the articulatory-motor region, which may reflect the richer action knowledge immanent to questions, namely the expectation of the partner action of answering the question. The present findings demonstrate a neurophysiological correlate of prosodic communicative information processing, which enables humans to rapidly detect and understand speaker intentions in linguistic interactions.

Brain signatures predict communicative function of speech production in interaction

People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.

Applied potential of task-free event-related paradigms for assessing neurocognitive functions in disorders of consciousness

Diagnosing patients with disorders of consciousness is immensely difficult and often results in misdiagnoses, which can have fatal consequences. Despite the severity of this well-known issue, a reliable assessment tool has not yet been developed and implemented in the clinic. The main aim of this focused review is to evaluate the various event-related potential paradigms, recorded using EEG, which may be used to improve the assessment of patients with disorders of consciousness; we also provide a brief comparison of these paradigms with other measures. Notably, most event-related potential studies on the topic have focused on testing a small set of components, or even just a single component. However, to be of practical use, we argue that an assessment should probe a range of cognitive and linguistic functions at once. We suggest a novel approach that combines a set of well-tested auditory event-related potential components: N100, mismatch negativity, P3a, N400, early left anterior negativity and lexical response enhancement. Combining these components in a single, task-free design will provide a multidimensional assessment of cognitive and linguistic processes, which may help physicians make a more precise diagnosis.

Neurophysiological evidence for rapid processing of verbal and gestural information in understanding communicative actions

During everyday social interaction, gestures are a fundamental part of human communication. The communicative pragmatic role of hand gestures and their interaction with spoken language has been documented at the earliest stage of language development, in which two types of indexical gestures are most prominent: the pointing gesture for directing attention to objects and the give-me gesture for making requests. Here we study, in adult human participants, the neurophysiological signatures of gestural-linguistic acts of communicating the pragmatic intentions of naming and requesting by simultaneously presenting written words and gestures. Already at ~150 ms, brain responses diverged between naming and request actions expressed by word-gesture combination, whereas the same gestures presented in isolation elicited their earliest neurophysiological dissociations significantly later (at ~210 ms). There was an early enhancement of request-evoked brain activity as compared with naming, which was due to sources in the frontocentral cortex, consistent with access to action knowledge in request understanding. In addition, an enhanced N400-like response indicated late semantic integration of gesture-language interaction. The present study demonstrates that word-gesture combinations used to express communicative pragmatic intentions speed up the brain correlates of comprehension processes – compared with gesture-only understanding – thereby calling into question current serial linguistic models viewing pragmatic function decoding at the end of a language comprehension cascade. Instead, information about the social-interactive role of communicative acts is processed instantaneously.

Morphological processing in the brain: The good (inflection), the bad (derivation) and the ugly (compounding)

There is considerable behavioral evidence that morphologically complex words such as 'tax-able' and 'kiss-es' are processed and represented combinatorially. In other words, they are decomposed into their constituents 'tax' and '-able' during comprehension (reading or listening), and producing them might also involve on-the-spot combination of these constituents (especially for inflections). However, despite increasing amount of neurocognitive research, the neural mechanisms underlying these processes are still not fully understood. The purpose of this critical review is to offer a comprehensive overview on the state-of-the-art of the research on the neural mechanisms of morphological processing. In order to take into account all types of complex words, we include findings on inflected, derived, and compound words presented both visually and aurally. More specifically, we cover a wide range of electro- and magnetoencephalography (EEG and MEG, respectively) as well as structural/functional magnetic resonance imaging (s/fMRI) studies that focus on morphological processing. We present the findings with respect to the temporal course and localization of morphologically complex word processing. We summarize the observed findings, their interpretations with respect to current psycholinguistic models, and discuss methodological approaches as well as their possible limitations.

Task-free auditory EEG paradigm for probing multiple levels of speech processing in the brain

While previous studies on language processing highlighted several ERP components in relation to specific stages of sound and speech processing, no study has yet combined them to obtain a comprehensive picture of language abilities in a single session. Here, we propose a novel task-free paradigm aimed at assessing multiple levels of speech processing by combining various speech and nonspeech sounds in an adaptation of a multifeature passive oddball design. We recorded EEG in healthy adult participants, who were presented with these sounds in the absence of sound-directed attention while being engaged in a primary visual task. This produced a range of responses indexing various levels of sound processing and language comprehension: (a) P1-N1 complex, indexing obligatory auditory processing; (b) P3-like dynamics associated with involuntary attention allocation for unusual sounds; (c) enhanced responses for native speech (as opposed to nonnative phonemes) from ∼50 ms from phoneme onset, indicating phonological processing; (d) amplitude advantage for familiar real words as opposed to meaningless pseudowords, indexing automatic lexical access; (e) topographic distribution differences in the cortical activation of action verbs versus concrete nouns, likely linked with the processing of lexical semantics. These multiple indices of speech-sound processing were acquired in a single attention-free setup that does not require any task or subject cooperation; subject to future research, the present protocol may potentially be developed into a useful tool for assessing the status of auditory and linguistic functions in uncooperative or unresponsive participants, including a range of clinical or developmental populations.

Syntactic processing in music and language: Parallel abnormalities observed in congenital amusia

Evidence is accumulating that similar cognitive resources are engaged to process syntactic structure in music and language. Congenital amusia – a neurodevelopmental disorder that primarily affects music perception, including musical syntax – provides a special opportunity to understand the nature of this overlap. Using electroencephalography (EEG), we investigated whether individuals with congenital amusia have parallel deficits in processing language syntax in comparison to control participants. Twelve amusic participants (eight females) and 12 control participants (eight females) were presented melodies in one session, and spoken sentences in another session, both of which had syntactic-congruent and -incongruent stimuli. They were asked to complete a music-related and a language-related task that were irrelevant to the syntactic incongruities. Our results show that amusic participants exhibit impairments in the early stages of both music- and language-syntactic processing. Specifically, we found that two event-related potential (ERP) components – namely Early Right Anterior Negativity (ERAN) and Left Anterior Negativity (LAN), associated with music- and language-syntactic processing respectively, were absent in the amusia group. However, at later processing stages, amusics showed similar brain responses as controls to syntactic incongruities in both music and language. This was reflected in a normal N5 in response to melodies and a normal P600 to spoken sentences. Notably, amusics' parallel music- and language-syntactic impairments were not accompanied by deficits in semantic processing (indexed by normal N400 in response to semantic incongruities). Together, our findings provide further evidence for shared music and language syntactic processing, particularly at early stages of processing.

•

Amusics displayed abnormal brain responses to music-syntactic irregularities.

•

They also exhibited abnormal brain responses to language-syntactic irregularities.

•

These impairments affect an early stage of syntactic processing not a later stage.

•

Music and language involve similar cognitive mechanisms for processing syntax.

Spread the word: MMN brain response reveals whole-form access of discontinuous particle verbs

The status of particle verbs such as rise (…) up as either lexically stored or combinatorially assembled is an issue which so far has not been settled decisively. In this study, we use the mismatch negativity (MMN) brain response to observe neurophysiological responses to discontinuous particle verbs. The MMN can be used to distinguish between whole-form storage and combinatorial processes, as it is enhanced to stored words compared to unknown pseudowords, whereas combinatorially legal strings elicit a reduced MMN relative to ungrammatical ones. Earlier work had found larger MMNs to congruent than to incongruent verb-particle combinations when particle and verb appeared as adjacent elements, thus suggesting whole-form storage at least in this case. However, it is still possible that particle verbs discontinuously spread out across a sentence would elicit the combinatorial, grammar-violation response pattern instead. Here, we tested the brain signatures of discontinuous verb-particle combinations, orthogonally varying congruence and semantic transparency. The results show for the first time brain indices of whole-form storage for discontinuous constituents, thus arguing in favour of access to whole-form-stored lexical elements in the processing of particle verbs, irrespective of their semantic opacity. Results are discussed in the context of linguistic debates about the status of particle verbs as words, lexical elements or syntactically generated combinations. The explanation of the pattern of results within a neurobiological language model is highlighted.

Therapy-Induced Neuroplasticity of Language in Chronic Post Stroke Aphasia: A Mismatch Negativity Study of (A)Grammatical and Meaningful/less Mini-Constructions

Clinical language performance and neurophysiological correlates of language processing were measured before and after intensive language therapy in patients with chronic (time post stroke >1 year) post stroke aphasia (PSA). As event-related potential (ERP) measure, the mismatch negativity (MMN) was recorded in a distracted oddball paradigm to short spoken sentences. Critical 'deviant' sentence stimuli where either well-formed and meaningful, or syntactically, or lexico-semantically incorrect. After 4 weeks of speech-language therapy (SLT) delivered with high intensity (10.5 h per week), clinical language assessment with the Aachen Aphasia Test battery demonstrated significant linguistic improvements, which were accompanied by enhanced MMN responses. More specifically, MMN amplitudes to grammatically correct and meaningful mini-constructions and to 'jabberwocky' sentences containing a pseudoword significantly increased after therapy. However, no therapy-related changes in MMN responses to syntactically incorrect strings including agreement violations were observed. While MMN increases to well-formed meaningful strings can be explained both at the word and construction levels, the neuroplastic change seen for 'jabberwocky' sentences suggests an explanation in terms of constructions. The results confirm previous reports that intensive SLT leads to improvements of linguistic skills in chronic aphasia patients and now demonstrate that this clinical improvement is associated with enhanced automatic brain indexes of construction processing, although no comparable change is present for ungrammatical strings. Furthermore, the data confirm that the language-induced MMN is a useful tool to map functional language recovery in PSA.

Electrophysiological Evidence for Early and Interactive Symbol Access and Rule Processing in Retrieving and Combining Language Constructions

The human brain stores an immense repertoire of linguistic symbols (morphemes, words) and combines them into a virtually unlimited set of well-formed strings (phrases, sentences) that serve as efficient communicative tools. Communication is hampered, however, if strings include meaningless items (e.g., "pseudomorphemes"), or if the rules for combining string elements are violated. Prior research suggests that, when participants attentively process sentences in a linguistic task, syntactic processing can occur quite early, but lexicosemantic processing, or any interaction involving this factor, is manifest later in time (ca. 400 msec or later). In contrast, recent evidence from passive speech perception paradigms suggests early processing of both combinatorial (morphosyntactic) and storage-related (lexicosemantic) properties. A crucial question is whether these parallel processes might also interact early in processing. Using ERPs in an orthogonal design, we presented spoken word strings to participants while they were distracted from incoming speech to obtain information about automatic language processing mechanisms unaffected by task-related strategies. Stimuli were either (1) well-formed miniconstructions (short pronoun-verb sentences), (2) "unstored" strings containing a pseudomorpheme, (3) "ill-combined" strings violating subject-verb agreement rules, or (4) double violations including both types of errors. We found that by 70-210 msec after the onset of the phrase-final syllable that disambiguated the strings, interactions of lexicosemantic and morphosyntactic deviance were evident in the ERPs. These results argue against serial processing of lexical storage, morphosyntactic combination and their interaction, and in favor of early, simultaneous, and interactive processing of symbols and their combinatorial structures.

Early neurophysiological indices of second language morphosyntax learning

Humans show variable degrees of success in acquiring a second language (L2). In many cases, morphological and syntactic knowledge remain deficient, although some learners succeed in reaching nativelike levels, even if they begin acquiring their L2 relatively late. In this study, we use psycholinguistic, online language proficiency tests and a neurophysiological index of syntactic processing, the syntactic mismatch negativity (sMMN) to local agreement violations, to compare behavioural and neurophysiological markers of grammar processing between native speakers (NS) of English and non-native speakers (NNS). Variable grammar proficiency was measured by psycholinguistic tests. When NS heard ungrammatical word sequences lacking agreement between subject and verb (e.g. *we kicks), the MMN was enhanced compared with syntactically legal sentences (e.g. he kicks). More proficient NNS also showed this difference, but less proficient NNS did not. The main cortical sources of the MMN responses were localised in bilateral superior temporal areas, where, crucially, source strength of grammar-related neuronal activity correlated significantly with grammatical proficiency of individual L2 speakers as revealed by the psycholinguistic tests. As our results show similar, early MMN indices to morpho-syntactic agreement violations among both native speakers and non-native speakers with high grammar proficiency, they appear consistent with the use of similar brain mechanisms for at least certain aspects of L1 and L2 grammars.

Sound representation in higher language areas during language generation

How language is encoded by neural activity in the higher-level language areas of humans is still largely unknown. We investigated whether the electrophysiological activity of Broca's area correlates with the sound of the utterances produced. During speech perception, the electric cortical activity of the auditory areas correlates with the sound envelope of the utterances. In our experiment, we compared the electrocorticogram recorded during awake neurosurgical operations in Broca's area and in the dominant temporal lobe with the sound envelope of single words versus sentences read aloud or mentally by the patients. Our results indicate that the electrocorticogram correlates with the sound envelope of the utterances, starting before any sound is produced and even in the absence of speech, when the patient is reading mentally. No correlations were found when the electrocorticogram was recorded in the superior parietal gyrus, an area not directly involved in language generation, or in Broca's area when the participants were executing a repetitive motor task, which did not include any linguistic content, with their dominant hand. The distribution of suprathreshold correlations across frequencies of cortical activities varied whether the sound envelope derived from words or sentences. Our results suggest the activity of language areas is organized by sound when language is generated before any utterance is produced or heard.

Neurophysiological evidence for whole form retrieval of complex derived words: a mismatch negativity study

Complex words can be seen as combinations of elementary units, decomposable into stems and affixes according to morphological rules. Alternatively, complex forms may be stored as single lexical entries and accessed as whole forms. This study uses an event-related potential brain response capable of indexing both whole-form retrieval and combinatorial processing, the Mismatch Negativity (MMN), to investigate early brain activity elicited by morphologically complex derived words in German. We presented complex words consisting of stems “sicher” (secure), or “sauber” (clean) combined with abstract nominalizing derivational affixes -heit or -keit, to form either congruent derived words: “Sicherheit” (security) and “Sauberkeit” (cleanliness), or incongruent derived pseudowords: *“Sicherkeit”, and *“Sauberheit”. Using this orthogonal design, it was possible to record brain responses for -heit and -keit in both congruent and incongruent contexts, therefore balancing acoustic variance. Previous research has shown that incongruent combinations of symbols elicit a stronger MMN than congruent combinations, but that single words or constructions stored as whole forms elicit a stronger MMN than pseudowords or non-existent constructions. We found that congruent derived words elicited a stronger MMN than incongruent derived words, beginning about 150 ms after perception of the critical morpheme. This pattern of results is consistent with whole-form storage of morphologically complex derived words as lexical units, or mini-constructions. Using distributed source localization methods, the MMN enhancement for well-formed derivationally complex words appeared to be most prominent in the left inferior anterior-temporal, bilateral superior parietal and bilateral post-central, supra-marginal areas. In addition, neurophysiological results reflected the frequency of derived forms, thus providing further converging evidence for whole form storage and against a combinatorial mechanism.