Localization of the syntactic mismatch negativity in the temporal cortex: An MEG study

Morphosyntactic prediction in automatic neural processing of spoken language: EEG evidence

Automatic parsing of syntactic information by the human brain is a well-established phenomenon, but its mechanisms remain poorly understood. Its best-known neurophysiological reflection is the so-called early left-anterior negativity (ELAN) component of event-related potentials (ERPs), with two alternative hypotheses for its origin: (1) error detection, or (2) morphosyntactic prediction/priming. To test these alternatives, we conducted two experiments using a non-attend passive design with visual distraction and recorded ERPs to spoken pronoun-verb phrases with/without agreement violations and to the same critical verbs presented in isolation without preceding pronouns. The results revealed an ELAN at ∼130-220 ms for pronoun-verb gender agreement violations, confirming a high degree of automaticity in early morphosyntactic parsing. Critically, the strongest ELAN was elicited by verbs outside phrasal context, which suggests that the typical ELAN pattern is underpinned by a reduction of ERP amplitudes for felicitous combinations, reflecting syntactic priming/predictability between related words/morphemes (potentially mediated by associative links formed during previous linguistic experience) rather than specialised error-detection processes.

Chunk boundaries disrupt dependency processing in an AG: Reconciling incremental processing and discrete sampling

Language is rooted in our ability to compose: We link words together, fusing their meanings. Links are not limited to neighboring words but often span intervening words. The ability to process these non-adjacent dependencies (NADs) conflicts with the brain's sampling of speech: We consume speech in chunks that are limited in time, containing only a limited number of words. It is unknown how we link words together that belong to separate chunks. Here, we report that we cannot-at least not so well. In our electroencephalography (EEG) study, 37 human listeners learned chunks and dependencies from an artificial grammar (AG) composed of syllables. Multi-syllable chunks to be learned were equal-sized, allowing us to employ a frequency-tagging approach. On top of chunks, syllable streams contained NADs that were either confined to a single chunk or crossed a chunk boundary. Frequency analyses of the EEG revealed a spectral peak at the chunk rate, showing that participants learned the chunks. NADs that cross boundaries were associated with smaller electrophysiological responses than within-chunk NADs. This shows that NADs are processed readily when they are confined to the same chunk, but not as well when crossing a chunk boundary. Our findings help to reconcile the classical notion that language is processed incrementally with recent evidence for discrete perceptual sampling of speech. This has implications for language acquisition and processing as well as for the general view of syntax in human language.

The Self-reference Effect Can Modulate Language Syntactic Processing Even Without Explicit Awareness: An Electroencephalography Study

Although it is well established that self-related information can rapidly capture our attention and bias cognitive functioning, whether this self-bias can affect language processing remains largely unknown. In addition, there is an ongoing debate as to the functional independence of language processes, notably regarding the syntactic domain. Hence, this study investigated the influence of self-related content on syntactic speech processing. Participants listened to sentences that could contain morphosyntactic anomalies while the masked face identity (self, friend, or unknown faces) was presented for 16 msec preceding the critical word. The language-related ERP components (left anterior negativity [LAN] and P600) appeared for all identity conditions. However, the largest LAN effect followed by a reduced P600 effect was observed for self-faces, whereas a larger LAN with no reduction of the P600 was found for friend faces compared with unknown faces. These data suggest that both early and late syntactic processes can be modulated by self-related content. In addition, alpha power was more suppressed over the left inferior frontal gyrus only when self-faces appeared before the critical word. This may reflect higher semantic demands concomitant to early syntactic operations (around 150-550 msec). Our data also provide further evidence of self-specific response, as reflected by the N250 component. Collectively, our results suggest that identity-related information is rapidly decoded from facial stimuli and may impact core linguistic processes, supporting an interactive view of syntactic processing. This study provides evidence that the self-reference effect can be extended to syntactic processing.

CompreTAP: Feasibility and reliability of a new language comprehension mapping task via preoperative navigated transcranial magnetic stimulation

OBJECTIVE

Stimulation-based language mapping approaches that are used pre- and intraoperatively employ predominantly overt language tasks requiring sufficient language production abilities. Yet, these production-based setups are often not feasible in brain tumor patients with severe expressive aphasia. This pilot study evaluated the feasibility and reliability of a newly developed language comprehension task with preoperative navigated transcranial magnetic stimulation (nTMS).

METHODS

Fifteen healthy subjects and six brain tumor patients with severe expressive aphasia unable to perform classic overt naming tasks underwent preoperative nTMS language mapping based on an auditory single-word Comprehension TAsk for Perioperative mapping (CompreTAP). Comprehension was probed by button-press responses to auditory stimuli, hence not requiring overt language responses. Positive comprehension areas were identified when stimulation elicited an incorrect or delayed button press. Error categories, case-wise cortical error rate distribution and inter-rater reliability between two experienced specialists were examined.

RESULTS

Overall, the new setup showed to be feasible. Comprehension-disruptions induced by nTMS manifested in no responses, delayed or hesitant responses, searching behavior or selection of wrong target items across all patients and controls and could be performed even in patients with severe expressive aphasia. The analysis agreement between both specialists was substantial for classifying comprehension-positive and -negative sites. Extensive left-hemispheric individual cortical comprehension sites were identified for all patients. Apart from one case presenting with transient worsening of aphasic symptoms, pre-existing language deficits did not aggravate if results were used for subsequent surgical planning.

CONCLUSION

Employing this new comprehension-based nTMS setup allowed to identify language relevant cortical sites in all healthy subjects and severely aphasic patients who were thus far precluded from classic production-based mapping. This pilot study, moreover, provides first indications that the CompreTAP mapping results may support the preservation of residual language function if used for subsequent surgical planning.

Online neurostimulation of Broca's area does not interfere with syntactic predictions: A combined TMS-EEG approach to basic linguistic combination

Categorical predictions have been proposed as the key mechanism supporting the fast pace of syntactic composition in language. Accordingly, grammar-based expectations are formed-e.g., the determiner "a" triggers the prediction for a noun-and facilitate the analysis of incoming syntactic information, which is then checked against a single or few other word categories. Previous functional neuroimaging studies point towards Broca's area in the left inferior frontal gyrus (IFG) as one fundamental cortical region involved in categorical prediction during incremental language processing. Causal evidence for this hypothesis is however still missing. In this study, we combined Electroencephalography (EEG) and Transcranial Magnetic Stimulation (TMS) to test whether Broca's area is functionally relevant in predictive mechanisms for language. We transiently perturbed Broca's area during the first word in a two-word construction, while simultaneously measuring the Event-Related Potential (ERP) correlates of syntactic composition. We reasoned that if Broca's area is involved in predictive mechanisms for syntax, disruptive TMS during the first word would mitigate the difference in the ERP responses for predicted and unpredicted categories in basic two-word constructions. Contrary to this hypothesis, perturbation of Broca's area at the predictive stage did not affect the ERP correlates of basic composition. The correlation strength between the electrical field induced by TMS and the ERP responses further confirmed this pattern. We discuss the present results considering an alternative account of the role of Broca's area in syntactic composition, namely the bottom-up integration of words into constituents, and of compensatory mechanisms within the language predictive network.

Native language experience shapes pre-attentive foreign tone processing and guides rapid memory trace build-up: An ERP study

Language experience, particularly from our native language (L1), shapes our perception of other languages around us. The present study examined how L1 experience moulds the initial processing of foreign (L2) tone during acquisition. In particular, we investigated whether learners were able to rapidly forge new neural memory traces for novel tonal words, which was tracked by recording learners’ ERP responses during two word acquisition sessions. We manipulated the degree of L1–L2 familiarity by comparing learners with a nontonal L1 (German) and a tonal L1 (Swedish) and by using tones that were similar (fall) or dissimilar (high, low, rise) to those occurring in Swedish. Our results indicate that a rapid, pre‐attentive memory trace build‐up for tone manifests in an early ERP component at ~50 ms but only at particularly high levels of L1–L2 similarity. Specifically, early processing was facilitated for an L2 tone that had a familiar pitch shape (fall) and word‐level function (inflection). This underlines the importance of these L1 properties for the early processing of L2 tone. In comparison, a later anterior negativity related to the processing of the tones’ grammatical content was unaffected by native language experience but was instead influenced by lexicality, pitch prominence, entrenchment, and successful learning. Behaviorally, learning effects emerged for all learners and tone types, regardless of L1–L2 familiarity or pitch prominence. Together, the findings suggest that while L1‐based facilitation effects occur, they mainly affect early processing stages and do not necessarily result in more successful L2 acquisition at behavioral level.

Our findings add important evidence that contributes to answering the open question of how similarity between native and target language influences target language processing and acquisition. We found facilitative effects of similarity only at pre‐attentive levels and only when the degree of similarity was high. Late processing and successful acquisition, on the other hand, were unaffected by the target words’ similarity to native language properties.

Functional connectivity of spoken language processing in early-stage Parkinson's disease: An MEG study

Parkinson's disease (PD) is a neurodegenerative disorder, well-known for its motor symptoms; however, it also adversely affects cognitive functions, including language, a highly important human ability. PD pathology is associated, even in the early stage of the disease, with alterations in the functional connectivity within cortico-subcortical circuitry of the basal ganglia as well as within cortical networks. Here, we investigated functional cortical connectivity related to spoken language processing in early-stage PD patients. We employed a patient-friendly passive attention-free paradigm to probe neurophysiological correlates of language processing in PD patients without confounds related to active attention and overt motor responses. MEG data were recorded from a group of newly diagnosed PD patients and age-matched healthy controls who were passively presented with spoken word stimuli (action and abstract verbs, as well as grammatically correct and incorrect inflectional forms) while focussing on watching a silent movie. For each of the examined linguistic aspects, a logistic regression classifier was used to classify participants as either PD patients or healthy controls based on functional connectivity within the temporo-fronto-parietal cortical language networks. Classification was successful for action verbs (accuracy = 0.781, p-value = 0.003) and, with lower accuracy, for abstract verbs (accuracy = 0.688, p-value = 0.041) and incorrectly inflected forms (accuracy = 0.648, p-value = 0.021), but not for correctly inflected forms (accuracy = 0.523, p-value = 0.384). Our findings point to quantifiable differences in functional connectivity within the cortical systems underpinning language processing in newly diagnosed PD patients compared to healthy controls, which arise early, in the absence of clinical evidence of deficits in cognitive or general language functions. The techniques presented here may aid future work on establishing neurolinguistic markers to objectively and noninvasively identify functional changes in the brain's language networks even before clinical symptoms emerge.

Hierarchical syntactic processing is beyond mere associating: Functional magnetic resonance imaging evidence from a novel artificial grammar

Grammar is central to any natural language. In the past decades, the artificial grammar of the AⁿBⁿ type in which a pair of associated elements can be nested in the other pair was considered as a desirable model to mimic human language syntax without semantic interference. However, such a grammar relies on mere associating mechanisms, thus insufficient to reflect the hierarchical nature of human syntax. Here, we test how the brain imposes syntactic hierarchies according to the category relations on linearized sequences by designing a novel artificial “Hierarchical syntactic structure‐building Grammar” (HG), and compare this to the AⁿBⁿ grammar as a “Nested associating Grammar” (NG) based on multilevel associations. Thirty‐six healthy German native speakers were randomly assigned to one of the two grammars. Both groups performed a grammaticality judgment task on auditorily presented word sequences generated by the corresponding grammar in the scanner after a successful explicit behavioral learning session. Compared to the NG group, we found that the HG group showed a (a) significantly higher involvement of Brodmann area (BA) 44 in Broca's area and the posterior superior temporal gyrus (pSTG); and (b) qualitatively distinct connectivity between the two regions. Thus, the present study demonstrates that the build‐up process of syntactic hierarchies on the basis of category relations critically relies on a distinctive left‐hemispheric syntactic network involving BA 44 and pSTG. This indicates that our novel artificial grammar can constitute a suitable experimental tool to investigate syntax‐specific processes in the human brain.

Language and action in Broca's area: Computational differentiation and cortical segregation

Actions have been proposed to follow hierarchical principles similar to those hypothesized for language syntax. These structural similarities are claimed to be reflected in the common involvement of certain neural populations of Broca's area, in the Inferior Frontal Gyrus (IFG). In this position paper, we follow an influential hypothesis in linguistic theory to introduce the syntactic operation Merge and the corresponding motor/conceptual interfaces. We argue that actions hierarchies do not follow the same principles ruling language syntax. We propose that hierarchy in the action domain lies in predictive processing mechanisms mapping sensory inputs and statistical regularities of action-goal relationships. At the cortical level, distinct Broca's subregions appear to support different types of computations across the two domains. We argue that anterior BA44 is a major hub for the implementation of the syntactic operation Merge. On the other hand, posterior BA44 is recruited in selecting premotor mental representations based on the information provided by contextual signals. This functional distinction is corroborated by a recent meta-analysis (Papitto, Friederici, & Zaccarella, 2020). We conclude by suggesting that action and language can meet only where the interfaces transfer abstract computations either to the external world or to the internal mental world.

Meaning before grammar: A review of ERP experiments on the neurodevelopmental origins of semantic processing

According to traditional linguistic theories, the construction of complex meanings relies firmly on syntactic structure-building operations. Recently, however, new models have been proposed in which semantics is viewed as being partly autonomous from syntax. In this paper, we discuss some of the developmental implications of syntax-based and autonomous models of semantics. We review event-related brain potential (ERP) studies on semantic processing in infants and toddlers, focusing on experiments reporting modulations of N400 amplitudes using visual or auditory stimuli and different temporal structures of trials. Our review suggests that infants can relate or integrate semantic information from temporally overlapping stimuli across modalities by 6 months of age. The ability to relate or integrate semantic information over time, within and across modalities, emerges by 9 months. The capacity to relate or integrate information from spoken words in sequences and sentences appears by 18 months. We also review behavioral and ERP studies showing that grammatical and syntactic processing skills develop only later, between 18 and 32 months. These results provide preliminary evidence for the availability of some semantic processes prior to the full developmental emergence of syntax: non-syntactic meaning-building operations are available to infants, albeit in restricted ways, months before the abstract machinery of grammar is in place. We discuss this hypothesis in light of research on early language acquisition and human brain development.

Head errors of syntactic dependency increase neuromagnetic mismatch intensities

Mismatch-related brain activation in healthy individuals is an important area of neural investigation. Previously, we evaluated sentence-level syntactic dependencies, composed of a head and a dependent between two syntactically related words in head-initial English structures. We demonstrated that prominent mismatch effects were induced by within-category dependent errors when semantic interpretation was preserved. However, the following issues were not addressed: (1) whether head errors of syntactic dependency in head-final structures would elicit large mismatch field (MMF) intensities, and (2) whether an MMF effect of syntactic errors would be seen in the left superior temporal cortex alone. In this study, auditory MMFs were obtained by magnetocephalography (MEG) from healthy Japanese adults (n = 8) who were subjected to a passive auditory oddball paradigm with syntactically legal or illegal utterances and single words in Japanese. The results demonstrate that the source waveforms had significantly higher MMF cortical activation in response to the head error, which involved altered polarity of the predicate. This resulted in a syntactically incorrect and semantically incomprehensible expression, when compared to the syntactically correct expression and the non-structural lexical item. This mismatch effect, with a peak latency of 164 ms, was confined to the anterior region of the left superior temporal cortex. The current results clearly indicate that the representation of syntactic dependency is stored in long-term memory and tends to be activated in automatic auditory processing.

Magnetoencephalography and Language

This article provides an overview of research that uses magnetoencephalography to understand the brain basis of human language. The cognitive processes and brain networks that have been implicated in written and spoken language comprehension and production are discussed in relation to different methodologies: we review event-related brain responses, research on the coupling of neural oscillations to speech, oscillatory coupling between brain regions (eg, auditory-motor coupling), and neural decoding approaches in naturalistic language comprehension.

Tracking Training-Related Plasticity by Combining fMRI and DTI: The Right Hemisphere Ventral Stream Mediates Musical Syntax Processing

As a functional homolog for left-hemispheric syntax processing in language, neuroimaging studies evidenced involvement of right prefrontal regions in musical syntax processing, of which underlying white matter connectivity remains unexplored so far. In the current experiment, we investigated the underlying pathway architecture in subjects with 3 levels of musical expertise. Employing diffusion tensor imaging tractography, departing from seeds from our previous functional magnetic resonance imaging study on music syntax processing in the same participants, we identified a pathway in the right ventral stream that connects the middle temporal lobe with the inferior frontal cortex via the extreme capsule, and corresponds to the left hemisphere ventral stream, classically attributed to syntax processing in language comprehension. Additional morphometric consistency analyses allowed dissociating tract core from more dispersed fiber portions. Musical expertise related to higher tract consistency of the right ventral stream pathway. Specifically, tract consistency in this pathway predicted the sensitivity for musical syntax violations. We conclude that enduring musical practice sculpts ventral stream architecture. Our results suggest that training-related pathway plasticity facilitates the right hemisphere ventral stream information transfer, supporting an improved sound-to-meaning mapping in music.

Neural substrates of word category information as the basis of syntactic processing

The ability to use word category information (WCI) for syntactic structure building has been hypothesized to be the essence of human language faculty. The neural substrate of the ability of using the WCI for the complex syntactic hierarchical structure processing, however, is yet unknown. Therefore, we directly conducted an fMRI experiment by using a pseudo-Chinese artificial language with syntactic structures containing a center-embedded relative clause. Thirty non-Chinese native (Korean) speakers were randomly divided into two groups: one acquired WCI and WCI-based syntactic rules (the WCI group) before the scanning session, and the other did not (the non-WCI group). Both groups were required to judge the grammaticality of the testing sentences, with critical long-distance dependencies between two elements (the main verb and the relativizer). Behaviorally, the WCI group's accuracy was significantly higher and its reaction time was shorter. The scanning results showed that the left superior temporal gyrus (STG) and Broca's area were more strongly activated for the WCI group, and the dynamic causal modeling analyses revealed a distinct effective connectivity pattern for this group. Therefore, the present research, for the first time, reveals that the activation of and the functional connectivity between Broca's area and the left STG play a critical role in the ability of the rule-based use of the WCI which is crucial for complex hierarchical structure building, and might be substantially corresponding to the "labeling competence" within the linguistic framework.

Synchronization of Electrophysiological Responses with Speech Benefits Syntactic Information Processing

In auditory neuroscience, electrophysiological synchronization to low-level acoustic and high-level linguistic features is well established—but its functional purpose for verbal information transmission is unclear. Based on prior evidence for a dependence of auditory task performance on delta-band oscillatory phase, we hypothesized that the synchronization of electrophysiological responses at delta-band frequency to the speech stimulus serves to implicitly align neural excitability with syntactic information. The experimental paradigm of our auditory EEG study uniformly distributed morphosyntactic violations across syntactic phrases of natural sentences, such that violations would occur at points differing in linguistic information content. In support of our hypothesis, we found behavioral responses to morphosyntactic violations to increase with decreasing syntactic information content—in significant correlation with delta-band phase, which had synchronized to our speech stimuli. Our findings indicate that rhythmic electrophysiological synchronization to the speech stream is a functional mechanism that may align neural excitability with linguistic information content, optimizing language comprehension.

Automatic processing of pragmatic information in the human brain: a mismatch negativity study

Language comprehension involves pragmatic information processing, which allows world knowledge to influence the interpretation of a sentence. This study explored whether pragmatic information can be automatically processed during spoken sentence comprehension. The experiment adopted the mismatch negativity (MMN) paradigm to capture the neurophysiological indicators of automatic processing of spoken sentences. Pragmatically incorrect ('Foxes have wings') and correct ('Butterflies have wings') sentences were used as the experimental stimuli. In condition 1, the pragmatically correct sentence was the deviant and the pragmatically incorrect sentence was the standard stimulus, whereas the opposite case was presented in condition 2. The experimental results showed that, compared with the condition that the pragmatically correct sentence is the deviant stimulus, when the condition that the pragmatically incorrect sentence is the deviant stimulus MMN effects were induced within 60-120 and 220-260 ms. The results indicated that the human brain can monitor for incorrect pragmatic information in the inattentive state and can automatically process pragmatic information at the beginning of spoken sentence comprehension.

The development of linguistic prediction: Predictions of sound and meaning in 2- to 5-year-olds

Language processing in adults is facilitated by an expert ability to generate detailed predictions about upcoming words. This may seem like an acquired skill, but some models of language acquisition assume that the ability to predict is a prerequisite for learning. This raises a question: Do children learn to predict, or do they predict to learn? We tested whether children, like adults, can generate expectations about not just the meanings of upcoming words but also their sounds, which would be critical for using prediction to learn about language. In two looking-while-listening experiments, we show that 2-year-olds can generate expectations about meaning based on a determiner (Can you see one…ball/two…ice creams?) but that even children as old as 5 years do not show an adult-like ability to predict the phonology of upcoming words based on a determiner (Can you see a…ball/an…ice cream?). Our results, therefore, suggest that the ability to generate detailed predictions is a late-acquired skill. We argue that prediction might not be the key mechanism driving children's learning, but that the ability to generate accurate semantic predictions may nevertheless have facilitative effects on language development.

Neuromagnetic correlates of audiovisual word processing in the developing brain

The brain undergoes enormous changes during childhood. Little is known about how the brain develops to serve word processing. The objective of the present study was to investigate the maturational changes of word processing in children and adolescents using magnetoencephalography (MEG). Responses to a word processing task were investigated in sixty healthy participants. Each participant was presented with simultaneous visual and auditory word pairs in "match" and "mismatch" conditions. The patterns of neuromagnetic activation from MEG recordings were analyzed at both sensor and source levels. Topography and source imaging revealed that word processing transitioned from bilateral connections to unilateral connections as age increased from 6 to 17 years old. Correlation analyses of language networks revealed that the path length of word processing networks negatively correlated with age (r = -0.833, p < 0.0001), while the connection strength (r = 0.541, p < 0.01) and the clustering coefficient (r = 0.705, p < 0.001) of word processing networks were positively correlated with age. In addition, males had more visual connections, whereas females had more auditory connections. The correlations between gender and path length, gender and connection strength, and gender and clustering coefficient demonstrated a developmental trend without reaching statistical significance. The results indicate that the developmental trajectory of word processing is gender specific. Since the neuromagnetic signatures of these gender-specific paths to adult word processing were determined using non-invasive, objective, and quantitative methods, the results may play a key role in understanding language impairments in pediatric patients in the future.

Magnetoencephalography Reveals Mismatch Field Enhancement from Unexpected Syntactic Category Errors in English Sentences

The type of syntactic operations that increase neuronal activation in humans as a result of syntactically erroneous, unexpected lexical items in hearing sentences has remained unclear. In the present study, we used recordings of magnetoencephalographic (MEG) activity to compare bare infinitive and full infinitive constructions in English. This research aims to identify the type of syntactic deviance that may trigger an early syntax-related mismatch field (MMF) component when unexpected words appear in sentences. Six speakers of English as a first language were presented with auditory stimuli of sentences or words in a passive odd-ball paradigm while watching a silent movie. The experimental protocol included four sessions, specifically investigating the sentential (structural) versions of full (with the 'to' infinitival particle) and bare infinitival structures (without the particle) and the lexical (non-structure) versions of the verb either with or without the particle to determine whether the structure processing of sentences was a more crucial factor in the detection of the MMF than the simple processing of lexical items in verb-only conditions. The amplitude analysis of the resulting evoked fields showed that the presence of the syntactic category error of bare infinitival structures against syntactic predictions evoked a significantly larger MMF activation with a peak latency of approximately 200ms in the anterior superior temporal sulci in the left hemisphere, compared with the lexical items that did not have any syntactic status. These results clearly demonstrate that syntactically unexpected, illegal input in the bare infinitival structure is likely to be noticed more robustly in the brain while processing the structural information of the entire sentence than the corresponding verb-only items.

Spread the word: MMN brain response reveals whole-form access of discontinuous particle verbs

The status of particle verbs such as rise (…) up as either lexically stored or combinatorially assembled is an issue which so far has not been settled decisively. In this study, we use the mismatch negativity (MMN) brain response to observe neurophysiological responses to discontinuous particle verbs. The MMN can be used to distinguish between whole-form storage and combinatorial processes, as it is enhanced to stored words compared to unknown pseudowords, whereas combinatorially legal strings elicit a reduced MMN relative to ungrammatical ones. Earlier work had found larger MMNs to congruent than to incongruent verb-particle combinations when particle and verb appeared as adjacent elements, thus suggesting whole-form storage at least in this case. However, it is still possible that particle verbs discontinuously spread out across a sentence would elicit the combinatorial, grammar-violation response pattern instead. Here, we tested the brain signatures of discontinuous verb-particle combinations, orthogonally varying congruence and semantic transparency. The results show for the first time brain indices of whole-form storage for discontinuous constituents, thus arguing in favour of access to whole-form-stored lexical elements in the processing of particle verbs, irrespective of their semantic opacity. Results are discussed in the context of linguistic debates about the status of particle verbs as words, lexical elements or syntactically generated combinations. The explanation of the pattern of results within a neurobiological language model is highlighted.

Electrophysiological Evidence for Early and Interactive Symbol Access and Rule Processing in Retrieving and Combining Language Constructions

The human brain stores an immense repertoire of linguistic symbols (morphemes, words) and combines them into a virtually unlimited set of well-formed strings (phrases, sentences) that serve as efficient communicative tools. Communication is hampered, however, if strings include meaningless items (e.g., "pseudomorphemes"), or if the rules for combining string elements are violated. Prior research suggests that, when participants attentively process sentences in a linguistic task, syntactic processing can occur quite early, but lexicosemantic processing, or any interaction involving this factor, is manifest later in time (ca. 400 msec or later). In contrast, recent evidence from passive speech perception paradigms suggests early processing of both combinatorial (morphosyntactic) and storage-related (lexicosemantic) properties. A crucial question is whether these parallel processes might also interact early in processing. Using ERPs in an orthogonal design, we presented spoken word strings to participants while they were distracted from incoming speech to obtain information about automatic language processing mechanisms unaffected by task-related strategies. Stimuli were either (1) well-formed miniconstructions (short pronoun-verb sentences), (2) "unstored" strings containing a pseudomorpheme, (3) "ill-combined" strings violating subject-verb agreement rules, or (4) double violations including both types of errors. We found that by 70-210 msec after the onset of the phrase-final syllable that disambiguated the strings, interactions of lexicosemantic and morphosyntactic deviance were evident in the ERPs. These results argue against serial processing of lexical storage, morphosyntactic combination and their interaction, and in favor of early, simultaneous, and interactive processing of symbols and their combinatorial structures.

Acquisition and consolidation of novel morphology in human neocortex: A neuromagnetic study

Research into neurobiological mechanisms of morphosyntactic processing of language has suggested specialised systems for decomposition and storage, which are used flexibly during the processing of complex polymorphemic words (such as those formed through affixation, e.g., boy + s = noun + plural marker or boy + ish = noun plus attenuator). However, neural underpinnings of acquisition of novel morphology are still unknown. We implicitly trained our participants with new derivational affixes through a word-picture association task and investigated the neural processes underlying formation of neural memory traces for new affixes. The participants' brain activity was recorded using magnetoencephalography (MEG), as they passively listened to the newly trained and untrained suffixes combined with real word and pseudoword stems. The MEG recording was repeated after a night's sleep using the same stimuli, to test the effects of overnight consolidation. The newly trained suffixes combined with real stems elicited stronger source activity in the left inferior frontal gyrus (LIFG) at ∼50 msec after the suffix onset than untrained suffixes, suggesting memory trace formation for the newly learned suffixes already on the same day. The following day, the suffix learning effect spread to the left superior temporal gyrus (STG) where it was again manifest as a response enhancement, particularly at ∼200-300 msec after the suffix onset, which might reflect an additional effect of overnight consolidation. Overall, the results demonstrate the rapid and dynamic processes of both immediate build-up and longer-term consolidation of neocortical memory traces for novel morphology, taking place after a short period of exposure to novel morphology and involving fronto-temporal perisylvian language circuitry.

Early neurophysiological indices of second language morphosyntax learning

Humans show variable degrees of success in acquiring a second language (L2). In many cases, morphological and syntactic knowledge remain deficient, although some learners succeed in reaching nativelike levels, even if they begin acquiring their L2 relatively late. In this study, we use psycholinguistic, online language proficiency tests and a neurophysiological index of syntactic processing, the syntactic mismatch negativity (sMMN) to local agreement violations, to compare behavioural and neurophysiological markers of grammar processing between native speakers (NS) of English and non-native speakers (NNS). Variable grammar proficiency was measured by psycholinguistic tests. When NS heard ungrammatical word sequences lacking agreement between subject and verb (e.g. *we kicks), the MMN was enhanced compared with syntactically legal sentences (e.g. he kicks). More proficient NNS also showed this difference, but less proficient NNS did not. The main cortical sources of the MMN responses were localised in bilateral superior temporal areas, where, crucially, source strength of grammar-related neuronal activity correlated significantly with grammatical proficiency of individual L2 speakers as revealed by the psycholinguistic tests. As our results show similar, early MMN indices to morpho-syntactic agreement violations among both native speakers and non-native speakers with high grammar proficiency, they appear consistent with the use of similar brain mechanisms for at least certain aspects of L1 and L2 grammars.

Are scalar implicatures automatically processed and different for each individual? A mismatch negativity (MMN) study

Scalar implicatures are ordinarily activated in human communication when the speaker uses a weak expression (e.g., some) from a set of stronger alternatives (e.g., many, all). It has been debated whether scalar inferences are generated by default. To clarify this issue and examine whether individual pragmatic ability will affect the mechanism of scalar inference processing, we performed experiment with an MMN paradigm to capture the neurophysiological indicators of automatic processing of spoken sentences and divided participants into high and low pragmatic ability groups. Experimental results showed that compared with the condition that an informative sentence ("Some animals have tails") is the deviant stimuli, when an underinformative sentence ("Some tigers have tails") is the deviant stimuli, the high pragmatic ability group induced mismatch negativity (MMN) and sustained negativity, while the low pragmatic ability group had no ERP effects. These results indicated that at least some people can automatically activate the scalar implicatures when encountering scalar trigger words, even in the inattentive status.

Discrimination of fearful and angry emotional voices in sleeping human neonates: a study of the mismatch brain responses

Appropriate processing of human voices with different threat-related emotions is of evolutionarily adaptive value for the survival of individuals. Nevertheless, it is still not clear whether the sensitivity to threat-related information is present at birth. Using an odd-ball paradigm, the current study investigated the neural correlates underlying automatic processing of emotional voices of fear and anger in sleeping neonates. Event-related potential data showed that the fronto-central scalp distribution of the neonatal brain could discriminate fearful voices from angry voices; the mismatch response (MMR) was larger in response to the deviant stimuli of anger, compared with the standard stimuli of fear. Furthermore, this fear–anger MMR discrimination was observed only when neonates were in active sleep state. Although the neonates' sensitivity to threat-related voices is not likely associated with a conceptual understanding of fearful and angry emotions, this special discrimination in early life may provide a foundation for later emotion and social cognition development.

Neurophysiological evidence for whole form retrieval of complex derived words: a mismatch negativity study

Complex words can be seen as combinations of elementary units, decomposable into stems and affixes according to morphological rules. Alternatively, complex forms may be stored as single lexical entries and accessed as whole forms. This study uses an event-related potential brain response capable of indexing both whole-form retrieval and combinatorial processing, the Mismatch Negativity (MMN), to investigate early brain activity elicited by morphologically complex derived words in German. We presented complex words consisting of stems “sicher” (secure), or “sauber” (clean) combined with abstract nominalizing derivational affixes -heit or -keit, to form either congruent derived words: “Sicherheit” (security) and “Sauberkeit” (cleanliness), or incongruent derived pseudowords: *“Sicherkeit”, and *“Sauberheit”. Using this orthogonal design, it was possible to record brain responses for -heit and -keit in both congruent and incongruent contexts, therefore balancing acoustic variance. Previous research has shown that incongruent combinations of symbols elicit a stronger MMN than congruent combinations, but that single words or constructions stored as whole forms elicit a stronger MMN than pseudowords or non-existent constructions. We found that congruent derived words elicited a stronger MMN than incongruent derived words, beginning about 150 ms after perception of the critical morpheme. This pattern of results is consistent with whole-form storage of morphologically complex derived words as lexical units, or mini-constructions. Using distributed source localization methods, the MMN enhancement for well-formed derivationally complex words appeared to be most prominent in the left inferior anterior-temporal, bilateral superior parietal and bilateral post-central, supra-marginal areas. In addition, neurophysiological results reflected the frequency of derived forms, thus providing further converging evidence for whole form storage and against a combinatorial mechanism.

Multi-voxel pattern analysis of noun and verb differences in ventral temporal cortex

Recent evidence suggests a probabilistic relationship exists between the phonological/orthographic form of a word and its lexical-syntactic category (specifically nouns vs. verbs) such that syntactic prediction may elicit form-based estimates in sensory cortex. We tested this hypothesis by conducting multi-voxel pattern analysis (MVPA) of fMRI data from early visual cortex (EVC), left ventral temporal (VT) cortex, and a subregion of the latter - the left mid fusiform gyrus (mid FG), sometimes called the "visual word form area." Crucially, we examined only those volumes sampled when subjects were predicting, but not viewing, nouns and verbs. This allowed us to investigate prediction effects in visual areas without any bottom-up orthographic input. We found that voxels in VT and mid FG, but not in EVC, were able to classify noun-predictive trials vs. verb-predictive trials in sentence contexts, suggesting that sentence-level predictions are sufficient to generate word form-based estimates in visual areas.

Prediction in the service of comprehension: modulated early brain responses to omitted speech segments

Speech signals are often compromised by disruptions originating from external (e.g., masking noise) or internal (e.g., inaccurate articulation) sources. Speech comprehension thus entails detecting and replacing missing information based on predictive and restorative neural mechanisms. The present study targets predictive mechanisms by investigating the influence of a speech segment's predictability on early, modality-specific electrophysiological responses to this segment's omission. Predictability was manipulated in simple physical terms in a single-word framework (Experiment 1) or in more complex semantic terms in a sentence framework (Experiment 2). In both experiments, final consonants of the German words Lachs ([laks], salmon) or Latz ([lats], bib) were occasionally omitted, resulting in the syllable La ([la], no semantic meaning), while brain responses were measured with multi-channel electroencephalography (EEG). In both experiments, the occasional presentation of the fragment La elicited a larger omission response when the final speech segment had been predictable. The omission response occurred ∼125-165 msec after the expected onset of the final segment and showed characteristics of the omission mismatch negativity (MMN), with generators in auditory cortical areas. Suggestive of a general auditory predictive mechanism at work, this main observation was robust against varying source of predictive information or attentional allocation, differing between the two experiments. Source localization further suggested the omission response enhancement by predictability to emerge from left superior temporal gyrus and left angular gyrus in both experiments, with additional experiment-specific contributions. These results are consistent with the existence of predictive coding mechanisms in the central auditory system, and suggestive of the general predictive properties of the auditory system to support spoken word recognition.

Generating predictions: lesion evidence on the role of left inferior frontal cortex in rapid syntactic analysis

A well-documented phenomenon in event-related electroencephalography (EEG) and magnetoencephalography (MEG) studies on language processing is that syntactic violations of different types elicit negativities as early as 100 msec after the violation point. Recently, these responses have been associated with activations in or very close to sensory cortices, suggesting the involvement of basic sensory mechanisms in the detection of syntactic violations. The present study investigated whether intact auditory cortices and adjacent temporal regions are sufficient to generate early syntactic negativities in the auditory event-related potential (ERP). We tested ten clinically non-aphasic patients with left inferior frontal lesions, but intact temporal cortices in a passive auditory ERP paradigm that had reliably elicited early negativities in response to violations of subject-verb agreement and word category in the past. Subject-verb agreement violations failed to elicit early grammaticality effects in these patients, whereas a group of ten age-matched controls showed a reliable early negativity. This finding supports the idea that sensory aspects of syntactic analysis as reflected in early syntactic negativities critically depend on top-down predictions generated by the left inferior frontal cortex. In contrast, word category violations elicited a small, marginally significant early negativity both in controls and patients, suggesting an additional involvement of temporal regions in early phrase structure processing. In an additional auditory oddball experiment patients showed a regular P300, but no N2b component in response to deviant tones, indicating that their deficit in generating sensory predictions extends beyond the language domain.

Mapping cortical dynamics using simultaneous MEG/EEG and anatomically-constrained minimum-norm estimates: an auditory attention example

Magneto- and electroencephalography (MEG/EEG) are neuroimaging techniques that provide a high temporal resolution particularly suitable to investigate the cortical networks involved in dynamical perceptual and cognitive tasks, such as attending to different sounds in a cocktail party. Many past studies have employed data recorded at the sensor level only, i.e., the magnetic fields or the electric potentials recorded outside and on the scalp, and have usually focused on activity that is time-locked to the stimulus presentation. This type of event-related field / potential analysis is particularly useful when there are only a small number of distinct dipolar patterns that can be isolated and identified in space and time. Alternatively, by utilizing anatomical information, these distinct field patterns can be localized as current sources on the cortex. However, for a more sustained response that may not be time-locked to a specific stimulus (e.g., in preparation for listening to one of the two simultaneously presented spoken digits based on the cued auditory feature) or may be distributed across multiple spatial locations unknown a priori, the recruitment of a distributed cortical network may not be adequately captured by using a limited number of focal sources. Here, we describe a procedure that employs individual anatomical MRI data to establish a relationship between the sensor information and the dipole activation on the cortex through the use of minimum-norm estimates (MNE). This inverse imaging approach provides us a tool for distributed source analysis. For illustrative purposes, we will describe all procedures using FreeSurfer and MNE software, both freely available. We will summarize the MRI sequences and analysis steps required to produce a forward model that enables us to relate the expected field pattern caused by the dipoles distributed on the cortex onto the M/EEG sensors. Next, we will step through the necessary processes that facilitate us in denoising the sensor data from environmental and physiological contaminants. We will then outline the procedure for combining and mapping MEG/EEG sensor data onto the cortical space, thereby producing a family of time-series of cortical dipole activation on the brain surface (or "brain movies") related to each experimental condition. Finally, we will highlight a few statistical techniques that enable us to make scientific inference across a subject population (i.e., perform group-level analysis) based on a common cortical coordinate space.

Co-localizing linguistic and musical syntax with intracranial EEG

Despite general agreement on shared syntactic resources in music and language, the neuroanatomical underpinnings of this overlap remain largely unexplored. While previous studies mainly considered frontal areas as supramodal grammar processors, the domain-general syntactic role of temporal areas has been so far neglected. Here we capitalized on the excellent spatial and temporal resolution of subdural EEG recordings to co-localize low-level syntactic processes in music and language in the temporal lobe in a within-subject design. We used Brain Surface Current Density mapping to localize and compare neural generators of the early negativities evoked by violations of phrase structure grammar in both music and spoken language. The results show that the processing of syntactic violations relies in both domains on bilateral temporo-fronto-parietal neural networks. We found considerable overlap of these networks in the superior temporal lobe, but also differences in the hemispheric timing and relative weighting of their fronto-temporal constituents. While alluding to the dissimilarity in how shared neural resources may be configured depending on the musical or linguistic nature of the perceived stimulus, the combined data lend support for a co-localization of early musical and linguistic syntax processing in the temporal lobe.

Voice and Emotion Processing in the Human Neonatal Brain

Although the voice-sensitive neural system emerges very early in development, it has yet to be demonstrated whether the neonatal brain is sensitive to voice perception. We measured the EEG mismatch response (MMR) elicited by emotionally spoken syllables “dada” along with correspondingly synthesized nonvocal sounds, whose fundamental frequency contours were matched, in 98 full-term newborns aged 1–5 days. In Experiment 1, happy syllables relative to nonvocal sounds elicited an MMR lateralized to the right hemisphere. In Experiment 2, fearful syllables elicited stronger amplitudes than happy or neutral syllables, and this response had no sex differences. In Experiment 3, angry versus happy syllables elicited an MMR, although their corresponding nonvocal sounds did not. Here, we show that affective discrimination is selectively driven by voice processing per se rather than low-level acoustical features and that the cerebral specialization for human voice and emotion processing emerges over the right hemisphere during the first days of life.

Dissociable neural imprints of perception and grammar in auditory functional imaging

In language processing, the relative contribution of early sensory and higher cognitive brain areas is still an open issue. A recent controversial hypothesis proposes that sensory cortices show sensitivity to syntactic processes, whereas other studies suggest a wider neural network outside sensory regions. The goal of the current event-related fMRI study is to clarify the contribution of sensory cortices in auditory syntactic processing in a 2 × 2 design. Two-word utterances were presented auditorily and varied both in perceptual markedness (presence or absence of an overt word category marking "-t"), and in grammaticality (syntactically correct or incorrect). A multivariate pattern classification approach was applied to the data, flanked by conventional cognitive subtraction analyses. The combination of methods and the 2 × 2 design revealed a clear picture: The cognitive subtraction analysis found initial syntactic processing signatures in a neural network including the left IFG, the left aSTG, the left superior temporal sulcus (STS), as well as the right STS/STG. Classification of local multivariate patterns indicated the left-hemispheric regions in IFG, aSTG, and STS to be more syntax-specific than the right-hemispheric regions. Importantly, auditory sensory cortices were only sensitive to the overt perceptual marking, but not to the grammaticality, speaking against syntax-inflicted sensory cortex modulations. Instead, our data provide clear evidence for a distinction between regions involved in pure perceptual processes and regions involved in initial syntactic processes.

The time-course and spatial distribution of brain activity associated with sentence processing

Sentence comprehension involves a host of highly interrelated processes, including syntactic parsing, semantic composition, and pragmatic inferencing. In neuroimaging, a primary paradigm for examining the brain bases of sentence processing has been to compare brain activity elicited by sentences versus unstructured lists of words. These studies commonly find an effect of increased activity for sentences in the anterior temporal lobes (aTL). Together with neuropsychological data, these findings have motivated the hypothesis that the aTL is engaged in sentence level combinatorics. Combinatoric processing during language comprehension, however, occurs within tens and hundreds of milliseconds, i.e., at a time-scale much faster than the temporal resolution of hemodynamic measures. Here, we examined the time-course of sentence-level processing using magnetoencephalography (MEG) to better understand the temporal profile of activation in this common paradigm and to test a key prediction of the combinatoric hypothesis: because sentences are interpreted incrementally, word-by-word, activity associated with basic linguistic combinatorics should be time-locked to word-presentation. Our results reveal increased anterior temporal activity for sentences compared to word lists beginning approximately 250 ms after word onset. We also observed increased activation in a network of other brain areas, extending across posterior temporal, inferior frontal, and ventral medial areas. These findings confirm a key prediction of the combinatoric hypothesis for the aTL and further elucidate the spatio-temporal characteristics of sentence-level computations in the brain.

The brain basis of language processing: from structure to function

Language processing is a trait of human species. The knowledge about its neurobiological basis has been increased considerably over the past decades. Different brain regions in the left and right hemisphere have been identified to support particular language functions. Networks involving the temporal cortex and the inferior frontal cortex with a clear left lateralization were shown to support syntactic processes, whereas less lateralized temporo-frontal networks subserve semantic processes. These networks have been substantiated both by functional as well as by structural connectivity data. Electrophysiological measures indicate that within these networks syntactic processes of local structure building precede the assignment of grammatical and semantic relations in a sentence. Suprasegmental prosodic information overtly available in the acoustic language input is processed predominantly in a temporo-frontal network in the right hemisphere associated with a clear electrophysiological marker. Studies with patients suffering from lesions in the corpus callosum reveal that the posterior portion of this structure plays a crucial role in the interaction of syntactic and prosodic information during language processing.

Before the N400: effects of lexical-semantic violations in visual cortex

There exists an increasing body of research demonstrating that language processing is aided by context-based predictions. Recent findings suggest that the brain generates estimates about the likely physical appearance of upcoming words based on syntactic predictions: words that do not physically look like the expected syntactic category show increased amplitudes in the visual M100 component, the first salient MEG response to visual stimulation. This research asks whether violations of predictions based on lexical-semantic information might similarly generate early visual effects. In a picture-noun matching task, we found early visual effects for words that did not accurately describe the preceding pictures. These results demonstrate that, just like syntactic predictions, lexical-semantic predictions can affect early visual processing around ∼100ms, suggesting that the M100 response is not exclusively tuned to recognizing visual features relevant to syntactic category analysis. Rather, the brain might generate predictions about upcoming visual input whenever it can. However, visual effects of lexical-semantic violations only occurred when a single lexical item could be predicted. We argue that this may be due to the fact that in natural language processing, there is typically no straightforward mapping between lexical-semantic fields (e.g., flowers) and visual or auditory forms (e.g., tulip, rose, magnolia). For syntactic categories, in contrast, certain form features do reliably correlate with category membership. This difference may, in part, explain why certain syntactic effects typically occur much earlier than lexical-semantic effects.

Dynamic causal modeling of subcortical connectivity of language

Subcortical-cortical interactions in the language network were investigated using dynamic causal modeling of magnetoencephalographic data recorded during auditory comprehension. Participants heard sentences that either were correct or contained violations. Sentences containing violations had syntactic or prosodic violations or both. We show that a hidden source, modeling magnetically silent deep nuclei, is required to explain the data best. This is in line with recent brain imaging studies and intracranial recordings suggesting an involvement of subcortical structures in language processing. Here, the processing of syntactic and prosodic violations elicited a global increase in the amplitude of evoked responses, both at the cortical and subcortical levels. As estimated by Bayesian model averaging, this was accompanied by various changes in cortical-cortical and subcortical-cortical connectivity. The most consistent findings in relation to violations were a decrease of reentrant inputs to Heschl's gyrus (HG) and of transcallosal lateral connections. These results suggest that in conditions where one hemisphere detects a violation, possibly via fast thalamocortical (HG) loops, the intercallosal connectivity is reduced to allow independent processing of syntax (left hemisphere) and of prosody (right hemisphere). This study is the first demonstration in cognitive neuroscience that subcortical-cortical loops can be empirically investigated using noninvasive electrophysiological recordings.

Who was the agent? The neural correlates of reanalysis processes during sentence comprehension

Sentence comprehension is a complex process. Besides identifying the meaning of each word and processing the syntactic structure of a sentence, it requires the computation of thematic information, that is, information about who did what to whom. The present fMRI study investigated the neural basis for thematic reanalysis (reanalysis of the thematic roles initially assigned to noun phrases in a sentence) and its interplay with syntactic reanalysis (reanalysis of the underlying syntactic structure originally constructed for a sentence). Thematic reanalysis recruited a network consisting of Broca's area, that is, the left pars triangularis (LPT), and the left posterior superior temporal gyrus, whereas only LPT showed greater sensitivity to syntactic reanalysis. These data provide direct evidence for a functional neuroanatomical basis for two linguistically motivated reanalysis processes during sentence comprehension.