Syntactically Based Sentence Processing Classes: Evidence from Event-Related Brain Potentials

Portions and sorts in Icelandic: an ERP study

An ERP study investigated the processing of mass nouns used to convey 'portions' vs. 'sorts' interpretations in Icelandic. The sorts interpretation requires semantic Coercion to a count noun; the portions interpretation entails extra syntactic processing. Compared to a Neutral condition, Coercion escaped the expected penalty (N400), but the Extra Syntax condition incurred the anticipated costs (anterior negativity followed by P600). Furthermore, we examined the effects of having to revise an initial commitment to head-noun status. When another noun follows the mass noun (creating a compound), the second noun becomes the head-noun. We hypothesized, for Icelandic, there would be no effect for Extra Syntax because the compound should have been built before the second noun was encountered; by contrast, for the Coercion and Neutral conditions, processing costs would be incurred to detect and reconfigure the second noun as the head. These predictions were largely borne out (early and sustained anterior negativities).

Going the Distance: Memory and Control Processes in Active Dependency Construction

Filler–gap dependencies make strong demands on working memory in language comprehension because they cannot always be immediately resolved. In a series of three reading-time studies, we test the idea that these demands can be decomposed into active maintenance processes and retrieval events. Results indicate that the fact that a displaced phrase exists and the identity of its basic syntactic category both immediately impact comprehension at potential gap sites. In contrast, specific lexical details of the displaced phrase show an immediate effect only for short dependencies and a much later effect for longer dependencies. We argue that coarse-grained information about the filler is actively maintained and is used to make phrase structure parsing decisions, whereas finer grained information is more quickly released from active maintenance and consequently has to be retrieved at the gap site.

The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics

How do people make sense of the sequential images in visual narratives like comics? A growing literature of recent research has suggested that this comprehension involves the interaction of multiple systems: The creation of meaning across sequential images relies on a "narrative grammar" that packages conceptual information into categorical roles organized in hierarchic constituents. These images are encapsulated into panels arranged in the layout of a physical page. Finally, how panels frame information can impact both the narrative structure and page layout. Altogether, these systems operate in parallel to construct the Gestalt whole of comprehension of this visual language found in comics.

The association between aerobic fitness and language processing in children: implications for academic achievement

Event-related brain potentials (ERPs) have been instrumental for discerning the relationship between children's aerobic fitness and aspects of cognition, yet language processing remains unexplored. ERPs linked to the processing of semantic information (the N400) and the analysis of language structure (the P600) were recorded from higher and lower aerobically fit children as they read normal sentences and those containing semantic or syntactic violations. Results revealed that higher fit children exhibited greater N400 amplitude and shorter latency across all sentence types, and a larger P600 effect for syntactic violations. Such findings suggest that higher fitness may be associated with a richer network of words and their meanings, and a greater ability to detect and/or repair syntactic errors. The current findings extend previous ERP research explicating the cognitive benefits associated with greater aerobic fitness in children and may have important implications for learning and academic performance.

Shining light on infants' discovery of structure

Learning and discovery seem often to begin with noting patterns. Human infants are skilled at pattern detection, even patterns only definable at an abstract level, which is key to their acquisition of complex knowledge systems such as language and music. However, research examining infants' abstract rule learning has generated inconsistent results. We propose that apparent domain differences in infants' abstract rule learning may be the result of extraneous stimulus variation and discrepancies in the methodologies employed across studies probing this skill. We discuss how a behavioral methodology indexing infants' online learning would be valuable in furthering understanding of infants' (as well as adults') abstract rule learning and its neurophysiological concomitants. We outline current research aimed at developing such an index, and we propose future research, pairing such techniques with neurophysiological methods, aimed at shining more light on human skill at discovering structure.

Distinguishing neurocognitive processes reflected by P600 effects: evidence from ERPs and neural oscillations

Research on language comprehension using event-related potentials (ERPs) reported distinct ERP components reliably related to the processing of semantic (N400) and syntactic information (P600). Recent ERP studies have challenged this well-defined distinction by showing P600 effects for semantic and pragmatic anomalies. So far, it is still unresolved whether the P600 reflects specific or rather common processes. The present study addresses this question by investigating ERPs in response to a syntactic and pragmatic (irony) manipulation, as well as a combined syntactic and pragmatic manipulation. For the syntactic condition, a morphosyntactic violation was applied, whereas for the pragmatic condition, such as “That is rich”, either an ironic or literal interpretation was achieved, depending on the prior context. The ERPs at the critical word showed a LAN-P600 pattern for syntactically incorrect sentences relative to correct ones. For ironic compared to literal sentences, ERPs showed a P200 effect followed by a P600 component. In comparison of the syntax-related P600 to the irony-related P600, distributional differences were found. Moreover, for the P600 time window (i.e., 500–900 ms), different changes in theta power between the syntax and pragmatics effects were found, suggesting that different patterns of neural activity contributed to each respective effect. Thus, both late positivities seem to be differently sensitive to these two types of linguistic information, and might reflect distinct neurocognitive processes, such as reanalysis of the sentence structure versus pragmatic reanalysis.

Toward a computational framework for cognitive biology: unifying approaches from cognitive neuroscience and comparative cognition

Progress in understanding cognition requires a quantitative, theoretical framework, grounded in the other natural sciences and able to bridge between implementational, algorithmic and computational levels of explanation. I review recent results in neuroscience and cognitive biology that, when combined, provide key components of such an improved conceptual framework for contemporary cognitive science. Starting at the neuronal level, I first discuss the contemporary realization that single neurons are powerful tree-shaped computers, which implies a reorientation of computational models of learning and plasticity to a lower, cellular, level. I then turn to predictive systems theory (predictive coding and prediction-based learning) which provides a powerful formal framework for understanding brain function at a more global level. Although most formal models concerning predictive coding are framed in associationist terms, I argue that modern data necessitate a reinterpretation of such models in cognitive terms: as model-based predictive systems. Finally, I review the role of the theory of computation and formal language theory in the recent explosion of comparative biological research attempting to isolate and explore how different species differ in their cognitive capacities. Experiments to date strongly suggest that there is an important difference between humans and most other species, best characterized cognitively as a propensity by our species to infer tree structures from sequential data. Computationally, this capacity entails generative capacities above the regular (finite-state) level; implementationally, it requires some neural equivalent of a push-down stack. I dub this unusual human propensity "dendrophilia", and make a number of concrete suggestions about how such a system may be implemented in the human brain, about how and why it evolved, and what this implies for models of language acquisition. I conclude that, although much remains to be done, a neurally-grounded framework for theoretical cognitive science is within reach that can move beyond polarized debates and provide a more adequate theoretical future for cognitive biology.

ERP correlates of German Sign Language processing in deaf native signers

Background

The present study investigated the neural correlates of sign language processing of Deaf people who had learned German Sign Language (Deutsche Gebärdensprache, DGS) from their Deaf parents as their first language. Correct and incorrect signed sentences were presented sign by sign on a computer screen. At the end of each sentence the participants had to judge whether or not the sentence was an appropriate DGS sentence. Two types of violations were introduced: (1) semantically incorrect sentences containing a selectional restriction violation (implausible object); (2) morphosyntactically incorrect sentences containing a verb that was incorrectly inflected (i.e., incorrect direction of movement). Event-related brain potentials (ERPs) were recorded from 74 scalp electrodes.

Results

Semantic violations (implausible signs) elicited an N400 effect followed by a positivity. Sentences with a morphosyntactic violation (verb agreement violation) elicited a negativity followed by a broad centro-parietal positivity.

Conclusions

ERP correlates of semantic and morphosyntactic aspects of DGS clearly differed from each other and showed a number of similarities with those observed in other signed and oral languages. These data suggest a similar functional organization of signed and oral languages despite the visual-spacial modality of sign language.

The fox and the cabra: an ERP analysis of reading code switched nouns and verbs in bilingual short stories

Comprehending a language (or code) switch within a sentence context triggers 2 electrophysiological signatures: an early left anterior negativity post code switch onset - a LAN - followed by a Late Positive Component (LPC). Word class and word position modulate lexico-semantic processes in the monolingual brain, e.g., larger N400 amplitude for nouns than verbs and for earlier than later words in the sentence. Here we test whether the bilingual brain is affected by word class and word position when code switching, or if the cost of switching overrides these lexico-semantic and sentence context factors. Adult bilinguals read short stories in English containing 8 target words. Targets were nouns or verbs, occurred early or late in a story and were presented alternately in English (non-switch) or Spanish (switch) across different story versions. Overall, switched words elicited larger LAN and LPC amplitude than non-switched words. The N400 amplitude was larger for nouns than verbs, more focal for switches than non-switches, and for early than late nouns but not for early than late verbs. Moreover, an early LPC effect was observed only for switched nouns, but not verbs. Together, this indicates that referential elements (nouns) may be harder to process and integrate than relational elements (verbs) in discourse, and when switched, nouns incur higher integration cost. Word position did not modulate the code switching effects, implying that switching between languages may invoke discourse independent processes.

Can the meaning of multiple words be integrated unconsciously?

What are the limits of unconscious language processing? Can language circuits process simple grammatical constructions unconsciously and integrate the meaning of several unseen words? Using behavioural priming and electroencephalography (EEG), we studied a specific rule-based linguistic operation traditionally thought to require conscious cognitive control: the negation of valence. In a masked priming paradigm, two masked words were successively (Experiment 1) or simultaneously presented (Experiment 2), a modifier ('not'/'very') and an adjective (e.g. 'good'/'bad'), followed by a visible target noun (e.g. 'peace'/'murder'). Subjects indicated whether the target noun had a positive or negative valence. The combination of these three words could either be contextually consistent (e.g. 'very bad - murder') or inconsistent (e.g. 'not bad - murder'). EEG recordings revealed that grammatical negations could unfold partly unconsciously, as reflected in similar occipito-parietal N400 effects for conscious and unconscious three-word sequences forming inconsistent combinations. However, only conscious word sequences elicited P600 effects, later in time. Overall, these results suggest that multiple unconscious words can be rapidly integrated and that an unconscious negation can automatically 'flip the sign' of an unconscious adjective. These findings not only extend the limits of subliminal combinatorial language processes, but also highlight how consciousness modulates the grammatical integration of multiple words.

Differential task effects on N400 and P600 elicited by semantic and syntactic violations

Syntactic violations in sentences elicit a P600 component in the event-related potential, which is frequently interpreted as signaling reanalysis or repair of the sentence structure. However, P600 components have been reported also for semantic and combined semantic and syntactic violations, giving rise to still other interpretations. In many of these studies, the violation might be of special significance for the task of the participants; however there is a lack of studies directly targeting task effects on the P600. Here we repeated a previously published study but using a probe verification task, focusing on individual words rather than on sentence correctness and directly compared the results with the previous ones. Although a (somewhat smaller) N400 component occurred also in the present study, we did not observe a parietal P600 component. Instead, we found a late anterior negativity. Possibly, the parietal P600 observed in sentence acceptability paradigms relates to the target value of the violations or to late sentence structure-specific processes that are more task-sensitive than the N400 and which are or not initiated in the probe verification task. In any case the present findings show a strong dependency of P600-eliciting processes from attention to the sentences context whereas the N400 eliciting processes appear relatively robust.

Morphosyntax can modulate the N400 component: event related potentials to gender-marked post-nominal adjectives

Event-related potential studies of grammatical gender agreement often report a left anterior negativity (LAN) when agreement violations occur. Some studies have shown that during sentence comprehension gender violations can also interact with semantic processing to modulate a negativity associated with processing meaning - the N400. Given that the LAN and N400 overlap in time, they are identified by their scalp distributions and purported functional roles. Critically, grammatical gender violations also elicit a right posterior positivity that can overlap temporally and potentially affect the scalp distribution of the LAN/N400. We measured the effect of grammatical gender violations in the LAN/N400 window and late positive component (LPC) during comprehension of Spanish sentences. A post-nominal adjective could either make sense or not, and either agree or disagree in gender with the preceding noun. We observed a negativity to gender agreement violations in the LAN/N400 window (300-500ms post stimulus onset) that was smaller than the semantic-congruity N400, but overlapped with it in time and distribution. The early portion of the LPC to gender violations was modulated by sentence constraint, occurring as early as 450ms in highly constraining sentences. A subadditive interaction occurred at the later portion of the LPC with equivalent effects for single and double violations (gender and semantics), reflecting a general stage of reprocessing. Overall, our data support models of language comprehension whereby both semantic and morphosyntactic information can affect processing at similar time points.

Processing ser and estar to locate objects and events: An ERP study with L2 speakers of Spanish

In Spanish locative constructions, a different form of the copula is selected in relation to the semantic properties of the grammatical subject: sentences that locate objects require estar while those that locate events require ser (both translated in English as 'to be'). In an ERP study, we examined whether second language (L2) speakers of Spanish are sensitive to the selectional restrictions that the different types of subjects impose on the choice of the two copulas. Twenty-four native speakers of Spanish and two groups of L2 Spanish speakers (24 beginners and 18 advanced speakers) were recruited to investigate the processing of 'object/event + estar/ser' permutations. Participants provided grammaticality judgments on correct (object + estar; event + ser) and incorrect (object + ser; event + estar) sentences while their brain activity was recorded. In line with previous studies (Leone-Fernández, Molinaro, Carreiras, & Barber, 2012; Sera, Gathje, & Pintado, 1999), the results of the grammaticality judgment for the native speakers showed that participants correctly accepted object + estar and event + ser constructions. In addition, while 'object + ser' constructions were considered grossly ungrammatical, 'event + estar' combinations were perceived as unacceptable to a lesser degree. For these same participants, ERP recording time-locked to the onset of the critical word 'en' showed a larger P600 for the ser predicates when the subject was an object than when it was an event (*La silla es en la cocina vs. La fiesta es en la cocina). This P600 effect is consistent with syntactic repair of the defining predicate when it does not fit with the adequate semantic properties of the subject. For estar predicates (La silla está en la cocina vs. *La fiesta está en la cocina), the findings showed a central-frontal negativity between 500-700 ms. Grammaticality judgment data for the L2 speakers of Spanish showed that beginners were significantly less accurate than native speakers in all conditions, while the advanced speakers only differed from the natives in the event+ser and event+estar conditions. For the ERPs, the beginning learners did not show any effects in the time-windows under analysis. The advanced speakers showed a pattern similar to that of native speakers: (1) a P600 response to 'object + ser' violation more central and frontally distributed, and (2) a central-frontal negativity between 500-700 ms for 'event + estar' violation. Findings for the advanced speakers suggest that behavioral methods commonly used to assess grammatical knowledge in the L2 may be underestimating what L2 speakers have actually learned.

Native-like brain processing of syntax can be attained by university foreign language learners

Using event-related potentials (ERPs), we examined the neurocognition of late-learned second language (L2) Spanish in two groups of typical university foreign-language learners (as compared to native (L1) speakers): one group with only one year of college classroom experience, and low-intermediate proficiency (L2 Low), and another group with over three years of college classroom experience as well as 1-2 semesters of immersion experience abroad, and advanced proficiency (L2 Advanced). Semantic violations elicited N400s in all three groups, whereas syntactic word-order violations elicited LAN/P600 responses in the L1 and L2 Advanced groups, but not the L2 Low group. Indeed, the LAN and P600 responses were statistically indistinguishable between the L1 and L2 Advanced groups. The results support and extend previous findings. Consistent with previous research, the results suggest that L2 semantic processing always depends on L1-like neurocognitive mechanisms, whereas L2 syntactic processing initially differs from L1, but can shift to native-like processes with sufficient proficiency or exposure, and perhaps with immersion experience in particular. The findings further demonstrate that substantial native-like brain processing of syntax can be achieved even by typical university foreign-language learners.

An ERP study of structural anomalies in native and semantic free artificial grammar: evidence for shared processing mechanisms

Artificial grammars have been widely applied to the study of sequential learning in language, but few studies have directly compared the neural correlates of artificial and native grammar processing. In this study, we examined Event Related Potentials (ERPs) elicited by structural anomalies in semantic-free artificial grammar sequences and sentences in the subjects' native language (Spanish). Although ERPs differed during early stages, we observed similar posterior negativities (N400) and P600 effects in a late stage. We interpret these results as evidence of at least partially shared neural mechanisms for processing of language and artificial grammars. We suggest that in both the natural and artificial grammars, the N400 and P600 components we observed can be explained as the result of unfulfilled predictions about incoming stimuli.

The human brain processes syntax in the absence of conscious awareness

Syntax is the core computational component of language. A longstanding idea about syntactic processing is that it is generally not available to conscious access, operating autonomously and automatically. However, there is little direct neurocognitive evidence on this issue. By measuring event-related potentials while human observers performed a novel cross-modal distraction task, we demonstrated that syntactic violations that were not consciously detected nonetheless produced a characteristic early neural response pattern, and also significantly delayed reaction times to a concurrent task. This early neural response was distinct from later neural activity that was observed only to syntactic violations that were consciously detected. These findings provide direct evidence that the human brain reacts to violations of syntax even when these violations are not consciously detected, indicating that even highly complex computational processes such as syntactic processing can occur outside the narrow window of conscious awareness.

Syntax does not necessarily precede semantics in sentence processing: ERP evidence from Chinese

Two event-related potential experiments were conducted to examine whether the processing of syntactic category or syntactic subcategorization frame always needs to temporally precede semantic processing during the reading of Chinese sentences of object-subject-verb construction. The sentences contained (a) no anomalies, (b) semantic only anomalies, (c) syntactic category plus semantic anomalies, or (d) transitivity plus semantic anomalies. In both experiments, all three types of anomalies elicited a broad negativity between 300 and 500 ms. This negativity included an N400 effect, given its distribution. Moreover, syntactic category plus semantic anomalies elicited a P600 response, whereas the other two types of anomalies did not. The finding of N400 effects suggests that semantic integration can be attempted even when the processing of syntactic category or syntactic subcategorization frame is unsuccessful. Thus, syntactic processing is not a necessary prerequisite for the initiation of semantic integration in Chinese.

Generating predictions: lesion evidence on the role of left inferior frontal cortex in rapid syntactic analysis

A well-documented phenomenon in event-related electroencephalography (EEG) and magnetoencephalography (MEG) studies on language processing is that syntactic violations of different types elicit negativities as early as 100 msec after the violation point. Recently, these responses have been associated with activations in or very close to sensory cortices, suggesting the involvement of basic sensory mechanisms in the detection of syntactic violations. The present study investigated whether intact auditory cortices and adjacent temporal regions are sufficient to generate early syntactic negativities in the auditory event-related potential (ERP). We tested ten clinically non-aphasic patients with left inferior frontal lesions, but intact temporal cortices in a passive auditory ERP paradigm that had reliably elicited early negativities in response to violations of subject-verb agreement and word category in the past. Subject-verb agreement violations failed to elicit early grammaticality effects in these patients, whereas a group of ten age-matched controls showed a reliable early negativity. This finding supports the idea that sensory aspects of syntactic analysis as reflected in early syntactic negativities critically depend on top-down predictions generated by the left inferior frontal cortex. In contrast, word category violations elicited a small, marginally significant early negativity both in controls and patients, suggesting an additional involvement of temporal regions in early phrase structure processing. In an additional auditory oddball experiment patients showed a regular P300, but no N2b component in response to deviant tones, indicating that their deficit in generating sensory predictions extends beyond the language domain.

Multilingualism and fMRI: Longitudinal Study of Second Language Acquisition

BOLD fMRI is often used for the study of human language. However, there are still very few attempts to conduct longitudinal fMRI studies in the study of language acquisition by measuring auditory comprehension and reading. The following paper is the first in a series concerning a unique longitudinal study devoted to the analysis of bi- and multilingual subjects who are: (1) already proficient in at least two languages; or (2) are acquiring Russian as a second/third language. The focus of the current analysis is to present data from the auditory sections of a set of three scans acquired from April, 2011 through April, 2012 on a five-person subject pool who are learning Russian during the study. All subjects were scanned using the same protocol for auditory comprehension on the same General Electric LX 3T Signa scanner in Duke University Hospital. Using a multivariate analysis of covariance (MANCOVA) for statistical analysis, proficiency measurements are shown to correlate significantly with scan results in the Russian conditions over time. The importance of both the left and right hemispheres in language processing is discussed. Special attention is devoted to the importance of contextualizing imaging data with corresponding behavioral and empirical testing data using a multivariate analysis of variance. This is the only study to date that includes: (1) longitudinal fMRI data with subject-based proficiency and behavioral data acquired in the same time frame; and (2) statistical modeling that demonstrates the importance of covariate language proficiency data for understanding imaging results of language acquisition.

Syntactic computations in the language network: characterizing dynamic network properties using representational similarity analysis

The core human capacity of syntactic analysis involves a left hemisphere network involving left inferior frontal gyrus (LIFG) and posterior middle temporal gyrus (LMTG) and the anatomical connections between them. Here we use magnetoencephalography (MEG) to determine the spatio-temporal properties of syntactic computations in this network. Listeners heard spoken sentences containing a local syntactic ambiguity (e.g., “… landing planes …”), at the offset of which they heard a disambiguating verb and decided whether it was an acceptable/unacceptable continuation of the sentence. We charted the time-course of processing and resolving syntactic ambiguity by measuring MEG responses from the onset of each word in the ambiguous phrase and the disambiguating word. We used representational similarity analysis (RSA) to characterize syntactic information represented in the LIFG and left posterior middle temporal gyrus (LpMTG) over time and to investigate their relationship to each other. Testing a variety of lexico-syntactic and ambiguity models against the MEG data, our results suggest early lexico-syntactic responses in the LpMTG and later effects of ambiguity in the LIFG, pointing to a clear differentiation in the functional roles of these two regions. Our results suggest the LpMTG represents and transmits lexical information to the LIFG, which responds to and resolves the ambiguity.

Implicit and explicit second language training recruit common neural mechanisms for syntactic processing

In contrast to native language acquisition, adult second-language (L2) acquisition occurs under highly variable learning conditions. Although most adults acquire their L2 at least partially through explicit instruction, as in a classroom setting, many others acquire their L2 primarily through implicit exposure, as is typical of an immersion environment. Whether these differences in acquisition environment play a role in determining the neural mechanisms that are ultimately recruited to process L2 grammar has not been well characterized. This study investigated this issue by comparing the ERP response to novel L2 syntactic rules acquired under conditions of implicit exposure and explicit instruction, using a novel laboratory language-learning paradigm. Native speakers tested on these stimuli showed a biphasic response to syntactic violations, consisting of an earlier negativity followed by a later P600 effect. After merely an hour of training, both implicitly and explicitly trained learners who were capable of detecting grammatical violations also elicited P600 effects. In contrast, learners who were unable to discriminate between grammatically correct and incorrect sentences did not show significant P600 effects. The magnitude of the P600 effect was found to correlate with learners' behavioral proficiency. Behavioral measures revealed that successful learners from both the implicit and explicit groups gained explicit, verbalizable knowledge about the L2 grammar rules. Taken together, these results indicate that late, controlled mechanisms indexed by the P600 play a crucial role in processing a late-learned L2 grammar, regardless of training condition. These findings underscore the remarkable plasticity of later, attention-dependent processes and their importance in lifelong learning.

Neural mechanisms of rapid sensitivity to syntactic anomaly

Recent psycholinguistic models hypothesize that anticipatory processing can speed the response to linguistic input during language comprehension by pre-activating representations necessary for word recognition. We investigated the neurocognitive mechanisms of anticipatory processing by recording event-related potentials (ERPs) to syntactically anomalous (The thief was caught byfor police) and well-formed (e.g., The thief was caught bythe police) sentences. One group of participants saw anomalies elicited by the same word in every instance (e.g., for; low-variability stimuli), providing high affordances for predictions about the word-form appearing in the critical position. A second group saw anomalies elicited by seven different prepositions (at, of, on, for, from, over, with; high-variability stimuli) across the study, creating a more difficult prediction task. Syntactic category anomalies enhanced the occipital-temporal N170 component of the ERP, indicating rapid sensitivity – within 200 ms of word-onset – to syntactic anomaly. For low-variability but not the high-variability stimuli, syntactic anomaly also enhanced the earlier occipital-temporal P1 component, around 130 ms after word-onset, indicating that affordances for prediction engendered earlier sensitivity to syntactic anomaly. Independent components analysis revealed three sources within the ERP signal whose functional dynamics were consistent with predictive processing and early responses to syntactic anomaly. Distributed neural source modeling (sLORETA) of these early active sources produced a candidate network for early responses to words during reading in the right posterior occipital, left occipital-temporal, and medial parietal cortex.

Deduction electrified: ERPs elicited by the processing of words in conditional arguments

This study investigates the ERP components associated with the processing of words that are critical to generating and rejecting deductive conditional Modus Ponens arguments (If P then Q; P//Therefore, Q). The generation of a logical inference is investigated by placing a verb in the minor premise that matches the one used in the antecedent of the conditional premise so that the inference can be carried out (If John is sleeping then he is snoring; John is sleeping). Rejections are examined by placing verbs that are associates of the verb that would make the conclusion valid (Conclusion 'therefore John is dreaming' in the example above). The inference generation phase was characterized by two ERP components, namely the P3b and the PSW. Rejections were associated with an N2 and a late positive component. The implications of these results regarding the processing of words in an inferential context are discussed.

Musical expertise modulates early processing of syntactic violations in language

Syntactic violations in speech and music have been shown to elicit an anterior negativity (AN) as early as 100 ms after violation onset and a posterior positivity that peaks at roughly 600 ms (P600/LPC). The language AN is typically reported as left-lateralized (LAN), whereas the music AN is typically reported as right-lateralized (RAN). However, several lines of evidence suggest syntactic processing of language and music rely on overlapping neural systems. The current study tested the hypothesis that syntactic processing of speech and music share neural resources by examining whether musical proficiency modulates ERP indices of linguistic syntactic processing. ERPs were measured in response to syntactic violations in sentences and chord progressions in musicians and non-musicians. Violations in speech were insertion errors in normal and semantically impoverished English sentences. Violations in music were out-of-key chord substitutions from distantly and closely related keys. Phrase-structure violations elicited an AN and P600 in both groups. Harmonic violations elicited an LPC in both groups, blatant harmonic violations also elicited a RAN in musicians only. Cross-domain effects of musical proficiency were similar to previously reported within-domain effects of linguistic proficiency on the distribution of the language AN; syntactic violations in normal English sentences elicited a LAN in musicians and a bilateral AN in non-musicians. The late positivities elicited by violations differed in latency and distribution between domains. These results suggest that initial processing of syntactic violations in language and music relies on shared neural resources in the general population, and that musical expertise results in more specialized cortical organization of syntactic processing in both domains.

Cross-linguistic differences in the neural representation of human language: evidence from users of signed languages

Studies of deaf individuals who are users of signed languages have provided profound insight into the neural representation of human language. Case studies of deaf signers who have incurred left- and right-hemisphere damage have shown that left-hemisphere resources are a necessary component of sign language processing. These data suggest that, despite frank differences in the input and output modality of language, core left perisylvian regions universally serve linguistic function. Neuroimaging studies of deaf signers have generally provided support for this claim. However, more fine-tuned studies of linguistic processing in deaf signers are beginning to show evidence of important differences in the representation of signed and spoken languages. In this paper, we provide a critical review of this literature and present compelling evidence for language-specific cortical representations in deaf signers. These data lend support to the claim that the neural representation of language may show substantive cross-linguistic differences. We discuss the theoretical implications of these findings with respect to an emerging understanding of the neurobiology of language.

Brain responses before and after intensive second language learning: proficiency based changes and first language background effects in adult learners

This longitudinal study tracked the neuro-cognitive changes associated with second language (L2) grammar learning in adults in order to investigate how L2 processing is shaped by a learner's first language (L1) background and L2 proficiency. Previous studies using event-related potentials (ERPs) have argued that late L2 learners cannot elicit a P600 in response to L2 grammatical structures that do not exist in the L1 or that are different in the L1 and L2. We tested whether the neuro-cognitive processes underlying this component become available after intensive L2 instruction. Korean- and Chinese late-L2-learners of English were tested at the beginning and end of a 9-week intensive English-L2 course. ERPs were recorded while participants read English sentences containing violations of regular past tense (a grammatical structure that operates differently in Korean and does not exist in Chinese). Whereas no P600 effects were present at the start of instruction, by the end of instruction, significant P600s were observed for both L1 groups. Latency differences in the P600 exhibited by Chinese and Korean speakers may be attributed to differences in L1-L2 reading strategies. Across all participants, larger P600 effects at session 2 were associated with: 1) higher levels of behavioural performance on an online grammaticality judgment task; and 2) with correct, rather than incorrect, behavioural responses. These findings suggest that the neuro-cognitive processes underlying the P600 (e.g., "grammaticalization") are modulated by individual levels of L2 behavioural performance and learning.

Electrophysiological responses to argument structure violations in healthy adults and individuals with agrammatic aphasia

Sentence comprehension requires processing of argument structure information associated with verbs, i.e. the number and type of arguments that they select. Many individuals with agrammatic aphasia show impaired production of verbs with greater argument structure density. The extent to which these participants also show argument structure deficits during comprehension, however, is unclear. Some studies find normal access to verb arguments, whereas others report impaired ability. The present study investigated verb argument structure processing in agrammatic aphasia by examining event-related potentials associated with argument structure violations in healthy young and older adults as well as aphasic individuals. A semantic violation condition was included to investigate possible differences in sensitivity to semantic and argument structure information during sentence processing. Results for the healthy control participants showed a negativity followed by a positive shift (N400-P600) in the argument structure violation condition, as found in previous ERP studies (Friederici & Frisch, 2000; Frisch, Hahne, & Friederici, 2004). In contrast, individuals with agrammatic aphasia showed a P600, but no N400, response to argument structure mismatches. Additionally, compared to the control groups, the agrammatic participants showed an attenuated, but relatively preserved, N400 response to semantic violations. These data show that agrammatic individuals do not demonstrate normal real-time sensitivity to verb argument structure requirements during sentence processing.

Information structure influences depth of syntactic processing: event-related potential evidence for the Chomsky illusion

Information structure facilitates communication between interlocutors by highlighting relevant information. It has previously been shown that information structure modulates the depth of semantic processing. Here we used event-related potentials to investigate whether information structure can modulate the depth of syntactic processing. In question-answer pairs, subtle (number agreement) or salient (phrase structure) syntactic violations were placed either in focus or out of focus through information structure marking. P600 effects to these violations reflect the depth of syntactic processing. For subtle violations, a P600 effect was observed in the focus condition, but not in the non-focus condition. For salient violations, comparable P600 effects were found in both conditions. These results indicate that information structure can modulate the depth of syntactic processing, but that this effect depends on the salience of the information. When subtle violations are not in focus, they are processed less elaborately. We label this phenomenon the Chomsky illusion.

Composition in distributional models of semantics

Vector-based models of word meaning have become increasingly popular in cognitive science. The appeal of these models lies in their ability to represent meaning simply by using distributional information under the assumption that words occurring within similar contexts are semantically similar. Despite their widespread use, vector-based models are typically directed at representing words in isolation, and methods for constructing representations for phrases or sentences have received little attention in the literature. This is in marked contrast to experimental evidence (e.g., in sentential priming) suggesting that semantic similarity is more complex than simply a relation between isolated words. This article proposes a framework for representing the meaning of word combinations in vector space. Central to our approach is vector composition, which we operationalize in terms of additive and multiplicative functions. Under this framework, we introduce a wide range of composition models that we evaluate empirically on a phrase similarity task.

Effects of verbal event structure on online thematic role assignment

Event structure describes the relationships between general semantics (Aktionsart) of the verb and its syntactic properties, separating verbs into two classes: telic verbs, which denote change of state events with an inherent end-point or boundary (catch, rescue), and atelic, which refer to homogenous activities (tease, host). As telic verbs describe events, in which the internal argument (Patient) is affected, we hypothesized that processing of telic verb template would activate syntactic position of the Patient during sentence comprehension. Event-related brain potentials (ERPs) were recorded from 20 English speakers, who read sentences with reduced Object relative clauses, in which the verb was either telic or atelic. ERPs in relative clauses diverged on the definite article preceding the Agent: the atelic condition was characterized by larger amplitude negativity at the N100. Such processing differences are explained by activation of the syntactic position for the Patient by the event structure template of telic verbs.

Co-localizing linguistic and musical syntax with intracranial EEG

Despite general agreement on shared syntactic resources in music and language, the neuroanatomical underpinnings of this overlap remain largely unexplored. While previous studies mainly considered frontal areas as supramodal grammar processors, the domain-general syntactic role of temporal areas has been so far neglected. Here we capitalized on the excellent spatial and temporal resolution of subdural EEG recordings to co-localize low-level syntactic processes in music and language in the temporal lobe in a within-subject design. We used Brain Surface Current Density mapping to localize and compare neural generators of the early negativities evoked by violations of phrase structure grammar in both music and spoken language. The results show that the processing of syntactic violations relies in both domains on bilateral temporo-fronto-parietal neural networks. We found considerable overlap of these networks in the superior temporal lobe, but also differences in the hemispheric timing and relative weighting of their fronto-temporal constituents. While alluding to the dissimilarity in how shared neural resources may be configured depending on the musical or linguistic nature of the perceived stimulus, the combined data lend support for a co-localization of early musical and linguistic syntax processing in the temporal lobe.

(Pea)nuts and bolts of visual narrative: structure and meaning in sequential image comprehension

Just as syntax differentiates coherent sentences from scrambled word strings, the comprehension of sequential images must also use a cognitive system to distinguish coherent narrative sequences from random strings of images. We conducted experiments analogous to two classic studies of language processing to examine the contributions of narrative structure and semantic relatedness to processing sequential images. We compared four types of comic strips: (1) Normal sequences with both structure and meaning, (2) Semantic Only sequences (in which the panels were related to a common semantic theme, but had no narrative structure), (3) Structural Only sequences (narrative structure but no semantic relatedness), and (4) Scrambled sequences of randomly-ordered panels. In Experiment 1, participants monitored for target panels in sequences presented panel-by-panel. Reaction times were slowest to panels in Scrambled sequences, intermediate in both Structural Only and Semantic Only sequences, and fastest in Normal sequences. This suggests that both semantic relatedness and narrative structure offer advantages to processing. Experiment 2 measured ERPs to all panels across the whole sequence. The N300/N400 was largest to panels in both the Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in the Normal sequences. This implies that a combination of narrative structure and semantic relatedness can facilitate semantic processing of upcoming panels (as reflected by the N300/N400). Also, panels in the Scrambled sequences evoked a larger left-lateralized anterior negativity than panels in the Structural Only sequences. This localized effect was distinct from the N300/N400, and appeared despite the fact that these two sequence types were matched on local semantic relatedness between individual panels. These findings suggest that sequential image comprehension uses a narrative structure that may be independent of semantic relatedness. Altogether, we argue that the comprehension of visual narrative is guided by an interaction between structure and meaning.

Artificial grammar learning meets formal language theory: an overview

Formal language theory (FLT), part of the broader mathematical theory of computation, provides a systematic terminology and set of conventions for describing rules and the structures they generate, along with a rich body of discoveries and theorems concerning generative rule systems. Despite its name, FLT is not limited to human language, but is equally applicable to computer programs, music, visual patterns, animal vocalizations, RNA structure and even dance. In the last decade, this theory has been profitably used to frame hypotheses and to design brain imaging and animal-learning experiments, mostly using the 'artificial grammar-learning' paradigm. We offer a brief, non-technical introduction to FLT and then a more detailed analysis of empirical research based on this theory. We suggest that progress has been hampered by a pervasive conflation of distinct issues, including hierarchy, dependency, complexity and recursion. We offer clarifications of several relevant hypotheses and the experimental designs necessary to test them. We finally review the recent brain imaging literature, using formal languages, identifying areas of convergence and outstanding debates. We conclude that FLT has much to offer scientists who are interested in rigorous empirical investigations of human cognition from a neuroscientific and comparative perspective.

The influence of language deprivation in early childhood on L2 processing: An ERP comparison of deaf native signers and deaf signers with a delayed language acquisition

Background

To examine which language function depends on early experience, the present study compared deaf native signers, deaf non-native signers and hearing German native speakers while processing German sentences. The participants watched simple written sentences while event-related potentials (ERPs) were recorded. At the end of each sentence they were asked to judge whether the sentence was correct or not. Two types of violations were introduced in the middle of the sentence: a semantically implausible noun or a violation of subject-verb number agreement.

Results

The results showed a similar ERP pattern after semantic violations (an N400 followed by a positivity) in all three groups. After syntactic violations, native German speakers and native signers of German sign language (DGS) with German as second language (L2) showed a left anterior negativity (LAN) followed by a P600, whereas no LAN but a negativity over the right hemisphere instead was found in deaf participants with a delayed onset of first language (L1) acquisition. The P600 of this group had a smaller amplitude and a different scalp distribution as compared to German native speakers.

Conclusions

The results of the present study suggest that language deprivation in early childhood alters the cerebral organization of syntactic language processing mechanisms for L2. Semantic language processing instead was unaffected.

Dissociating linguistic and non-linguistic gesture processing: electrophysiological evidence from American Sign Language

A fundamental advance in our understanding of human language would come from a detailed account of how non-linguistic and linguistic manual actions are differentiated in real time by language users. To explore this issue, we targeted the N400, an ERP component known to be sensitive to semantic context. Deaf signers saw 120 American Sign Language sentences, each consisting of a "frame" (a sentence without the last word; e.g. BOY SLEEP IN HIS) followed by a "last item" belonging to one of four categories: a high-close-probability sign (a "semantically reasonable" completion to the sentence; e.g. BED), a low-close-probability sign (a real sign that is nonetheless a "semantically odd" completion to the sentence; e.g. LEMON), a pseudo-sign (phonologically legal but non-lexical form), or a non-linguistic grooming gesture (e.g. the performer scratching her face). We found significant N400-like responses in the incongruent and pseudo-sign contexts, while the gestures elicited a large positivity.

Second language processing shows increased native-like neural responses after months of no exposure

Although learning a second language (L2) as an adult is notoriously difficult, research has shown that adults can indeed attain native language-like brain processing and high proficiency levels. However, it is important to then retain what has been attained, even in the absence of continued exposure to the L2—particularly since periods of minimal or no L2 exposure are common. This event-related potential (ERP) study of an artificial language tested performance and neural processing following a substantial period of no exposure. Adults learned to speak and comprehend the artificial language to high proficiency with either explicit, classroom-like, or implicit, immersion-like training, and then underwent several months of no exposure to the language. Surprisingly, proficiency did not decrease during this delay. Instead, it remained unchanged, and there was an increase in native-like neural processing of syntax, as evidenced by several ERP changes—including earlier, more reliable, and more left-lateralized anterior negativities, and more robust P600s, in response to word-order violations. Moreover, both the explicitly and implicitly trained groups showed increased native-like ERP patterns over the delay, indicating that such changes can hold independently of L2 training type. The results demonstrate that substantial periods with no L2 exposure are not necessarily detrimental. Rather, benefits may ensue from such periods of time even when there is no L2 exposure. Interestingly, both before and after the delay the implicitly trained group showed more native-like processing than the explicitly trained group, indicating that type of training also affects the attainment of native-like processing in the brain. Overall, the findings may be largely explained by a combination of forgetting and consolidation in declarative and procedural memory, on which L2 grammar learning appears to depend. The study has a range of implications, and suggests a research program with potentially important consequences for second language acquisition and related fields.

Examining the evidence for an independent semantic analyzer: an ERP study in Spanish

Recent ERP findings challenge the widespread assumption that syntactic and semantic processes are tightly coupled. Syntactically well-formed sentences that are semantically anomalous due to thematic mismatches elicit a P600, the component standardly associated with syntactic anomaly. This 'thematic P600' effect has been attributed to detection of semantically plausible thematic relations that conflict with the surface syntactic structure of the sentence, implying a processing architecture with an independent semantic analyzer. A key finding is that the P600 is selectively sensitive to the presence of plausible verb-argument relations, and that otherwise an N400 is elicited (The hearty meal was devouring … vs. The dusty tabletop was devouring …: Kim & Osterhout, 2005). The current study investigates in Spanish whether the evidence for an independent semantic analyzer is better explained by a standard architecture that rapidly integrates multiple sources of lexical, syntactic, and semantic information. The study manipulated the presence of plausible thematic relations, and varied the choice of auxiliary between passive-biased fue and active-progressive biased estaba. Results show a late positivity that appeared as soon as comprehenders detected an improbable combination of subject animacy, auxiliary bias, or verb voice morphology. This effect appeared at the lexical verb in the fue conditions and at the auxiliary in the estaba conditions. The late positivity elicited by surface thematic anomalies was the same, regardless of the presence of a plausible non-surface interpretation, and no N400 effects were elicited. These findings do not implicate an independent semantic analyzer, and are compatible with standard language processing architectures.

On the early left-anterior negativity (ELAN) in syntax studies

Within the framework of Friederici's (2002) neurocognitive model of sentence processing, the early left anterior negativity (ELAN) in event-related potentials (ERPs) has been claimed to be a brain marker of syntactic first-pass parsing. As ELAN components seem to be exclusively elicited by word category violations (phrase structure violations), they have been taken as strong empirical support for syntax-first models of sentence processing and have gained considerable impact on psycholinguistic theory in a variety of domains. The present article reviews relevant ELAN studies and raises a number of serious issues concerning the reliability and validity of the findings. We also discuss how baseline problems and contextual factors can contribute to early ERP effects in studies examining word category violations. We conclude that--despite the apparent wealth of ELAN data--the functional significance of these findings remains largely unclear. The present paper does not claim to have falsified the existence of ELANs or syntax-related early frontal negativities. However, by separating facts from myths, the paper attempts to make a constructive contribution to how future ERP research in the area of syntax processing may better advance our understanding of online sentence comprehension.

The time-course and spatial distribution of brain activity associated with sentence processing

Sentence comprehension involves a host of highly interrelated processes, including syntactic parsing, semantic composition, and pragmatic inferencing. In neuroimaging, a primary paradigm for examining the brain bases of sentence processing has been to compare brain activity elicited by sentences versus unstructured lists of words. These studies commonly find an effect of increased activity for sentences in the anterior temporal lobes (aTL). Together with neuropsychological data, these findings have motivated the hypothesis that the aTL is engaged in sentence level combinatorics. Combinatoric processing during language comprehension, however, occurs within tens and hundreds of milliseconds, i.e., at a time-scale much faster than the temporal resolution of hemodynamic measures. Here, we examined the time-course of sentence-level processing using magnetoencephalography (MEG) to better understand the temporal profile of activation in this common paradigm and to test a key prediction of the combinatoric hypothesis: because sentences are interpreted incrementally, word-by-word, activity associated with basic linguistic combinatorics should be time-locked to word-presentation. Our results reveal increased anterior temporal activity for sentences compared to word lists beginning approximately 250 ms after word onset. We also observed increased activation in a network of other brain areas, extending across posterior temporal, inferior frontal, and ventral medial areas. These findings confirm a key prediction of the combinatoric hypothesis for the aTL and further elucidate the spatio-temporal characteristics of sentence-level computations in the brain.

Similar Neural Correlates for Language and Sequential Learning: Evidence from Event-Related Brain Potentials

We used event-related potentials (ERPs) to investigate the time course and distribution of brain activity while adults performed (a) a sequential learning task involving complex structured sequences, and (b) a language processing task. The same positive ERP deflection, the P600 effect, typically linked to difficult or ungrammatical syntactic processing, was found for structural incongruencies in both sequential learning as well as natural language, and with similar topographical distributions. Additionally, a left anterior negativity (LAN) was observed for language but not for sequential learning. These results are interpreted as an indication that the P600 provides an index of violations and the cost of integration of expectations for upcoming material when processing complex sequential structure. We conclude that the same neural mechanisms may be recruited for both syntactic processing of linguistic stimuli and sequential learning of structured sequence patterns more generally.

The brain basis of language processing: from structure to function

Language processing is a trait of human species. The knowledge about its neurobiological basis has been increased considerably over the past decades. Different brain regions in the left and right hemisphere have been identified to support particular language functions. Networks involving the temporal cortex and the inferior frontal cortex with a clear left lateralization were shown to support syntactic processes, whereas less lateralized temporo-frontal networks subserve semantic processes. These networks have been substantiated both by functional as well as by structural connectivity data. Electrophysiological measures indicate that within these networks syntactic processes of local structure building precede the assignment of grammatical and semantic relations in a sentence. Suprasegmental prosodic information overtly available in the acoustic language input is processed predominantly in a temporo-frontal network in the right hemisphere associated with a clear electrophysiological marker. Studies with patients suffering from lesions in the corpus callosum reveal that the posterior portion of this structure plays a crucial role in the interaction of syntactic and prosodic information during language processing.