Before the N400: effects of lexical-semantic violations in visual cortex

A further specification of the effects of font emphasis on reading comprehension: Evidence from event-related potentials and neural oscillations

The attention hypothesis, which assumes that font emphasis captures readers' attention, is usually used to explain the mechanism by which such emphasis operates. This study further delineates the attention hypothesis by investigating the ways in which font emphasis captures attention and its effects on the integration of emphasized information into the previous context. We computed event-related potentials and frequency band-specific electroencephalographic power changes occurring while participants read sentences containing critical words that were either emphasized (i.e., displayed in a color different from the other words in the sentence) or not (i.e., shown in the same color as the rest of the sentence) and semantically congruent with prior words or not. The results showed that the emphasized words (as compared to control words) elicited a reduced N1 and increased P2, indicating that font emphasis reduced familiarity-based visuo-orthographic processing and instead increased controlled attentional processing. We also observed greater P300 and power decreases in the alpha and beta frequency range in response to critical words in the emphasized condition, suggesting that font emphasis enhances focal attention to promote a fuller integration of information into the sentence context. Furthermore, relative to the control condition, the emphasized condition induced delta and theta power increases for the incongruent words. These results suggest that font emphasis increases the efficiency of glyph processing, which facilitates lexical access.

Rapid adaptation of predictive models during language comprehension: Aperiodic EEG slope, individual alpha frequency and idea density modulate individual differences in real-time model updating

Predictive coding provides a compelling, unified theory of neural information processing, including for language. However, there is insufficient understanding of how predictive models adapt to changing contextual and environmental demands and the extent to which such adaptive processes differ between individuals. Here, we used electroencephalography (EEG) to track prediction error responses during a naturalistic language processing paradigm. In Experiment 1, 45 native speakers of English listened to a series of short passages. Via a speaker manipulation, we introduced changing intra-experimental adjective order probabilities for two-adjective noun phrases embedded within the passages and investigated whether prediction error responses adapt to reflect these intra-experimental predictive contingencies. To this end, we calculated a novel measure of speaker-based, intra-experimental surprisal (“speaker-based surprisal”) as defined on a trial-by-trial basis and by clustering together adjectives with a similar meaning. N400 amplitude at the position of the critical second adjective was used as an outcome measure of prediction error. Results showed that N400 responses attuned to speaker-based surprisal over the course of the experiment, thus indicating that listeners rapidly adapt their predictive models to reflect local environmental contingencies (here: the probability of one type of adjective following another when uttered by a particular speaker). Strikingly, this occurs in spite of the wealth of prior linguistic experience that participants bring to the laboratory. Model adaptation effects were strongest for participants with a steep aperiodic (1/f) slope in resting EEG and low individual alpha frequency (IAF), with idea density (ID) showing a more complex pattern. These results were replicated in a separate sample of 40 participants in Experiment 2, which employed a highly similar design to Experiment 1. Overall, our results suggest that individuals with a steep aperiodic slope adapt their predictive models most strongly to context-specific probabilistic information. Steep aperiodic slope is thought to reflect low neural noise, which in turn may be associated with higher neural gain control and better cognitive control. Individuals with a steep aperiodic slope may thus be able to more effectively and dynamically reconfigure their prediction-related neural networks to meet current task demands. We conclude that predictive mechanisms in language are highly malleable and dynamic, reflecting both the affordances of the present environment as well as intrinsic information processing capabilities of the individual.

ERP Evidences of Rapid Semantic Learning in Foreign Language Word Comprehension

The event-related potential (ERP) of electroencephalography (EEG) signals has been well studied in the case of native language speech comprehension using semantically matched and mis-matched end-words. The presence of semantic incongruity in the audio stimulus elicits a N400 component in the ERP waveform. However, it is unclear whether the semantic dissimilarity effects in ERP also appear for foreign language words that were learned in a rapid language learning task. In this study, we introduced the semantics of Japanese words to subjects who had no prior exposure to Japanese language. Following this language learning task, we performed ERP analysis using English sentences of semantically matched and mis-matched nature where the end-words were replaced with their Japanese counterparts. The ERP analysis revealed that, even with a short learning cycle, the semantically matched and mis-matched end-words elicited different EEG patterns (similar to the native language case). However, the patterns seen for the newly learnt word stimuli showed the presence of P600 component (delayed and opposite in polarity to those seen in the known language). A topographical analysis revealed that P600 responses were pre-dominantly observed in the parietal region and in the left hemisphere. The absence of N400 component in this rapid learning task can be considered as evidence for its association with long-term memory processing. Further, the ERP waveform for the Japanese end-words, prior to semantic learning, showed a P3a component owing to the subject's reaction to a novel stimulus. These differences were more pronounced in the centro-parietal scalp electrodes.

Predictive pre-activation of orthographic and lexical-semantic representations facilitates visual word recognition

To a crucial extent, the efficiency of reading results from the fact that visual word recognition is faster in predictive contexts. Predictive coding models suggest that this facilitation results from pre-activation of predictable stimulus features across multiple representational levels before stimulus onset. Still, it is not sufficiently understood which aspects of the rich set of linguistic representations that are activated during reading-visual, orthographic, phonological, and/or lexical-semantic-contribute to context-dependent facilitation. To investigate in detail which linguistic representations are pre-activated in a predictive context and how they affect subsequent stimulus processing, we combined a well-controlled repetition priming paradigm, including words and pseudowords (i.e., pronounceable nonwords), with behavioral and magnetoencephalography measurements. For statistical analysis, we used linear mixed modeling, which we found had a higher statistical power compared to conventional multivariate pattern decoding analysis. Behavioral data from 49 participants indicate that word predictability (i.e., context present vs. absent) facilitated orthographic and lexical-semantic, but not visual or phonological processes. Magnetoencephalography data from 38 participants show sustained activation of orthographic and lexical-semantic representations in the interval before processing the predicted stimulus, suggesting selective pre-activation at multiple levels of linguistic representation as proposed by predictive coding. However, we found more robust lexical-semantic representations when processing predictable in contrast to unpredictable letter strings, and pre-activation effects mainly resembled brain responses elicited when processing the expected letter string. This finding suggests that pre-activation did not result in "explaining away" predictable stimulus features, but rather in a "sharpening" of brain responses involved in word processing.

First Language Attrition: What It Is, What It Isn't, and What It Can Be

This review aims at clarifying the concept of first language attrition by tracing its limits, identifying its phenomenological and contextual constraints, discussing controversies associated with its definition, and suggesting potential directions for future research. We start by reviewing different definitions of attrition as well as associated inconsistencies. We then discuss the underlying mechanisms of first language attrition and review available evidence supporting different background hypotheses. Finally, we attempt to provide the groundwork to build a unified theoretical framework allowing for generalizable results. To this end, we suggest the deployment of a rigorous neuroscientific approach, in search of neural markers of first language attrition in different linguistic domains, putting forward hypothetical experimental ways to identify attrition's neural traces and formulating predictions for each of the proposed experimental paradigms.

Representational Pattern Similarity of Electrical Brain Activity Reveals Rapid and Specific Prediction during Language Comprehension

Predicting upcoming events is a critical function of the brain, and language provides a fertile testing ground for studying prediction, as comprehenders use context to predict features of upcoming words. Many aspects of the mechanisms of prediction remain elusive, partly due to a lack of methodological tools to probe prediction formation in the moment. To elucidate what features are neurally preactivated and when, we used representational similarity analysis on previously collected sentence reading data. We compared EEG activity patterns elicited by expected and unexpected sentence final words to patterns from the preceding words of the sentence, in both strongly and weakly constraining sentences. Pattern similarity with the final word was increased in an early time window following the presentation of the pre-final word, and this increase was modulated by both expectancy and constraint. This was not seen at earlier words, suggesting that predictions were precisely timed. Additionally, pre-final word activity-the predicted representation-had negative similarity with later final word activity, but only for strongly expected words. These findings shed light on the mechanisms of prediction in the brain: rapid preactivation occurs following certain cues, but the predicted features may receive reduced processing upon confirmation.

Cloze enough? Hemodynamic effects of predictive processing during natural reading

Evidence accrues that readers form multiple hypotheses about upcoming words. The present study investigated the hemodynamic effects of predictive processing during natural reading by means of combining fMRI and eye movement recordings. In particular, we investigated the neural and behavioral correlates of precision-weighted prediction errors, which are thought to be indicative of subsequent belief updating. Participants silently read sentences in which we manipulated the cloze probability and the semantic congruency of the final word that served as an index for precision and prediction error respectively. With respect to the neural correlates, our findings indicate an enhanced activation within the left inferior frontal and middle temporal gyrus suggesting an effect of precision on prediction update in higher (lexico-)semantic levels. Despite being evident at the neural level, we did not observe any evidence that this mechanism resulted in disproportionate reading times on participants' eye movements. The results speak against discrete predictions, but favor the notion that multiple words are activated in parallel during reading.

Rapid acquisition of novel written word-forms: ERP evidence

Background

Novel word acquisition is generally believed to be a rapid process, essential for ensuring a flexible and efficient communication system; at least in spoken language, learners are able to construct memory traces for new linguistic stimuli after just a few exposures. However, such rapid word learning has not been systematically found in visual domain, with different confounding factors obscuring the orthographic learning of novel words. This study explored the changes in human brain activity occurring online, during a brief training with novel written word-forms using a silent reading task

Results

Single-trial, cluster-based random permutation analysis revealed that training caused an extremely fast (after just one repetition) and stable facilitation in novel word processing, reflected in the modulation of P200 and N400 components, possibly indicating rapid dynamics at early and late stages of the lexical processing. Furthermore, neural source estimation of these effects revealed the recruitment of brain areas involved in orthographic and lexico-semantic processing, respectively.

Conclusions

These results suggest the formation of neural memory traces for novel written word-forms after a minimal exposure to them even in the absence of a semantic reference, resembling the rapid learning processes known to occur in spoken language.

Catecholaminergic Modulation of Semantic Processing in Sentence Comprehension

Catecholamine (CA) function has been widely implicated in cognitive functions that are tied to the prefrontal cortex and striatal areas. The present study investigated the effects of methylphenidate, which is a CA agonist, on the electroencephalogram (EEG) response related to semantic processing using a double-blind, placebo-controlled, randomized, crossover, within-subject design. Forty-eight healthy participants read semantically congruent or incongruent sentences after receiving 20-mg methylphenidate or a placebo while their brain activity was monitored with EEG. To probe whether the catecholaminergic modulation is task-dependent, in one condition participants had to focus on comprehending the sentences, while in the other condition, they only had to attend to the font size of the sentence. The results demonstrate that methylphenidate has a task-dependent effect on semantic processing. Compared to placebo, when semantic processing was task-irrelevant, methylphenidate enhanced the detection of semantic incongruence as indexed by a larger N400 amplitude in the incongruent sentences; when semantic processing was task-relevant, methylphenidate induced a larger N400 amplitude in the semantically congruent condition, which was followed by a larger late positive complex effect. These results suggest that CA-related neurotransmitters influence language processing, possibly through the projections between the prefrontal cortex and the striatum, which contain many CA receptors.

The temporal course of vicarious embarrassment: An electrophysiological study

Vicarious embarrassment as a social pain emotion has been studied from cognitive and neuroimaging perspectives. However, the dynamic time course of vicarious embarrassment remains unclear. We conducted an event-related potential (ERP) study to investigate the temporal dynamics of vicarious embarrassment and examine the role of perspective-taking in an emotional judgment task. The ERP results showed that vicarious embarrassment evoked more negative MFN (250-310 ms) and N400 (400-520 ms) components than the neutral condition over the frontal region. The time-frequency analysis results showed that, in the 200-1,600 ms time window, greater alpha power was elicited when participants imagined themselves rather than others in an awkward situation; in the 500-1,900 ms time window, for both groups, vicarious embarrassment involved greater theta oscillations than the neutral condition over the frontal region. These results extend previous findings that vicarious emotion involves mentalizing processes, and demonstrate that people first detect and evaluate the target's misconduct and negative outcomes and then experience the vicarious embarrassment.

Neural correlates of semantic and syntactic processing in German Sign Language

The study of deaf and hearing native users of signed languages can offer unique insights into how biological constraints and environmental input interact to shape the neural bases of language processing. Here, we use functional magnetic resonance imaging (fMRI) to address two questions: (1) Do semantic and syntactic processing in a signed language rely on anatomically and functionally distinct neural substrates as it has been shown for spoken languages? and (2) Does hearing status affect the neural correlates of these two types of linguistic processing? Deaf and hearing native signers performed a sentence judgement task on German Sign Language (Deutsche Gebärdensprache: DGS) sentences which were correct or contained either syntactic or semantic violations. We hypothesized that processing of semantic and syntactic violations in DGS relies on distinct neural substrates as it has been shown for spoken languages. Moreover, we hypothesized that effects of hearing status are observed within auditory regions, as deaf native signers have been shown to activate auditory areas to a greater extent than hearing native signers when processing a signed language. Semantic processing activated low-level visual areas and the left inferior frontal gyrus (IFG), suggesting both modality-dependent and independent processing mechanisms. Syntactic processing elicited increased activation in the right supramarginal gyrus (SMG). Moreover, psychophysiological interaction (PPI) analyses revealed a cluster in left middle occipital regions showing increased functional coupling with the right SMG during syntactic relative to semantic processing, possibly indicating spatial processing mechanisms that are specific to signed syntax. Effects of hearing status were observed in the right superior temporal cortex (STC): deaf but not hearing native signers showed greater activation for semantic violations than for syntactic violations in this region. Taken together, the present findings suggest that the neural correlates of language processing are partly determined by biological constraints, but that they may additionally be influenced by the unique processing demands of the language modality and different sensory experiences.

Toward a Neurobiologically Plausible Model of Language-Related, Negative Event-Related Potentials

Language-related event-related potential (ERP) components such as the N400 have traditionally been associated with linguistic or cognitive functional interpretations. By contrast, it has been considerably more difficult to relate these components to neurobiologically grounded accounts of language. Here, we propose a theoretical framework based on a predictive coding architecture, within which negative language-related ERP components such as the N400 can be accounted for in a neurobiologically plausible manner. Specifically, we posit that the amplitude of negative language-related ERP components reflects precision-weighted prediction error signals, i.e., prediction errors weighted by the relevance of the information source leading to the error. From this perspective, precision has a direct link to cue validity in a particular language and, thereby, to relevance of individual linguistic features for internal model updating. We view components such as the N400 and LAN as members of a family with similar functional characteristics and suggest that latency and topography differences between these components reflect the locus of prediction errors and model updating within a hierarchically organized cortical predictive coding architecture. This account has the potential to unify findings from the full range of the N400 literature, including word-level, sentence-, and discourse-level results as well as cross-linguistic differences.

Dissociating predictability, plausibility and possibility of sentence continuations in reading: evidence from late-positivity ERPs

Late positive event-related potential (ERP) components occurring after the N400, traditionally linked to reanalysis due to syntactic incongruence, are increasingly considered to also reflect reanalysis and repair due to semantic difficulty. Semantic problems can have different origins, such as a mismatch of specific predictions based on the context, low plausibility, or even semantic impossibility of a word in the given context. DeLong, Quante & Kutas (2014) provided the first direct evidence for topographically different late positivities for prediction mismatch (left frontal late positivity for plausible but unexpected words) and plausibility violation (posterior-parietal late positivity for implausible, incongruent words). The aim of the current study is twofold: (1) to replicate this dissociation of ERP effects for plausibility violations and prediction mismatch in a different language, and (2) to test an additional contrast within implausible words, comparing impossible and possible sentence continuations. Our results replicate DeLong, Quante & Kutas (2014) with different materials in a different language, showing graded effects for predictability and plausibility at the level of the N400, a dissociation of plausible and implausible, anomalous continuations in posterior late positivities and an effect of prediction mismatch on late positivities at left-frontal sites. In addition, we found some evidence for a dissociation, at these left-frontal sites, between implausible words that were fully incompatible with the preceding discourse and those for which an interpretation is possible.

Lexical prediction in the aging brain: The effects of predictiveness and congruency on the N400 ERP component

Although the N400 ERP component has been extensively studied in younger adults, the underlying mechanisms are still unclear. In older adults, N400 ERP studies have shown less efficient use of context compared to younger adults. Here, we asked whether the mechanisms underlying the N400 effect are the same in terms of predictiveness and congruency in younger and older adults. We used a simple picture-word matching task in which we crossed predictiveness and congruency. Our results indicate a three-way interaction between predictiveness, congruency, and age, in that, younger adults showed an N400 effect only in strongly constrained conditions; whereas, older adults showed an effect in both strongly- and weakly constrained conditions. This interaction was not modulated by language experience or cognitive decline. Our results support either two separate mechanisms (lexical access and integration) that run in parallel and are modulated by age or a common prediction error mechanism that changes with age.

Gamma Oscillatory Activity Related to Language Prediction

Using magnetoencephalography, the current study examined gamma activity associated with language prediction. Participants read high- and low-constraining sentences in which the final word of the sentence was either expected or unexpected. Although no consistent gamma power difference induced by the sentence-final words was found between the expected and unexpected conditions, the correlation of gamma power during the prediction and activation intervals of the sentence-final words was larger when the presented words matched with the prediction compared with when the prediction was violated or when no prediction was available. This suggests that gamma magnitude relates to the match between predicted and perceived words. Moreover, the expected words induced activity with a slower gamma frequency compared with that induced by unexpected words. Overall, the current study establishes that prediction is related to gamma power correlations and a slowing of the gamma frequency.

Some inferences still take time: Prosody, predictability, and the speed of scalar implicatures

Experimental pragmatics has gained many insights from understanding how people use weak scalar terms (like some) to infer that a stronger alternative (like all) is false. Early studies found that comprehenders initially interpret some without an upper bound, but later results suggest that this inference is sometimes immediate (e.g., Grodner, Klein, Carbary, & Tanenhaus, 2010). The present paper explores whether rapid inferencing depends on the prosody (i.e., summa rather than some of) or predictability of referring expressions (e.g., consistently using some to describe subsets). Eye-tracking experiments examined looks to subsets (2-of-4 socks) and total sets (3-of-3 soccer balls) following some and found early preferences for subsets in predictable contexts but not in less predictable contexts (Experiment 1 and 2). In contrast, there was no reliable prosody effect on inferencing. Changes in predictability did not affect judgments of the naturalness of some, when a discourse context was available (Experiment 3). However, predictable contexts reduced variability in speakers' descriptions of subsets and total sets (Experiment 4). Together, these results demonstrate that scalar inferences are often delayed during comprehension, but reference restriction is rapid when set descriptions can be formulated beforehand.

Talking about SOME and ALL: What determines the usage of quantity-denoting expressions?

Reference production is often studied through single dimensions of contrast (e.g., "tall glass" when there are one or two glasses of varying height). Yet real-world communication is rarely so simple, raising questions about the factors guiding more complex referents. The current study examines decisions to mention set relations (e.g., using quantity-denoting expressions like "some of the houses" to refer to 2-out-of-5 houses) versus object categories only (e.g., using bare plurals like "houses"). Two experiments used vignettes to vary discourse focus on objects (prominent vs. non-prominent) and scenes to vary the set type described (subset vs. total set). Speakers were more likely to communicate set relations of prominent objects, particularly when they elicited high name agreement in the case of total sets. Speakers' use of quantity-denoting expressions also increased listeners' sensitivity to set relations in an object-matching task. This suggests that unlike simpler forms of modification that often decrease with greater focus, quantity-denoting expressions provide additional information about the set relations of prominent referents.

Prediction Signatures in the Brain: Semantic Pre-Activation during Language Comprehension

There is broad agreement that context-based predictions facilitate lexical-semantic processing. A robust index of semantic prediction during language comprehension is an evoked response, known as the N400, whose amplitude is modulated as a function of semantic context. However, the underlying neural mechanisms that utilize relations of the prior context and the embedded word within it are largely unknown. We measured magnetoencephalography (MEG) data while participants were listening to simple German sentences in which the verbs were either highly predictive for the occurrence of a particular noun (i.e., provided context) or not. The identical set of nouns was presented in both conditions. Hence, differences for the evoked responses of the nouns can only be due to differences in the earlier context. We observed a reduction of the N400 response for highly predicted nouns. Interestingly, the opposite pattern was observed for the preceding verbs: highly predictive (that is more informative) verbs yielded stronger neural magnitude compared to less predictive verbs. A negative correlation between the N400 effect of the verb and that of the noun was found in a distributed brain network, indicating an integral relation between the predictive power of the verb and the processing of the subsequent noun. This network consisted of left hemispheric superior and middle temporal areas and a subcortical area; the parahippocampus. Enhanced activity for highly predictive relative to less predictive verbs, likely reflects establishing semantic features associated with the expected nouns, that is a pre-activation of the expected nouns.

Sentence understanding depends on contextual use of semantic and real world knowledge

Human language allows us to express our thoughts and ideas by combining entities, concepts and actions into multi-event episodes. Yet, the functional neuroanatomy engaged in interpretation of such high-level linguistic input remains poorly understood. Here, we used easy to detect and more subtle "borderline" anomalies to investigate the brain regions and mechanistic principles involved in the use of real-world event knowledge in language comprehension. Overall, the results showed that the processing of sentences in context engages a complex set of bilateral brain regions in the frontal, temporal and inferior parietal lobes. Easy anomalies preferentially engaged lower-order cortical areas adjacent to the primary auditory cortex. In addition, the left supramarginal gyrus and anterior temporal sulcus as well as the right posterior middle temporal gyrus contributed to the processing of easy and borderline anomalies. The observed pattern of results is explained in terms of (i) hierarchical processing along a dorsal-ventral axis and (ii) the assumption of high-order association areas serving as cortical hubs in the convergence of information in a distributed network. Finally, the observed modulation of BOLD signal in prefrontal areas provides support for their role in the implementation of executive control processes.

When expectancies collide: Action dynamics reveal the interaction between stimulus plausibility and congruency

The cognitive architecture routinely relies on expectancy mechanisms to process the plausibility of stimuli and establish their sequential congruency. In two computer mouse-tracking experiments, we use a cross-modal verification task to uncover the interaction between plausibility and congruency by examining their temporal signatures of activation competition as expressed in a computer- mouse movement decision response. In this task, participants verified the content congruency of sentence and scene pairs that varied in plausibility. The order of presentation (sentence-scene, scene-sentence) was varied between participants to uncover any differential processing. Our results show that implausible but congruent stimuli triggered less accurate and slower responses than implausible and incongruent stimuli, and were associated with more complex angular mouse trajectories independent of the order of presentation. This study provides novel evidence of a disassociation between the temporal signatures of plausibility and congruency detection on decision responses.

Oscillatory Brain Dynamics during Sentence Reading: A Fixation-Related Spectral Perturbation Analysis

The present study investigated oscillatory brain dynamics during self-paced sentence-level processing. Participants read fully correct sentences, sentences containing a semantic violation and "sentences" in which the order of the words was randomized. At the target word level, fixations on semantically unrelated words elicited a lower-beta band (13-18 Hz) desynchronization. At the sentence level, gamma power (31-55 Hz) increased linearly for syntactically correct sentences, but not when the order of the words was randomized. In the 300-900 ms time window after sentence onsets, theta power (4-7 Hz) was greater for syntactically correct sentences as compared to sentences where no syntactic structure was preserved (random words condition). We interpret our results as conforming with a recently formulated predictive-coding framework for oscillatory neural dynamics during sentence-level language comprehension. Additionally, we discuss how our results relate to previous findings with serial visual presentation vs. self-paced reading.

Predicting visual information during sentence processing: Toddlers activate an object's shape before it is mentioned

We examined the contents of language-mediated prediction in toddlers by investigating the extent to which toddlers are sensitive to visual shape representations of upcoming words. Previous studies with adults suggest limits to the degree to which information about the visual form of a referent is predicted during language comprehension in low constraint sentences. Toddlers (30-month-olds) heard either contextually constraining sentences or contextually neutral sentences as they viewed images that were either identical or shape-related to the heard target label. We observed that toddlers activate shape information of upcoming linguistic input in contextually constraining semantic contexts; hearing a sentence context that was predictive of the target word activated perceptual information that subsequently influenced visual attention toward shape-related targets. Our findings suggest that visual shape is central to predictive language processing in toddlers.

What do we mean by prediction in language comprehension?

We consider several key aspects of prediction in language comprehension: its computational nature, the representational level(s) at which we predict, whether we use higher level representations to predictively pre-activate lower level representations, and whether we 'commit' in any way to our predictions, beyond pre-activation. We argue that the bulk of behavioral and neural evidence suggests that we predict probabilistically and at multiple levels and grains of representation. We also argue that we can, in principle, use higher level inferences to predictively pre-activate information at multiple lower representational levels. We also suggest that the degree and level of predictive pre-activation might be a function of the expected utility of prediction, which, in turn, may depend on comprehenders' goals and their estimates of the relative reliability of their prior knowledge and the bottom-up input. Finally, we argue that all these properties of language understanding can be naturally explained and productively explored within a multi-representational hierarchical actively generative architecture whose goal is to infer the message intended by the producer, and in which predictions play a crucial role in explaining the bottom-up input.

An fMRI study dissociating distance measures computed by Broca's area in movement processing: clause boundary vs. identity

Behavioral studies of sentence comprehension suggest that processing long-distance dependencies is subject to interference effects when Noun Phrases (NP) similar to the dependency head intervene in the dependency. Neuroimaging studies converge in localizing such effects to Broca's area, showing that activity in Broca's area increases with the number of NP interveners crossed by a moved NP of the same type. To test if NP interference effects are modulated by adding an intervening clause boundary, which should by hypothesis increase the number of successive-cyclic movements, we conducted an fMRI study contrasting NP interveners with clausal (CP) interveners. Our design thus had two components: (I) the number of NP interveners crossed by movement was parametrically modulated; (II) CP-intervention was contrasted with NP-intervention. The number of NP interveners parametrically modulated a cluster straddling left BA44/45 of Broca's area, replicating earlier studies. Adding an intervening clause boundary did not significantly modulate the size of the NP interference effect in Broca's area. Yet, such an interaction effect was observed in the Superior Frontal Gyrus (SFG). Therefore, the involvement of Broca's area in processing syntactic movement is best captured by memory mechanisms affected by a grammatically instantiated type-identity (i.e., NP) intervention.

Lexical Preactivation in Basic Linguistic Phrases

Many previous studies have shown that predictable words are read faster and lead to reduced neural activation, consistent with a model of reading in which words are activated in advance of being encountered. The nature of such preactivation, however, has typically been studied indirectly through its subsequent effect on word recognition. Here, we use magnetoencephalography to study the dynamics of prediction within serially presented adjective-noun phrases, beginning at the point at which the predictive information is first available to the reader. Using corpus transitional probability to estimate the predictability of a noun, we found an increase in activity in the left middle temporal gyrus in response to the presentation of highly predictive adjectives (i.e., adjectives that license a strong noun prediction). Moreover, we found that adjective predictivity and expected noun frequency interacted, such that the response to the highly predictive adjectives (e.g., stainless) was modulated by the frequency of the expected noun (steel). These results likely reflect preactivation of nouns in highly predictive contexts. The fact that the preactivation process was modulated by the frequency of the predicted item is argued to provide support for a frequency-sensitive lexicon.

Effects of prediction and contextual support on lexical processing: prediction takes precedence

Readers may use contextual information to anticipate and pre-activate specific lexical items during reading. However, prior studies have not clearly dissociated the effects of accurate lexical prediction from other forms of contextual facilitation such as plausibility or semantic priming. In this study, we measured electrophysiological responses to predicted and unpredicted target words in passages providing varying levels of contextual support. This method was used to isolate the neural effects of prediction from other potential contextual influences on lexical processing. While both prediction and discourse context influenced ERP amplitudes within the time range of the N400, the effects of prediction occurred much more rapidly, preceding contextual facilitation by approximately 100 ms. In addition, a frontal, post-N400 positivity (PNP) was modulated by both prediction accuracy and the overall plausibility of the preceding passage. These results suggest a unique temporal primacy for prediction in facilitating lexical access. They also suggest that the frontal PNP may index the costs of revising discourse representations following an incorrect lexical prediction.

Restrictive vs. non-restrictive composition: a magnetoencephalography study

Recent research on the brain mechanisms underlying language processing has implicated the left anterior temporal lobe (LATL) as a central region for the composition of simple phrases. Because these studies typically present their critical stimuli without contextual information, the sensitivity of LATL responses to contextual factors is unknown. In this magnetoencephalography (MEG) study, we employed a simple question-answer paradigm to manipulate whether a prenominal adjective or determiner is interpreted restrictively, i.e., as limiting the set of entities under discussion. Our results show that the LATL is sensitive to restriction, with restrictive composition eliciting higher responses than non-restrictive composition. However, this effect was only observed when the restricting element was a determiner, adjectival stimuli showing the opposite pattern, which we hypothesise to be driven by the special pragmatic properties of non-restrictive adjectives. Overall, our results demonstrate a robust sensitivity of the LATL to high level contextual and potentially also pragmatic factors.

Predicting language: MEG evidence for lexical preactivation

It is widely assumed that prediction plays a substantial role in language processing. However, despite numerous studies demonstrating that contextual information facilitates both syntactic and lexical-semantic processing, there exists no direct evidence pertaining to the neural correlates of the prediction process itself. Using magnetoencephalography (MEG), this study found that brain activity was modulated by whether or not a specific noun could be predicted, given a picture prime. Specifically, before the noun was presented, predictive contexts triggered enhanced activation in left mid-temporal cortex (implicated in lexical access), ventro-medial prefrontal cortex (previously associated with top-down processing), and visual cortex (hypothesized to index the preactivation of predicted form features), successively. This finding suggests that predictive language processing recruits a top-down network where predicted words are activated at different levels of representation, from more 'abstract' lexical-semantic representations in temporal cortex, all the way down to visual word form features. The same brain regions that exhibited enhanced activation for predictive contexts before the onset of the noun showed effects of congruence during the target word. To our knowledge, this study is one of the first to directly investigate the anticipatory stage of predictive language processing.

Lexical prediction via forward models: N400 evidence from German Sign Language

Models of language processing in the human brain often emphasize the prediction of upcoming input-for example in order to explain the rapidity of language understanding. However, the precise mechanisms of prediction are still poorly understood. Forward models, which draw upon the language production system to set up expectations during comprehension, provide a promising approach in this regard. Here, we present an event-related potential (ERP) study on German Sign Language (DGS) which tested the hypotheses of a forward model perspective on prediction. Sign languages involve relatively long transition phases between one sign and the next, which should be anticipated as part of a forward model-based prediction even though they are semantically empty. Native speakers of DGS watched videos of naturally signed DGS sentences which either ended with an expected or a (semantically) unexpected sign. Unexpected signs engendered a biphasic N400-late positivity pattern. Crucially, N400 onset preceded critical sign onset and was thus clearly elicited by properties of the transition phase. The comprehension system thereby clearly anticipated modality-specific information about the realization of the predicted semantic item. These results provide strong converging support for the application of forward models in language comprehension.

An integrated theory of language production and comprehension

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal.

Good, bad and ugly word stress--fMRI evidence for foot structure driven processing of prosodic violations

Using a stress violation paradigm, we investigated whether metrical feet constrain the way prosodic patterns are processed and evaluated. Processing of correctly versus incorrectly stressed words was associated with activation in left posterior angular and retrosplenial cortex, indicating the recognition of an expected and familiar pattern, whereas the inverse contrast yielded enhanced bilateral activation in the superior temporal gyrus, reflecting higher costs in auditory (re-)analysis. More fine-grained analyses of severe versus mild stress violations revealed activations of the left superior temporal and left anterior angular gyrus whereas the opposite contrast led to frontal activations including Broca's area and its right-hemisphere homologue, suggesting that detection of mild violations lead to increased effort in working memory and deeper phonological processing. Our results provide first evidence that different incorrect stress patterns are processed in a qualitatively different way and that the underlying foot structure seems to determine potential stress positions in German words.

Long-range neural synchronization supports fast and efficient reading: EEG correlates of processing expected words in sentences

Word reading is heavily influenced by the information provided by previous context. In this study, we analyzed the neurophysiological bases of sentence reading through the EEG activity elicited during reading the same word embedded in differently constraining contexts: a) a low-constraining context; b) a high-constraining semantic compositional context; c) a high-constraining collocational context in which the item was in final position of a multi-word fixed-order expression. Cloze-probability of the two high-constraining contexts was equated. Before reading the target word we observed increased EEG gamma phase synchronization for the high-constraining compositional context and increased EEG theta synchronization for the collocational context (both compared to the low-constraining condition). After reading the target word we observed increased frontal positive EEG evoked activity (~220ms) for the high-constraining compositional context but an even earlier (~120ms) effect for the high-constraining collocational condition that was distributed over the scalp. A positive correlation was found only between the increased theta synchronization and the early EEG effect for the high-constraining collocational condition. Results indicate that long-range frontal-occipital interactions in the theta band - indexing working memory operations - support early visual-orthographic analysis of an incoming stimulus (such as the expected word); gamma-phase synchronization better represents binding operations between feed-forward activation and matching feedback. These data suggest that internal linguistic knowledge stored in long-term memory - if unambiguously pre-activated - supports the low-level perceptual processes involved in reading.

The maps problem and the mapping problem: two challenges for a cognitive neuroscience of speech and language

Research on the brain basis of speech and language faces theoretical and empirical challenges. Most current research, dominated by imaging, deficit-lesion, and electrophysiological techniques, seeks to identify regions that underpin aspects of language processing such as phonology, syntax, or semantics. The emphasis lies on localization and spatial characterization of function. The first part of the paper deals with a practical challenge that arises in the context of such a research programme. This maps problem concerns the extent to which spatial information and localization can satisfy the explanatory needs for perception and cognition. Several areas of investigation exemplify how the neural basis of speech and language is discussed in those terms (regions, streams, hemispheres, networks). The second part of the paper turns to a more troublesome challenge, namely how to formulate the formal links between neurobiology and cognition. This principled problem thus addresses the relation between the primitives of cognition (here speech, language) and neurobiology. Dealing with this mapping problem invites the development of linking hypotheses between the domains. The cognitive sciences provide granular, theoretically motivated claims about the structure of various domains (the "cognome"); neurobiology, similarly, provides a list of the available neural structures. However, explanatory connections will require crafting of computationally explicit linking hypotheses at the right level of abstraction. For both the practical maps problem and the principled mapping problem, developmental approaches and evidence can play a central role in the resolution.

Rapid Interactions between Lexical Semantic and Word Form Analysis during Word Recognition in Context: Evidence from ERPs

We used ERPs to investigate the time course of interactions between lexical semantic and sublexical visual word form processing during word recognition. Participants read sentence-embedded pseudowords that orthographically resembled a contextually supported real word (e.g., “She measured the flour so she could bake a ceke…”) or did not (e.g., “She measured the flour so she could bake a tont…”) along with nonword consonant strings (e.g., “She measured the flour so she could bake a srdt…”). Pseudowords that resembled a contextually supported real word (“ceke”) elicited an enhanced positivity at 130 msec (P130), relative to real words (e.g., “She measured the flour so she could bake a cake…”). Pseudowords that did not resemble a plausible real word (“tont”) enhanced the N170 component, as did nonword consonant strings (“srdt”). The effect pattern shows that the visual word recognition system is, perhaps, counterintuitively, more rapidly sensitive to minor than to flagrant deviations from contextually predicted inputs. The findings are consistent with rapid interactions between lexical and sublexical representations during word recognition, in which rapid lexical access of a contextually supported word (CAKE) provides top–down excitation of form features (“cake”), highlighting the anomaly of an unexpected word “ceke.”

The time-course and spatial distribution of brain activity associated with sentence processing

Sentence comprehension involves a host of highly interrelated processes, including syntactic parsing, semantic composition, and pragmatic inferencing. In neuroimaging, a primary paradigm for examining the brain bases of sentence processing has been to compare brain activity elicited by sentences versus unstructured lists of words. These studies commonly find an effect of increased activity for sentences in the anterior temporal lobes (aTL). Together with neuropsychological data, these findings have motivated the hypothesis that the aTL is engaged in sentence level combinatorics. Combinatoric processing during language comprehension, however, occurs within tens and hundreds of milliseconds, i.e., at a time-scale much faster than the temporal resolution of hemodynamic measures. Here, we examined the time-course of sentence-level processing using magnetoencephalography (MEG) to better understand the temporal profile of activation in this common paradigm and to test a key prediction of the combinatoric hypothesis: because sentences are interpreted incrementally, word-by-word, activity associated with basic linguistic combinatorics should be time-locked to word-presentation. Our results reveal increased anterior temporal activity for sentences compared to word lists beginning approximately 250 ms after word onset. We also observed increased activation in a network of other brain areas, extending across posterior temporal, inferior frontal, and ventral medial areas. These findings confirm a key prediction of the combinatoric hypothesis for the aTL and further elucidate the spatio-temporal characteristics of sentence-level computations in the brain.