Understanding words in context: A naturalistic EEG study of children's lexical processing

When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.

The concrete processing of Chinese action metaphors: an ERP study

The present research adopts ERP (Event-Related Potentials) technology to investigate whether there exists a concreteness effect in the processing of Chinese action verbs within metaphorical context. The mean amplitudes of N400 activated by action metaphors were compared with those activated by literal verbs and abstract verbs. The findings indicated that the Met verbs evoked a significantly larger N400 response at frontal brain region compared to the Abs verbs at a time window 200-500 ms, while the Met verbs elicited a notably greater N400 amplitude specifically at the posterior brain region in comparison to the Lit verbs at 300-500 ms time window. These results may be interpreted as indicating that the comprehension of the Met verbs is based on the concrete action semantics.

Predicting phonological information in language comprehension: evidence from ERP representational similarity analysis and Chinese idioms

Do comprehenders predict the meaning and even the phonological form of upcoming words during language comprehension? With a growing body of evidence suggesting that semantic representations may be predicted, the evidence for phonological prediction is less clear and largely derived from studies conducted in languages utilizing an alphabetic script. In this research, we aim to examine the prediction of phonological information in the processing of Chinese idioms through the use of ERP representational similarity analysis (RSA). The study utilizes four-character Chinese idioms, and phonological overlap was manipulated by varying the syllable at the idiom-final part between idiom pairs so that pairs of idioms share a syllable (i.e. within-pairs) or not (between-pairs). We quantified the similarity between patterns of neural activity of idioms for within- and between-pairs. RSA results revealed greater similarity in neural activity patterns for idioms within-pairs, compared with between-pairs, and critically this similarity effect was observed prior to the presentation of the phonological similarity, providing evidence for the pre-activation of upcoming phonological information, under circumstances that encourage predictive processing.

Predicting coarse-grained semantic features in language comprehension: evidence from ERP representational similarity analysis and Chinese classifier

Existing studies demonstrate that comprehenders can predict semantic information during language comprehension. Most evidence comes from a highly constraining context, in which a specific word is likely to be predicted. One question that has been investigated less is whether prediction can occur when prior context is less constraining for predicting specific words. Here, we aim to address this issue by examining the prediction of animacy features in low-constraining context, using electroencephalography (EEG), in combination with representational similarity analysis (RSA). In Chinese, a classifier follows a numeral and precedes a noun, and classifiers constrain animacy features of upcoming nouns. In the task, native Chinese Mandarin speakers were presented with either animate-constraining or inanimate-constraining classifiers followed by congruent or incongruent nouns. EEG amplitude analysis revealed an N400 effect for incongruent conditions, reflecting the difficulty of semantic integration when an incompatible noun is encountered. Critically, we quantified the similarity between patterns of neural activity following the classifiers. RSA results revealed that the similarity between patterns of neural activity following animate-constraining classifiers was greater than following inanimate-constraining classifiers, before the presentation of the nouns, reflecting pre-activation of animacy features of nouns. These findings provide evidence for the prediction of coarse-grained semantic feature of upcoming words.

Age-related differences in expectation-based novel word learning

Adult language users can infer the meaning of a previously unfamiliar word from a single exposure to this word in a semantically and thematically constrained context, henceforth, predictive context (Borovsky et al., 2010 Cognition, 116(2), 289-296; Borovsky et al., 2012 Language Learning and Development, 8(3), 278-302). Children use predictive contexts to anticipate upcoming stimuli (Borovsky et al., 2012 Language Learning and Development, 8(3), 278-302; Mani & Huettig, 2012 Journal of Experimental Psychology: Human Perception and Performance, 38(4), 843-847), but the extent to which they rely on prediction to learn novel word forms is unclear (Gambi et al., 2021 Cognition, 211, 104650). Here, we examine children's one-shot learning from predictive contexts using a modified version of the one-shot learning ERP paradigm for children aged 7-13 years. In a first learning phase, we presented audio recordings of expected words and unexpected novel pseudowords in strongly and weakly constraining sentence contexts. In the following priming phase, the same recorded words and pseudowords were used as primes to identical/synonymous, related, and unrelated target words. We measured N400 modulations to the word and pseudoword continuations in the learning phase and to the identical/synonymous, related, or unrelated target words in the priming phase. When initially presented in strongly constraining sentences, novel pseudowords primed synonymous targets equally well as word primes of the same intended meaning. This pattern was particularly pronounced in older children. Our findings suggest that, around early adolescence, children can use single exposures to constraining contexts to infer the meaning of novel words and to integrate these novel words in their lexicons.

Semantic word integration in children with cochlear implants: Electrophysiological evidence

Differential auditory experiences of children with hearing-loss who receive cochlear implants (CIs) may influence the integration of lexical and conceptual information. Here we measured event-related potentials during a word-picture priming task in CI-using children (n = 29, mean age = 81 months) and typically-hearing children (n = 19, mean age = 75 months) while they viewed audiovisual-word primes and picture targets that were semantically congruent or incongruent. In both groups, semantic relatedness modulated ERP amplitude 300-500ms after picture onset, signifying an N400 semantic effect. Critically, the CI-using children's responses to unrelated pairs were significantly more negative than hearing children's responses. Group differences were mirrored in an earlier 150-275ms time window associated with a P2 response. The present findings suggest attentional and/or strategic differences impact semantic processing and contribute to the N400 differences observed between groups.

Representational Pattern Similarity of Electrical Brain Activity Reveals Rapid and Specific Prediction during Language Comprehension

Predicting upcoming events is a critical function of the brain, and language provides a fertile testing ground for studying prediction, as comprehenders use context to predict features of upcoming words. Many aspects of the mechanisms of prediction remain elusive, partly due to a lack of methodological tools to probe prediction formation in the moment. To elucidate what features are neurally preactivated and when, we used representational similarity analysis on previously collected sentence reading data. We compared EEG activity patterns elicited by expected and unexpected sentence final words to patterns from the preceding words of the sentence, in both strongly and weakly constraining sentences. Pattern similarity with the final word was increased in an early time window following the presentation of the pre-final word, and this increase was modulated by both expectancy and constraint. This was not seen at earlier words, suggesting that predictions were precisely timed. Additionally, pre-final word activity-the predicted representation-had negative similarity with later final word activity, but only for strongly expected words. These findings shed light on the mechanisms of prediction in the brain: rapid preactivation occurs following certain cues, but the predicted features may receive reduced processing upon confirmation.

Unexpected words or unexpected languages? Two ERP effects of code-switching in naturalistic discourse

Bilingual speakers often switch between languages in conversation without any advance notice. Psycholinguistic research has found that these language shifts (or code-switches) can be costly for comprehenders in certain situations. The present study explores the nature of these costs by comparing code-switches to other types of unexpected linguistic material. To do this, we used a novel EEG paradigm, the Storytime task, in which we record readings of natural texts, and then experimentally manipulate their properties by splicing in words. In this study, we manipulated the language of our target words (English, Spanish) and their fit with the preceding context (strong-fit, weak-fit). If code-switching incurs a unique cost beyond that incurred by an unexpected word, then we should see an additive pattern in our ERP indices. If an effect is driven by lexical expectation alone, then there should be a non-additive interaction such that all unexpected forms incur a similar cost. We found three effects: a general prediction effect (a non-additive N400), a post-lexical recognition of the switch in languages (an LPC for code-switched words), and a prolonged integration difficulty associated with weak-fitting words regardless of language (a sustained negativity). We interpret these findings as suggesting that the processing difficulties experienced by bilinguals can largely be understood within more general frameworks for understanding language comprehension. Our findings are consistent with the broader literature demonstrating that bilinguals do not have two wholly separate language systems but rather a single language system capable of using two coding systems.

Everyday Language Exposure Shapes Prediction of Specific Words in Listening Comprehension: A Visual World Eye-Tracking Study

We investigated the effects of everyday language exposure on the prediction of orthographic and phonological forms of a highly predictable word during listening comprehension. Native Japanese speakers in Tokyo (Experiment 1) and Berlin (Experiment 2) listened to sentences that contained a predictable word and viewed four objects. The critical object represented the target word (e.g., /sakana/; fish), an orthographic competitor (e.g., /tuno/; horn), a phonological competitor (e.g., /sakura/; cherry blossom), or an unrelated word (e.g., /hon/; book). The three other objects were distractors. The Tokyo group fixated the target and the orthographic competitor over the unrelated objects before the target word was mentioned, suggesting that they pre-activated the orthographic form of the target word. The Berlin group showed a weaker bias toward the target than the Tokyo group, and they showed a tendency to fixate the orthographic competitor only when the orthographic similarity was very high. Thus, prediction effects were weaker in the Berlin group than in the Tokyo group. We found no evidence for the prediction of phonological information. The obtained group differences support probabilistic models of prediction, which regard the built-up language experience as a basis of prediction.

Testing limits: ERP evidence for word form preactivation during speeded sentence reading

While it is commonly agreed upon that language comprehenders preactivate information at multiple levels, there is less consensus regarding what and when information is predicted, under which circumstances, and via which mechanism(s). Regarding when, Ito, Corley, Pickering, Martin, & Nieuwland (2016) concluded that during sentence processing, word form-unlike semantic-preactivation crucially relies on the time available to generate late-stage predictions via language production mechanisms, setting this limit between 500 and 700 ms/word. The current event-related brain potential (ERP) study tests this proposal at a substantially faster serial visual presentation (SVP) rate of four words/s, on par with normal reading. We utilize the experimental design and replicate the general ERP findings of the two words/s SVP study of DeLong, Chan, & Kutas (2019), with results showing similar N400 reductions to unpredictable sentence continuations whether semantically or orthographically related to contextually predictable words, as well as an increased late posterior positivity to orthographic neighbors. These findings indicate that processing of written word information can be rapidly facilitated through context-based expectancies, establishing that if there is a time constraint for word form preactivation, it must be far less than limits specified by the prediction-by-production comprehension model championed by Ito et al.

Spatiotemporal properties of the neural representation of conceptual content for words and pictures - an MEG study

The entwined nature of perceptual and conceptual processes renders an understanding of the interplay between perceptual recognition and conceptual access a continuing challenge. Here, to disentangle perceptual and conceptual processing in the brain, we combine magnetoencephalography (MEG), picture and word presentation and representational similarity analysis (RSA). We replicate previous findings of early and robust sensitivity to semantic distances between objects presented as pictures and show earlier (~105 ​msec), but not later, representations can be accounted for by contemporary computer models of visual similarity (AlexNet). Conceptual content for word stimuli is reliably present in two temporal clusters, the first ranging from 230 to 335 ​msec, the second from 360 to ​585 msec. The time-course of picture induced semantic content and the spatial location of conceptual representation were highly convergent, and the spatial distribution of both differed from that of words. While this may reflect differences in picture and word induced conceptual access, this underscores potential confounds in visual perceptual and conceptual processing. On the other hand, using the stringent criterion that neural and conceptual spaces must align, the robust representation of semantic content by 230-240 msec for visually unconfounded word stimuli significantly advances estimates of the timeline of semantic access and its orthographic and lexical precursors.

The mental representation and social aspect of expressives

Despite increased focus on emotional language, research lacks for the most emotional language: Swearing. We used event-related potentials (ERPs) to investigate whether swear words have content distinct from function words, and if so, whether this content is emotional or social in nature. Stimuli included swear (e.g. shit, damn), negative but non-swear (e.g. kill, sick), open-class neutral (e.g. wood, lend), and closed-class neutral words (e.g. while, whom). Behaviourally, swears were recognised slower than valence- and arousal- matched negative words, meaning that there is more to the expressive dimension than merely a heightened emotional state. In ERPs, both swears and negative words elicited a larger positivity (250-550 ms) than open-class neutral words. Later, swears elicited a larger late positivity (550-750 ms) than negative words. We associate the earlier positivity effect with attention due to negative valence, and the later positivity effect with pragmatics due to social tabooness. Our findings suggest a view in which expressives are not merely function words or emotional words. Rather, expressives are emotionally and socially significant. Swears are more than what is indicated by valence ore arousal alone.

(Early) context effects on event-related potentials over natural inputs

Language understanding requires the integration of the input with preceding context. Event-related potentials (ERPs) have contributed significantly to our understanding of what contextual information is accessed and when. Much of this research has, however, been limited to experimenter-designed stimuli with highly atypical lexical and context statistics. This raises questions about the extent to which previous findings generalize to everyday language processing of natural stimuli with typical linguistic statistics. We ask whether context can affect ERPs over natural stimuli early, before the N400 time window. We re-analyzed a data set of ERPs over ~700 visually presented content words in sentences from English novels. To increase power, we employed linear mixed effects regression simultaneously modeling random variance by subject and by item. To reduce concerns about Type I error inflation common to any type of time series analysis, we introduced a simple approach to model and discount auto-correlations at multiple, empirically determined, time lags. We compared this approach to Bonferroni correction. Planned follow-up analyses used Generalized Additive Mixture Models to assess the linearity of contextual effects, including lexical surprisal, found within the N400 time window. We found that contextual information affects ERPs in both early (~200ms after word onset) and late (N400) time windows, supporting a cascading, interactive account of lexical access.

Concrete processing of action metaphors: Evidence from ERP

The timing of sensory-motor activation during the comprehension of action verbs used in a metaphorical sense is not well understood. In the present Event Related Potential (ERP) study, participants read verbs in metaphoric (The church bent the rules), literal-concrete (The bodyguard bent the rod), and literal-abstract (The church altered the rules) conditions. The literal concreteness effect, obtained by subtracting the abstract from the concrete, was revealed as an N400, frontally distributed. A metaphoric effect, obtained in the metaphor-abstract contrast, was a widespread N400, and included the frontal response seen in the literal concreteness effect. Another metaphoric effect, obtained in the metaphor-concrete contrast, was a posterior N400. Further time window analyses showed that the literal concreteness effect primarily came from 200 to 300 ms, the metaphoric-concrete effect primarily came from 200 to 400 ms, and the metaphoric-abstract effect was significant throughout 200-500 ms. These results suggest that a concrete but underspecified meaning consistent with metaphoric and literal readings, was activated early and was sustained throughout the 200-500 ms window. We concluded that the metaphoric sense is based in concrete action semantics, even if these senses are underspecified.

Context-Based Facilitation in Visual Word Recognition: Evidence for Visual and Lexical But Not Pre-Lexical Contributions

Word familiarity and predictive context facilitate visual word processing, leading to faster recognition times and reduced neuronal responses. Previously, models with and without top-down connections, including lexical-semantic, pre-lexical (e.g., orthographic/phonological), and visual processing levels were successful in accounting for these facilitation effects. Here we systematically assessed context-based facilitation with a repetition priming task and explicitly dissociated pre-lexical and lexical processing levels using a pseudoword (PW) familiarization procedure. Experiment 1 investigated the temporal dynamics of neuronal facilitation effects with magnetoencephalography (MEG; N = 38 human participants), while experiment 2 assessed behavioral facilitation effects (N = 24 human participants). Across all stimulus conditions, MEG demonstrated context-based facilitation across multiple time windows starting at 100 ms, in occipital brain areas. This finding indicates context-based facilitation at an early visual processing level. In both experiments, we furthermore found an interaction of context and lexical familiarity, such that stimuli with associated meaning showed the strongest context-dependent facilitation in brain activation and behavior. Using MEG, this facilitation effect could be localized to the left anterior temporal lobe at around 400 ms, indicating within-level (i.e., exclusively lexical-semantic) facilitation but no top-down effects on earlier processing stages. Increased pre-lexical familiarity (in PWs familiarized utilizing training) did not enhance or reduce context effects significantly. We conclude that context-based facilitation is achieved within visual and lexical processing levels. Finally, by testing alternative hypotheses derived from mechanistic accounts of repetition suppression, we suggest that the facilitatory context effects found here are implemented using a predictive coding mechanism.

Prediction of orthographic information during listening comprehension: A printed-word visual world study

Two visual world eye-tracking experiments examined the role of orthographic information in the visual context in pre-activation of orthographic and phonological information using Japanese. Participants heard sentences that contained a predictable target word and viewed a display showing four words in a logogram, kanji (Experiment 1) or in a phonogram, hiragana (Experiment 2). The four words included either the target word (e.g., /sakana/; fish), an orthographic competitor (e.g., /tuno/; horn), a phonological competitor (e.g., /sakura/; cherry blossom), or an unrelated word (e.g., /hon/; book), together with three distractor words. The orthographic competitor was orthographically or phonologically dissimilar to the target in hiragana. In Experiment 1, target and orthographic competitor words attracted more fixations than unrelated words before the target word was mentioned, suggesting that participants pre-activated orthographic form of the target word. In Experiment 2, target and phonological competitor words attracted more predictive fixations than unrelated words, but orthographic competitor words did not, suggesting a critical role of the visual context. This pre-activation pattern does not fit with the pattern of lexical activation in auditory word recognition, where orthography and phonology interact. However, it is compatible with the pattern of lexical activation in spoken word production, where orthographic information is not automatically activated, in line with production-based prediction accounts.

Similar time courses for word form and meaning preactivation during sentence comprehension

Current psycholinguistic research generally acknowledges that aspects of sentence comprehension benefit from neural preactivation of different types of information. However, despite strong support from a number of studies, routine specific word form preactivation has been challenged by Ito, Corley, Pickering, Martin, and Nieuwland (2016). They suggest that word form prediction is contingent upon having enough processing time and resources (afforded by slower input rates) to progress through unidirectional, productionlike stages of comprehension to arrive at word forms via semantic feature preactivation. This conclusion is based on findings from their ERP study, which used a related anomaly paradigm and reported form preactivation at a slow (700 ms) word presentation rate but not a faster one (500 ms). The present experimental design is a conceptual replication of Ito et al. (2016), testing young adults by measuring ERP amplitudes to unpredictable words related either semantically/associatively or orthographically to predictable sentence continuations, relative to unrelated continuations. Results showed that, at a visual presentation rate of two words per second, both types of related words show similarly reduced N400s, as well as varying degrees of increased posterior post-N400 positivity. These findings indicate that word form preactivation during sentence comprehension is detectable along a similar time course as semantic feature preactivation, and such processing does not necessarily require additional time beyond that afforded by near-normal reading rates.

Compounding matters: Event-related potential evidence for early semantic access to compound words

Reading words in a native language triggers a largely obligatory cognitive process that we accept as leading to comprehension of the word - we cannot suppress our understanding of word meaning. In this study, we investigated the early stages of this comprehension process by means of event-related potentials (ERPs) to identify when this processing of meaning - that is, semantic processing - first occurs. We report that, when processing visually presented compound words, semantic access at some level occurs as early as the P100 and persists through to the N400. Specifically, we focused on the P100 ERP component, and utilized the unique features of compound words (i.e. variation in the transparency of meaning) to investigate the speed with which we gain access to information about meaning (i.e. semantic access). Twenty-two participants performed a lexical decision task including 40 English compounds, which varied with respect to their constituent semantic transparency. Compounds ranged from full constituent semantic transparency (e.g. grapeseed) to partial transparency (e.g. grapefruit) to full opacity (e.g. hogwash). Regression analyses predicted ERP components from compound constituent transparency, adjusting for word frequency. Word frequency and transparency of both the first and second constituents each uniquely predicted P100 amplitude. Transparency of the second constituent, but not word frequency, predicted later component amplitudes, including that of the N400. The findings suggest that some level of semantic access occurs as early as the P100. Overall, these results support models which emphasize simultaneous processing of form and meaning as opposed to serial or hierarchical approaches.

Gamma Oscillatory Activity Related to Language Prediction

Using magnetoencephalography, the current study examined gamma activity associated with language prediction. Participants read high- and low-constraining sentences in which the final word of the sentence was either expected or unexpected. Although no consistent gamma power difference induced by the sentence-final words was found between the expected and unexpected conditions, the correlation of gamma power during the prediction and activation intervals of the sentence-final words was larger when the presented words matched with the prediction compared with when the prediction was violated or when no prediction was available. This suggests that gamma magnitude relates to the match between predicted and perceived words. Moreover, the expected words induced activity with a slower gamma frequency compared with that induced by unexpected words. Overall, the current study establishes that prediction is related to gamma power correlations and a slowing of the gamma frequency.

Theta oscillations mediate pre-activation of highly expected word initial phonemes

Prediction has been proposed to be a fundamental neurocognitive mechanism. However, its role in language comprehension is currently under debate. In this magnetoencephalography study we aimed to find evidence of word-form phonological pre-activation and to characterize the oscillatory mechanisms supporting this. Participants were presented firstly with a picture of an object, and then, after a delay (fixed or variable), they heard the corresponding word. Target words could contain a phoneme substitution, and participants’ task was to detect mispronunciations. Word-initial phonemes were either fricatives or plosives, generating two experimental conditions (expect-fricative and expect-plosive). In the pre-word interval, significant differences (α = 0.05) emerged between conditions both for fixed and variable delays. Source reconstruction of this effect showed a brain-wide network involving several frequency bands, including bilateral superior temporal areas commonly associated with phonological processing, in a theta range. These results show that phonological representations supported by the theta band may be active before word onset, even under temporal uncertainty. However, in the evoked response just prior to the word, differences between conditions were apparent under variable- but not fixed-delays. This suggests that additional top-down mechanisms sensitive to phonological form may be recruited when there is uncertainty in the signal.

Some inferences still take time: Prosody, predictability, and the speed of scalar implicatures

Experimental pragmatics has gained many insights from understanding how people use weak scalar terms (like some) to infer that a stronger alternative (like all) is false. Early studies found that comprehenders initially interpret some without an upper bound, but later results suggest that this inference is sometimes immediate (e.g., Grodner, Klein, Carbary, & Tanenhaus, 2010). The present paper explores whether rapid inferencing depends on the prosody (i.e., summa rather than some of) or predictability of referring expressions (e.g., consistently using some to describe subsets). Eye-tracking experiments examined looks to subsets (2-of-4 socks) and total sets (3-of-3 soccer balls) following some and found early preferences for subsets in predictable contexts but not in less predictable contexts (Experiment 1 and 2). In contrast, there was no reliable prosody effect on inferencing. Changes in predictability did not affect judgments of the naturalness of some, when a discourse context was available (Experiment 3). However, predictable contexts reduced variability in speakers' descriptions of subsets and total sets (Experiment 4). Together, these results demonstrate that scalar inferences are often delayed during comprehension, but reference restriction is rapid when set descriptions can be formulated beforehand.

Representational similarity analysis reveals task-dependent semantic influence of the visual word form area

Access to semantic information of visual word forms is a key component of reading comprehension. In this study, we examined the involvement of the visual word form area (VWFA) in this process by investigating whether and how the activity patterns of the VWFA are influenced by semantic information during semantic tasks. We asked participants to perform two semantic tasks - taxonomic or thematic categorization - on visual words while obtaining the blood-oxygen-level-dependent (BOLD) fMRI responses to each word. Representational similarity analysis with four types of semantic relations (taxonomic, thematic, subjective semantic rating and word2vec) revealed that neural activity patterns of the VWFA were associated with taxonomic information only in the taxonomic task, with thematic information only in the thematic task and with the composite semantic information measured by word2vec in both semantic tasks. Furthermore, the semantic information in the VWFA cannot be explained by confounding factors including orthographic, low-level visual and phonological information. These findings provide positive evidence for the presence of both orthographic and task-relevant semantic information in the VWFA and have significant implications for the neurobiological basis of reading.

Language Prediction Is Reflected by Coupling between Frontal Gamma and Posterior Alpha Oscillations

Readers and listeners actively predict upcoming words during language processing. These predictions might serve to support the unification of incoming words into sentence context and thus rely on interactions between areas in the language network. In the current magnetoencephalography study, participants read sentences that varied in contextual constraints so that the predictability of the sentence-final words was either high or low. Before the sentence-final words, we observed stronger alpha power suppression for the highly compared with low constraining sentences in the left inferior frontal cortex, left posterior temporal region, and visual word form area. Importantly, the temporal and visual word form area alpha power correlated negatively with left frontal gamma power for the highly constraining sentences. We suggest that the correlation between alpha power decrease in temporal language areas and left prefrontal gamma power reflects the initiation of an anticipatory unification process in the language network.

Predicting semantic features in Chinese: Evidence from ERPs

This article reports two ERP studies that exploited the classifier system of Mandarin Chinese to investigate semantic prediction. In Mandarin, in certain contexts, a noun has to be preceded by a classifier, which has to match the noun in semantically-defined features. In both experiments, an N400 effect was elicited in response to a classifier that mismatched an up-coming predictable noun, relative to a matching classifier. Among the mismatching classifiers, the N400 effect was graded, being smaller for classifiers that were semantically related to the predicted word, relative to classifiers that were semantically unrelated to the predicted word. Given that the classifier occurred before the predicted word, this result shows that fine-grained semantic features of nouns can be pre-activated in advance of bottom-up input. The studies thus extend previous findings based on a more restricted range of highly grammaticalized features such as gender or animacy in Indo-European languages (Szewczyk & Schriefers, 2013; Van Berkum, Brown, Zwitserlood, Kooijman, & Hagoort, 2005; Wicha, Bates, Moreno, & Kutas, 2003).

Hierarchical levels of representation in language prediction: The influence of first language acquisition in highly proficient bilinguals

Language comprehension is largely supported by predictive mechanisms that account for the ease and speed with which communication unfolds. Both native and proficient non-native speakers can efficiently handle contextual cues to generate reliable linguistic expectations. However, the link between the variability of the linguistic background of the speaker and the hierarchical format of the representations predicted is still not clear. We here investigate whether native language exposure to typologically highly diverse languages (Spanish and Basque) affects the way early balanced bilingual speakers carry out language predictions. During Spanish sentence comprehension, participants developed predictions of words the form of which (noun ending) could be either diagnostic of grammatical gender values (transparent) or totally ambiguous (opaque). We measured electrophysiological prediction effects time-locked both to the target word and to its determiner, with the former being expected or unexpected. Event-related (N200-N400) and oscillatory activity in the low beta-band (15-17Hz) frequency channel showed that both Spanish and Basque natives optimally carry out lexical predictions independently of word transparency. Crucially, in contrast to Spanish natives, Basque natives displayed visual word form predictions for transparent words, in consistency with the relevance that noun endings (post-nominal suffixes) play in their native language. We conclude that early language exposure largely shapes prediction mechanisms, so that bilinguals reading in their second language rely on the distributional regularities that are highly relevant in their first language. More importantly, we show that individual linguistic experience hierarchically modulates the format of the predicted representation.

What draws the line between perception and cognition?

The investigation of top-down effects on perception requires a rigorous definition of what qualifies as perceptual to begin with. Whereas Firestone & Scholl's (F&S's) phenomenological demarcation of perception from cognition appeals to intuition, we argue that the dividing line is best attained at the functional level. We exemplify how this approach facilitates scrutinizing putative interactions between judging and perceiving.

Neural activity during sentence processing as reflected in theta, alpha, beta, and gamma oscillations

We used magnetoencephalography (MEG) to explore the spatiotemporal dynamics of neural oscillations associated with sentence processing in 102 participants. We quantified changes in oscillatory power as the sentence unfolded, and in response to individual words in the sentence. For words early in a sentence compared to those late in the same sentence, we observed differences in left temporal and frontal areas, and bilateral frontal and right parietal regions for the theta, alpha, and beta frequency bands. The neural response to words in a sentence differed from the response to words in scrambled sentences in left-lateralized theta, alpha, beta, and gamma. The theta band effects suggest that a sentential context facilitates lexical retrieval, and that this facilitation is stronger for words late in the sentence. Effects in the alpha and beta bands may reflect the unification of semantic and syntactic information, and are suggestive of easier unification late in a sentence. The gamma oscillations are indicative of predicting the upcoming word during sentence processing. In conclusion, changes in oscillatory neuronal activity capture aspects of sentence processing. Our results support earlier claims that language (sentence) processing recruits areas distributed across both hemispheres, and extends beyond the classical language regions.

Prediction in the processing of repair disfluencies: Evidence from the visual-world paradigm

Two visual-world eye-tracking experiments investigated the role of prediction in the processing of repair disfluencies (e.g., "The chef reached for some salt uh I mean some ketchup . . ."). Experiment 1 showed that listeners were more likely to fixate a critical distractor item (e.g., pepper) during the processing of repair disfluencies compared with the processing of coordination structures (e.g., ". . . some salt and also some ketchup . . ."). Experiment 2 replicated the findings of Experiment 1 for disfluency versus coordination constructions and also showed that the pattern of fixations to the critical distractor for disfluency constructions was similar to the fixation patterns for sentences employing contrastive focus (e.g., ". . . not some salt but rather some ketchup . . ."). The results suggest that similar mechanisms underlie the processing of repair disfluencies and contrastive focus, with listeners generating sets of entities that stand in semantic contrast to the reparandum in the case of disfluencies or the negated entity in the case of contrastive focus. (PsycINFO Database Record

When Feelings Arise with Meanings: How Emotion and Meaning of a Native Language Affect Second Language Processing in Adult Learners

To determine when and how L2 learners start to process L2 words affectively and semantically, we conducted a longitudinal study on their interaction in adult L2 learners. In four test sessions, spanning half a year of L2 learning, we monitored behavioral and ERP learning-related changes for one and the same set of words by means of a primed lexical-decision paradigm with L1 primes and L2 targets. Sensitivity rates, accuracy rates, RTs, and N400 amplitude to L2 words and pseudowords improved significantly across sessions. A semantic priming effect (e.g, prime "driver"facilitating response to target "street") was found in accuracy rates and RTs when collapsing Sessions 1 to 4, while this effect modulated ERP amplitudes within the first 300 ms of L2 target processing. An overall affective priming effect (e.g., "sweet" facilitating"taste") was also found in RTs and ERPs (posterior P1). Importantly, the ERPs showed an L2 valence effect across sessions (e.g., positive words were easier to process than neutral words), indicating that L2 learners were sensitive to L2 affective meaning. Semantic and affective priming interacted in the N400 time-window only in Session 4, implying that they affected meaning integration during L2 immersion together. The results suggest that L1 and L2 are initially processed semantically and affectively via relatively separate channels that are more and more linked contingent on L2 exposure.

What do we mean by prediction in language comprehension?

We consider several key aspects of prediction in language comprehension: its computational nature, the representational level(s) at which we predict, whether we use higher level representations to predictively pre-activate lower level representations, and whether we 'commit' in any way to our predictions, beyond pre-activation. We argue that the bulk of behavioral and neural evidence suggests that we predict probabilistically and at multiple levels and grains of representation. We also argue that we can, in principle, use higher level inferences to predictively pre-activate information at multiple lower representational levels. We also suggest that the degree and level of predictive pre-activation might be a function of the expected utility of prediction, which, in turn, may depend on comprehenders' goals and their estimates of the relative reliability of their prior knowledge and the bottom-up input. Finally, we argue that all these properties of language understanding can be naturally explained and productively explored within a multi-representational hierarchical actively generative architecture whose goal is to infer the message intended by the producer, and in which predictions play a crucial role in explaining the bottom-up input.

The Processing of Biologically Plausible and Implausible forms in American Sign Language: Evidence for Perceptual Tuning

The human auditory system distinguishes speech-like information from general auditory signals in a remarkably fast and efficient way. Combining psychophysics and neurophysiology (MEG), we demonstrate a similar result for the processing of visual information used for language communication in users of sign languages. We demonstrate that the earliest visual cortical responses in deaf signers viewing American Sign Language (ASL) signs show specific modulations to violations of anatomic constraints that would make the sign either possible or impossible to articulate. These neural data are accompanied with a significantly increased perceptual sensitivity to the anatomical incongruity. The differential effects in the early visual evoked potentials arguably reflect an expectation-driven assessment of somatic representational integrity, suggesting that language experience and/or auditory deprivation may shape the neuronal mechanisms underlying the analysis of complex human form. The data demonstrate that the perceptual tuning that underlies the discrimination of language and non-language information is not limited to spoken languages but extends to languages expressed in the visual modality.

Lexical Preactivation in Basic Linguistic Phrases

Many previous studies have shown that predictable words are read faster and lead to reduced neural activation, consistent with a model of reading in which words are activated in advance of being encountered. The nature of such preactivation, however, has typically been studied indirectly through its subsequent effect on word recognition. Here, we use magnetoencephalography to study the dynamics of prediction within serially presented adjective-noun phrases, beginning at the point at which the predictive information is first available to the reader. Using corpus transitional probability to estimate the predictability of a noun, we found an increase in activity in the left middle temporal gyrus in response to the presentation of highly predictive adjectives (i.e., adjectives that license a strong noun prediction). Moreover, we found that adjective predictivity and expected noun frequency interacted, such that the response to the highly predictive adjectives (e.g., stainless) was modulated by the frequency of the expected noun (steel). These results likely reflect preactivation of nouns in highly predictive contexts. The fact that the preactivation process was modulated by the frequency of the predicted item is argued to provide support for a frequency-sensitive lexicon.

Magnetoencephalographic evidence of early right hemisphere overactivation during metaphor comprehension in schizophrenia

Whereas language processing in neurotypical brains is left lateralized, individuals with schizophrenia (SZ) display a bilateral or reversed pattern of lateralization. We used MEG to investigate the implications of this atypicality on fine (left hemisphere) versus coarse (right hemisphere) semantic processing. Ten SZ and 14 controls were presented with fine (conventional metaphor, literal, and unrelated expressions) and coarse (novel metaphor) linguistic stimuli. Results showed greater activation of the right hemisphere for novel metaphors and greater bilateral activation for unrelated expressions at the M170 window in SZ. Moreover, at the M350, SZ showed reduced bilateral activation. We conclude that SZ are overreliant on early-stage coarse semantic processing. As a result, they jump too quickly to remote conclusions, with limited control over the meanings they form. This may explain one of the core symptoms of the disorder-loose associations.

Effects of prediction and contextual support on lexical processing: prediction takes precedence

Readers may use contextual information to anticipate and pre-activate specific lexical items during reading. However, prior studies have not clearly dissociated the effects of accurate lexical prediction from other forms of contextual facilitation such as plausibility or semantic priming. In this study, we measured electrophysiological responses to predicted and unpredicted target words in passages providing varying levels of contextual support. This method was used to isolate the neural effects of prediction from other potential contextual influences on lexical processing. While both prediction and discourse context influenced ERP amplitudes within the time range of the N400, the effects of prediction occurred much more rapidly, preceding contextual facilitation by approximately 100 ms. In addition, a frontal, post-N400 positivity (PNP) was modulated by both prediction accuracy and the overall plausibility of the preceding passage. These results suggest a unique temporal primacy for prediction in facilitating lexical access. They also suggest that the frontal PNP may index the costs of revising discourse representations following an incorrect lexical prediction.

Restrictive vs. non-restrictive composition: a magnetoencephalography study

Recent research on the brain mechanisms underlying language processing has implicated the left anterior temporal lobe (LATL) as a central region for the composition of simple phrases. Because these studies typically present their critical stimuli without contextual information, the sensitivity of LATL responses to contextual factors is unknown. In this magnetoencephalography (MEG) study, we employed a simple question-answer paradigm to manipulate whether a prenominal adjective or determiner is interpreted restrictively, i.e., as limiting the set of entities under discussion. Our results show that the LATL is sensitive to restriction, with restrictive composition eliciting higher responses than non-restrictive composition. However, this effect was only observed when the restricting element was a determiner, adjectival stimuli showing the opposite pattern, which we hypothesise to be driven by the special pragmatic properties of non-restrictive adjectives. Overall, our results demonstrate a robust sensitivity of the LATL to high level contextual and potentially also pragmatic factors.

Item parameters dissociate between expectation formats: a regression analysis of time-frequency decomposed EEG data

During language comprehension, semantic contextual information is used to generate expectations about upcoming items. This has been commonly studied through the N400 event-related potential (ERP), as a measure of facilitated lexical retrieval. However, the associative relationships in multi-word expressions (MWE) may enable the generation of a categorical expectation, leading to lexical retrieval before target word onset. Processing of the target word would thus reflect a target-identification mechanism, possibly indexed by a P3 ERP component. However, given their time overlap (200-500 ms post-stimulus onset), differentiating between N400/P3 ERP responses (averaged over multiple linguistically variable trials) is problematic. In the present study, we analyzed EEG data from a previous experiment, which compared ERP responses to highly expected words that were placed either in a MWE or a regular non-fixed compositional context, and to low predictability controls. We focused on oscillatory dynamics and regression analyses, in order to dissociate between the two contexts by modeling the electrophysiological response as a function of item-level parameters. A significant interaction between word position and condition was found in the regression model for power in a theta range (~7-9 Hz), providing evidence for the presence of qualitative differences between conditions. Power levels within this band were lower for MWE than compositional contexts when the target word appeared later on in the sentence, confirming that in the former lexical retrieval would have taken place before word onset. On the other hand, gamma-power (~50-70 Hz) was also modulated by predictability of the item in all conditions, which is interpreted as an index of a similar "matching" sub-step for both types of contexts, binding an expected representation and the external input.

The interplay between semantic and phonological constraints during spoken-word comprehension

This study addresses how top-down predictions driven by phonological and semantic information interact on spoken-word comprehension. To do so, we measured event-related potentials to words embedded in sentences that varied in the degree of semantic constraint (high or low) and in regional accent (congruent or incongruent) with respect to the target word pronunciation. The data showed a negative amplitude shift following phonological mismatch (target pronunciation incongruent with respect to sentence regional accent). Here, we show that this shift is modulated by sentence-level semantic constraints over latencies encompassing auditory (N100) and lexical (N400) components. These findings suggest a fast influence of top-down predictions and the interplay with bottom-up processes at sublexical and lexical levels of analysis.

Cortical dynamics of semantic processing during sentence comprehension: evidence from event-related optical signals

Using the event-related optical signal (EROS) technique, this study investigated the dynamics of semantic brain activation during sentence comprehension. Participants read sentences constituent-by-constituent and made a semantic judgment at the end of each sentence. The EROSs were recorded simultaneously with ERPs and time-locked to expected or unexpected sentence-final target words. The unexpected words evoked a larger N400 and a late positivity than the expected ones. Critically, the EROS results revealed activations first in the left posterior middle temporal gyrus (LpMTG) between 128 and 192 ms, then in the left anterior inferior frontal gyrus (LaIFG), the left middle frontal gyrus (LMFG), and the LpMTG in the N400 time window, and finally in the left posterior inferior frontal gyrus (LpIFG) between 832 and 864 ms. Also, expected words elicited greater activation than unexpected words in the left anterior temporal lobe (LATL) between 192 and 256 ms. These results suggest that the early lexical-semantic retrieval reflected by the LpMTG activation is followed by two different semantic integration processes: a relatively rapid and transient integration in the LATL and a relatively slow but enduring integration in the LaIFG/LMFG and the LpMTG. The late activation in the LpIFG, however, may reflect cognitive control.

An integrated theory of language production and comprehension

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal.

Long-range neural synchronization supports fast and efficient reading: EEG correlates of processing expected words in sentences

Word reading is heavily influenced by the information provided by previous context. In this study, we analyzed the neurophysiological bases of sentence reading through the EEG activity elicited during reading the same word embedded in differently constraining contexts: a) a low-constraining context; b) a high-constraining semantic compositional context; c) a high-constraining collocational context in which the item was in final position of a multi-word fixed-order expression. Cloze-probability of the two high-constraining contexts was equated. Before reading the target word we observed increased EEG gamma phase synchronization for the high-constraining compositional context and increased EEG theta synchronization for the collocational context (both compared to the low-constraining condition). After reading the target word we observed increased frontal positive EEG evoked activity (~220ms) for the high-constraining compositional context but an even earlier (~120ms) effect for the high-constraining collocational condition that was distributed over the scalp. A positive correlation was found only between the increased theta synchronization and the early EEG effect for the high-constraining collocational condition. Results indicate that long-range frontal-occipital interactions in the theta band - indexing working memory operations - support early visual-orthographic analysis of an incoming stimulus (such as the expected word); gamma-phase synchronization better represents binding operations between feed-forward activation and matching feedback. These data suggest that internal linguistic knowledge stored in long-term memory - if unambiguously pre-activated - supports the low-level perceptual processes involved in reading.

Motion words selectively modulate direction discrimination sensitivity for threshold motion

Can speech selectively modulate the sensitivity of a sensory system so that, in the presence of a suitable linguistic context, the discrimination of certain perceptual features becomes more or less likely? In this study, participants heard upward or downward motion words followed by a single visual field of random dots moving upwards or downwards. The time interval between the onsets of the auditory and the visual stimuli was varied parametrically. Motion direction could be either discriminable (suprathreshold motion) or non-discriminable (threshold motion). Participants had to judge whether the dots were moving upward or downward. Results show a double dissociation between discrimination sensitivity (d′) and reaction times depending on whether vertical motion was above or at threshold. With suprathreshold motion, responses were faster for congruent directions of words and dots, but sensitivity was equal across conditions. With threshold motion, sensitivity was higher for congruent directions of words and dots, but responses were equally fast across conditions. The observed differences in sensitivity and response times were largest when the dots appeared 450 ms after word onset, that is, consistently with electrophysiology, at the time the up/down semantics of the word had become available. These data suggest that word meanings can alter the balance between signal and noise within the visual system and affect the perception of low-level sensory features.

Neural mechanisms of rapid sensitivity to syntactic anomaly

Recent psycholinguistic models hypothesize that anticipatory processing can speed the response to linguistic input during language comprehension by pre-activating representations necessary for word recognition. We investigated the neurocognitive mechanisms of anticipatory processing by recording event-related potentials (ERPs) to syntactically anomalous (The thief was caught byfor police) and well-formed (e.g., The thief was caught bythe police) sentences. One group of participants saw anomalies elicited by the same word in every instance (e.g., for; low-variability stimuli), providing high affordances for predictions about the word-form appearing in the critical position. A second group saw anomalies elicited by seven different prepositions (at, of, on, for, from, over, with; high-variability stimuli) across the study, creating a more difficult prediction task. Syntactic category anomalies enhanced the occipital-temporal N170 component of the ERP, indicating rapid sensitivity – within 200 ms of word-onset – to syntactic anomaly. For low-variability but not the high-variability stimuli, syntactic anomaly also enhanced the earlier occipital-temporal P1 component, around 130 ms after word-onset, indicating that affordances for prediction engendered earlier sensitivity to syntactic anomaly. Independent components analysis revealed three sources within the ERP signal whose functional dynamics were consistent with predictive processing and early responses to syntactic anomaly. Distributed neural source modeling (sLORETA) of these early active sources produced a candidate network for early responses to words during reading in the right posterior occipital, left occipital-temporal, and medial parietal cortex.