Prediction Signatures in the Brain: Semantic Pre-Activation during Language Comprehension

Integrated sentence-level speech perception evokes strengthened language networks and facilitates early speech development

Natural poetic speeches (i.e., proverbs, nursery rhymes, and commercial ads) with strong prosodic regularities are easily memorized by children and the harmonious acoustic patterns are suggested to facilitate their integrated sentence processing. Do children have specific neural pathways for perceiving such poetic utterances, and does their speech development benefit from it? We recorded the task-induced hemodynamic changes of 94 children aged 2 to 12 years using functional near-infrared spectroscopy (fNIRS) while they listened to poetic and non-poetic natural sentences. Seventy-three adult as controls were recruited to investigate the developmental specificity of children group. The results indicated that poetic sentences perceiving is a highly integrated process featured by a lower brain workload in both groups. However, an early activated large-scale network was induced only in the child group, coordinated by hubs for connectivity diversity. Additionally, poetic speeches evoked activation in the phonological encoding regions in the children's group rather than adult controls which decreases with children's ages. The neural responses to poetic speeches were positively linked to children's speech communication performance, especially the fluency and semantic aspects. These results reveal children's neural sensitivity to integrated speech perception which facilitate early speech development by strengthening more sophisticated language networks and the perception-production circuit.

An ERP study on the certainty of epistemic modality in predictive inference processing

Previous psychological experiments have shown that predictive inference processing under different textual constraints is modulated by the directionality function of epistemic modality (EM) certainty within the context. Nevertheless, recent neuroscientific studies have not presented positive evidence for such a function during text reading. Consequently, the current study deposited Chinese EMs "" (possibly) and "" (surely) into a predictive inference context to examine whether a directionality of EM certainty influences the processing of predictive inference via the ERP technique. Two independent variables, namely textual constraint and EM certainty, were manipulated, and 36 participants were recruited. The results revealed that, in the anticipatory stage of predictive inference processing while under a weak textual constraint, low certainty evoked a larger N400 (300-500 ms) in the fronto-central and centro-parietal regions, indicating the augmentation of cognitive loads in calculating the possibility of representations of the forthcoming information. Meanwhile, high certainty elicited a right fronto-central late positive component (LPC) (500-700 ms) associated with semantically congruent but lexically unpredicted words. In the integration stage, low certainty resulted in larger right fronto-central and centro-frontal N400 (300-500 ms) effects in the weak textual constraint condition, associated with the facilitation of lexical-semantic retrieval or pre-activation, and high certainty successively elicited right fronto-central and centro-parietal LPC (500-700 ms) effects, associated respectively with lexical unpredictability and reanalysis of the sentence meaning. The results support the directionality function of EM certainty and reveal the complete neural processing of predictive inferences with high and low certainties under different textual constraint conditions.

Characterizing directional dynamics of semantic prediction based on inter-regional temporal generalization

The event-related potential/field component N400(m) has been widely used as a neural index for semantic prediction. It has long been hypothesized that feedback information from inferior frontal areas plays a critical role in generating the N400. However, due to limitations in causal connectivity estimation, direct testing of this hypothesis has remained difficult. Here, magnetoencephalography (MEG) data was obtained during a classic N400 paradigm where the semantic predictability of a fixed target noun was manipulated in simple German sentences. To estimate causality, we implemented a novel approach based on machine learning and temporal generalization to estimate the effect of inferior frontal gyrus (IFG) on temporal areas. In this method, a support vector machine (SVM) classifier is trained on each time point of the neural activity in IFG to classify less predicted (LP) and highly predicted (HP) nouns and then tested on all time points of superior/middle temporal sub-regions activity (and vice versa, to establish spatio-temporal evidence for or against causality). The decoding accuracy was significantly above chance level when the classifier was trained on IFG activity and tested on future activity in superior and middle temporal gyrus (STG/MTG). The results present new evidence for a model predictive speech comprehension where predictive IFG activity is fed back to shape subsequent activity in STG/MTG, implying a feedback mechanism in N400 generation. In combination with the also observed strong feedforward effect from left STG/MTG to IFG, our findings provide evidence of dynamic feedback and feedforward influences between IFG and temporal areas during N400 generation.

Causal evidence for a coordinated temporal interplay within the language network

Recent neurobiological models on language suggest that auditory sentence comprehension is supported by a coordinated temporal interplay within a left-dominant brain network, including the posterior inferior frontal gyrus (pIFG), posterior superior temporal gyrus and sulcus (pSTG/STS), and angular gyrus (AG). Here, we probed the timing and causal relevance of the interplay between these regions by means of concurrent transcranial magnetic stimulation and electroencephalography (TMS-EEG). Our TMS-EEG experiments reveal region- and time-specific causal evidence for a bidirectional information flow from left pSTG/STS to left pIFG and back during auditory sentence processing. Adapting a condition-and-perturb approach, our findings further suggest that the left pSTG/STS can be supported by the left AG in a state-dependent manner.

Neural correlates of pronoun processing: An activation likelihood estimation meta-analysis

Pronouns are unique linguistic devices that allow for the expression of referential relationships. Despite their communicative utility, the neural correlates of the operations involved in reference assignment and/or resolution, are not well-understood. The present study synthesized the neuroimaging literature on pronoun processing to test extant theories of pronoun comprehension. Following the PRISMA guidelines and thebest-practice recommendations for neuroimaging meta-analyses, a systematic literature search and record assessment were performed. As a result, 16 fMRI studies were included in the meta-analysis, and were coded in Scribe 3.6 for inclusion in the BrainMap database. The activation coordinates for the contrasts of interest were transformed into Talairach space and submitted to an Activation Likelihood Estimation (ALE) meta-analysis in GingerALE 3.0.1. The results indicated that pronoun processing had functional convergence in the left posterior middle and superior temporal gyri, potentially reflecting the retrieval, prediction and integration roles of these areas for pronoun processing.

An electrophysiological exploration of modifiers embedded in different hierarchies: Insight from short passives in Mandarin

This event-related potentials study explored the processing of Mandarin short passives. We used a syntactic violation paradigm to compare the processing of two auxiliary phrases (i.e., a verb-modifier di-phrase and a noun-modifier de-phrase). In the control condition, the syntactic hierarchy of the di-phrase was lower than that of the de-phrase. In the violation condition, the low-level violation was created by replacing di with de in the auxiliary phrase, while the high-level violation was created by replacing de with di in the auxiliary phrase. The ERP data showed that the noun-modifier elicited the greater left anterior negativity (LAN) and the P600 than the verb-modifier, in both the control condition and the violation condition. We also observed that the LAN induced by the verb-modifier phrase was greater in the control condition than that in the violation condition, while the LAN induced by the noun-modifier was greater in the violation condition than that in the control condition. These results suggested that the greater cortical LAN-P600 might differentiate the high-level hierarchy from the low-level hierarchy. In addition, we tentatively claimed that given the same predicate argument structure, long passives might be the default representational mode of short passives (generally, a constructional alternation might be activated during the processing of the target structure).

Predictability and decomposability separately contribute to compositional processing of idiomatic language

When reading, comprehenders construct a message-level representation and integrate new information as it becomes available. Such compositional processing may differ for idioms, where the meanings of the individual words do not always relate to the figurative meaning. Here, we examined how predictability and idiom decomposability contribute to compositional processing. Participants' EEG was recorded while they read sentences containing idioms that varied in decomposability and phrase-final word cloze probability, or their literal match (break the ice/slip on the ice) with little context prior to the phrase, along with adjective insertion conditions (break the freezing ice/slip on the freezing ice). Cloze probability modulated N400 amplitudes to critical words for both idiomatic and literal phrases, whereas P600 amplitudes only elicited by idiomatic phrases were also modulated. Phrases with adjective insertions reduced acceptability judgments, particularly for idioms, and led to N400 amplitude differences compared to critical words, but only for idioms. N400 differences were also found between idiomatic and literal contexts at the point of adjective insertion. Additionally, both cloze probability and decomposability modulated gamma band activity, with greater gamma activity for more predictable and less decomposable idioms, but at different times and with different scalp topographies, suggesting dissociable components of processing. These results support a hybrid model of processing in which multiple linguistic factors determine the type of processing engaged by the brain to comprehend non-literal language. When prior context is minimal, compositional processing may still be engaged when reading idioms.

Does entropy modulate the prediction of German long-distance verb particles?

In this paper we examine the effect of uncertainty on readers’ predictions about meaning. In particular, we were interested in how uncertainty might influence the likelihood of committing to a specific sentence meaning. We conducted two event-related potential (ERP) experiments using particle verbs such as turn down and manipulated uncertainty by constraining the context such that readers could be either highly certain about the identity of a distant verb particle, such as turn the bed […] down, or less certain due to competing particles, such as turn the music […] up/down. The study was conducted in German, where verb particles appear clause-finally and may be separated from the verb by a large amount of material. We hypothesised that this separation would encourage readers to predict the particle, and that high certainty would make prediction of a specific particle more likely than lower certainty. If a specific particle was predicted, this would reflect a strong commitment to sentence meaning that should incur a higher processing cost if the prediction is wrong. If a specific particle was less likely to be predicted, commitment should be weaker and the processing cost of a wrong prediction lower. If true, this could suggest that uncertainty discourages predictions via an unacceptable cost-benefit ratio. However, given the clear predictions made by the literature, it was surprisingly unclear whether the uncertainty manipulation affected the two ERP components studied, the N400 and the PNP. Bayes factor analyses showed that evidence for our a priori hypothesised effect sizes was inconclusive, although there was decisive evidence against a priori hypothesised effect sizes larger than 1μV for the N400 and larger than 3μV for the PNP. We attribute the inconclusive finding to the properties of verb-particle dependencies that differ from the verb-noun dependencies in which the N400 and PNP are often studied.

Understanding Design Features of Music and Language: The Choric/Dialogic Distinction

Music and spoken language share certain characteristics: both consist of sequences of acoustic elements that are combinatorically combined, and these elements partition the same continuous acoustic dimensions (frequency, formant space and duration). However, the resulting categories differ sharply: scale tones and note durations of small integer ratios appear in music, while speech uses phonemes, lexical tone, and non-isochronous durations. Why did music and language diverge into the two systems we have today, differing in these specific features? We propose a framework based on information theory and a reverse-engineering perspective, suggesting that design features of music and language are a response to their differential deployment along three different continuous dimensions. These include the familiar propositional-aesthetic ('goal') and repetitive-novel ('novelty') dimensions, and a dialogic-choric ('interactivity') dimension that is our focus here. Specifically, we hypothesize that music exhibits specializations enhancing coherent production by several individuals concurrently-the 'choric' context. In contrast, language is specialized for exchange in tightly coordinated turn-taking-'dialogic' contexts. We examine the evidence for our framework, both from humans and non-human animals, and conclude that many proposed design features of music and language follow naturally from their use in distinct dialogic and choric communicative contexts. Furthermore, the hybrid nature of intermediate systems like poetry, chant, or solo lament follows from their deployment in the less typical interactive context.

The Neural Basis of Semantic Prediction in Sentence Comprehension

Although prediction plays an important role in language comprehension, its precise neural basis remains unclear. This fMRI study investigated whether and how semantic-category-specific and common cerebral areas are recruited in predictive semantic processing during sentence comprehension. We manipulated the semantic constraint of sentence contexts, upon which a tool-related, a building-related, or no specific category of noun is highly predictable. This noun-predictability effect was measured not only over the target nouns but also over their preceding transitive verbs. Both before and after the appearance of target nouns, left anterior supramarginal gyrus was specifically activated for tool-related nouns and left parahippocampal place area was activated specifically for building-related nouns. The semantic-category common areas included a subset of left inferior frontal gyrus during the anticipation of incoming target nouns (activity enhancement for high predictability) and included a wide spread of areas (bilateral inferior frontal gyrus, left superior/middle temporal gyrus, left medial pFC, and left TPJ) during the integration of actually perceived nouns (activity reduction for high predictability). These results indicated that the human brain recruits fine divisions of cortical areas to distinguish different semantic categories of predicted words, and anticipatory semantic processing relies, at least partially, on top-down prediction conducted in higher-level cortical areas.

Functional Role of the N400 and P600 in Language-Related ERP Studies with Respect to Semantic Anomalies: An Overview

In this study, the language-related ERP studies relevant to the functional role of the N400 and P600 in semantically anomalous sentences and the underlying reasons which may affect their functions were reviewed. Since their discovery, the N400 and P600 have been the most important language-related ERP components. The N400 has been mostly elicited as a result of processing sentences with lexical and semantic anomalies, but later on, in many studies instead of the expected lexical-semantic N400 effect, semantic anomalies elicited a P600 effect called semantic P600. However, the functional interpretation of these two ERP components has constantly been a matter of debate. Perhaps most notably, it is proposed that it is not just the N400 which is related to semantic anomalies but the P600 can also be reflected as a result of these kinds of anomalies. Reviewing the literature for explaining the functions of the two ERP components, the N400 and the P600, during the processing of semantic anomalies revealed that still there is a need for more research on language processing in order to make the researchers capable of describing the underlying factors influencing them, especially more focused investigations of the functional-anatomical and neurocomputational models may provide a clearer understanding of them. Moreover, any practical theory or model of the N400 and the P600 in language comprehension needs to consider the apparent inconsistencies in the elicitation pattern of the N400 and the P600 in order to successfully capture the full data spectrum.

Representational Pattern Similarity of Electrical Brain Activity Reveals Rapid and Specific Prediction during Language Comprehension

Predicting upcoming events is a critical function of the brain, and language provides a fertile testing ground for studying prediction, as comprehenders use context to predict features of upcoming words. Many aspects of the mechanisms of prediction remain elusive, partly due to a lack of methodological tools to probe prediction formation in the moment. To elucidate what features are neurally preactivated and when, we used representational similarity analysis on previously collected sentence reading data. We compared EEG activity patterns elicited by expected and unexpected sentence final words to patterns from the preceding words of the sentence, in both strongly and weakly constraining sentences. Pattern similarity with the final word was increased in an early time window following the presentation of the pre-final word, and this increase was modulated by both expectancy and constraint. This was not seen at earlier words, suggesting that predictions were precisely timed. Additionally, pre-final word activity-the predicted representation-had negative similarity with later final word activity, but only for strongly expected words. These findings shed light on the mechanisms of prediction in the brain: rapid preactivation occurs following certain cues, but the predicted features may receive reduced processing upon confirmation.

One Way or Another: Cortical Language Areas Flexibly Adapt Processing Strategies to Perceptual And Contextual Properties of Speech

Cortical circuits rely on the temporal regularities of speech to optimize signal parsing for sound-to-meaning mapping. Bottom-up speech analysis is accelerated by top-down predictions about upcoming words. In everyday communications, however, listeners are regularly presented with challenging input-fluctuations of speech rate or semantic content. In this study, we asked how reducing speech temporal regularity affects its processing-parsing, phonological analysis, and ability to generate context-based predictions. To ensure that spoken sentences were natural and approximated semantic constraints of spontaneous speech we built a neural network to select stimuli from large corpora. We analyzed brain activity recorded with magnetoencephalography during sentence listening using evoked responses, speech-to-brain synchronization and representational similarity analysis. For normal speech theta band (6.5-8 Hz) speech-to-brain synchronization was increased and the left fronto-temporal areas generated stronger contextual predictions. The reverse was true for temporally irregular speech-weaker theta synchronization and reduced top-down effects. Interestingly, delta-band (0.5 Hz) speech tracking was greater when contextual/semantic predictions were lower or if speech was temporally jittered. We conclude that speech temporal regularity is relevant for (theta) syllabic tracking and robust semantic predictions while the joint support of temporal and contextual predictability reduces word and phrase-level cortical tracking (delta).

Evidence and implications of abnormal predictive coding in dementia

The diversity of cognitive deficits and neuropathological processes associated with dementias has encouraged divergence in pathophysiological explanations of disease. Here, we review an alternative framework that emphasizes convergent critical features of cognitive pathophysiology. Rather than the loss of ‘memory centres’ or ‘language centres’, or singular neurotransmitter systems, cognitive deficits are interpreted in terms of aberrant predictive coding in hierarchical neural networks. This builds on advances in normative accounts of brain function, specifically the Bayesian integration of beliefs and sensory evidence in which hierarchical predictions and prediction errors underlie memory, perception, speech and behaviour. We describe how analogous impairments in predictive coding in parallel neurocognitive systems can generate diverse clinical phenomena, including the characteristics of dementias. The review presents evidence from behavioural and neurophysiological studies of perception, language, memory and decision-making. The reformulation of cognitive deficits in terms of predictive coding has several advantages. It brings diverse clinical phenomena into a common framework; it aligns cognitive and movement disorders; and it makes specific predictions on cognitive physiology that support translational and experimental medicine studies. The insights into complex human cognitive disorders from the predictive coding framework may therefore also inform future therapeutic strategies.

A Rose by Any Other Verb: The Effect of Expectations and Word Category on Processing Effort in Situated Sentence Comprehension

Recent work has shown that linguistic and visual contexts jointly modulate linguistic expectancy and, thus, the processing effort for a (more or less) expected critical word. According to these findings, uncertainty about the upcoming referent in a visually-situated sentence can be reduced by exploiting the selectional restrictions of a preceding word (e.g., a verb or an adjective), which then reduces processing effort on the critical word (e.g., a referential noun). Interestingly, however, no such modulation was observed in these studies on the expectation-generating word itself. The goal of the current study is to investigate whether the reduction of uncertainty (i.e., the generation of expectations) simply does not modulate processing effort-or whether the particular subject-verb-object (SVO) sentence structure used in these studies (which emphasizes the referential nature of the noun as direct pointer to visually co-present objects) accounts for the observed pattern. To test these questions, the current design reverses the functional roles of nouns and verbs by using sentence constructions in which the noun reduces uncertainty about upcoming verbs, and the verb provides the disambiguating and reference-resolving piece of information. Experiment 1 (a Visual World Paradigm study) and Experiment 2 (a Grammaticality Maze study) both replicate the effect found in previous work (i.e., the effect of visually-situated context on the word which uniquely identifies the referent), albeit on the verb in the current study. Results on the noun, where uncertainty is reduced and expectations are generated in the current design, were mixed and were most likely influenced by design decisions specific to each experiment. These results show that processing of the reference-resolving word—whether it be a noun or a verb—reliably benefits from the prior linguistic and visual information that lead to the generation of concrete expectations.

Graded expectations in visually situated comprehension: Costs and benefits as indexed by the N400

Recently, Ankener et al. (Frontiers in Psychology, 9, 2387, 2018) presented a visual world study which combined both attention and pupillary measures to demonstrate that anticipating a target results in lower effort to integrate that target (noun). However, they found no indication that the anticipatory processes themselves, i.e., the reduction of uncertainty about upcoming referents, results in processing effort (cf. Linzen and Jaeger, Cognitive Science, 40(6), 1382-1411, 2016). In contrast, Maess et al. (Frontiers in Human Neuroscience, 10, 1-11, 2016) found that more constraining verbs elicited a higher N400 amplitude than unconstraining verbs. The aim of the present study was therefore twofold: Firstly, we examined whether the graded ICA effect, which was previously found on the noun as a result of a likelihood manipulation, replicates in ERP measures. Secondly, we set out to investigate whether the processes leading to the generation of expectations (derived during verb and scene processing) induce an N400 modulation. Our results confirm that visual context is combined with the verb's meaning to establish expectations about upcoming nouns and that these expectations affect the retrieval of the upcoming noun (modulated N400 on the noun). Importantly, however, we find no evidence for different costs in generating more or less specific expectations for upcoming nouns. Thus, the benefits of generating expectations are not associated with any costs in situated language comprehension.

Examining the Role of General Cognitive Skills in Language Processing: A Window into Complex Cognition

Whenever we use language to communicate, sounds, signs, and/or letters are combined into words that, in turn, form sentences, which together tell a story. Both language production and language comprehension rely on representations at different levels of granularity that need to be continuously and rapidly activated, selected, and combined. The representations themselves are specific to language, but many processes that regulate their use, such as inhibition of competitors or updating of working memory with new information, are so-called domain-general abilities that apply across different kinds of tasks. Here, we provide an overview of the behavioral and neurophysiological evidence in favor of domain-general abilities underpinning language skills and describe which particular aspects of production and comprehension draw upon such cognitive resources. We discuss how this line of research not only reveals important similarities between production and comprehension but also helps to establish links between language and other cognitive domains. Finally, we argue that studying how domain-general abilities are used in language will lead to important insights into the highly dynamic and efficient communication between brain networks that is necessary to successfully go from sounds to stories.

The Influence of Form- and Meaning-Based Predictions on Cortical Speech Processing Under Challenging Listening Conditions: A MEG Study

Under adverse listening conditions, prior linguistic knowledge about the form (i.e., phonology) and meaning (i.e., semantics) help us to predict what an interlocutor is about to say. Previous research has shown that accurate predictions of incoming speech increase speech intelligibility, and that semantic predictions enhance the perceptual clarity of degraded speech even when exact phonological predictions are possible. In addition, working memory (WM) is thought to have specific influence over anticipatory mechanisms by actively maintaining and updating the relevance of predicted vs. unpredicted speech inputs. However, the relative impact on speech processing of deviations from expectations related to form and meaning is incompletely understood. Here, we use MEG to investigate the cortical temporal processing of deviations from the expected form and meaning of final words during sentence processing. Our overall aim was to observe how deviations from the expected form and meaning modulate cortical speech processing under adverse listening conditions and investigate the degree to which this is associated with WM capacity. Results indicated that different types of deviations are processed differently in the auditory N400 and Mismatch Negativity (MMN) components. In particular, MMN was sensitive to the type of deviation (form or meaning) whereas the N400 was sensitive to the magnitude of the deviation rather than its type. WM capacity was associated with the ability to process phonological incoming information and semantic integration.

Semantic Processing in Autism Spectrum Disorders Is Associated With the Timing of Language Acquisition: A Magnetoencephalographic Study

Individuals with autism show difficulties in using sentence context to identify the correct meaning of ambiguous words, such as homonyms. In this study, the brain basis of sentence context effects on word understanding during reading was examined in autism spectrum disorder (ASD) and typical development (TD) using magnetoencephalography. The correlates of a history of developmental language delay in ASD were also investigated. Event related field responses at early (150 ms after the onset of a final word) and N400 latencies are reported for three different types of sentence final words: dominant homonyms, subordinate homonyms, and unambiguous words. Clear evidence for semantic access was found at both early and conventional N400 latencies in both TD participants and individuals with ASD with no history of language delay. By contrast, modulation of evoked activity related to semantic access was weak and not significant at early latencies in individuals with ASD with a history of language delay. The reduced sensitivity to semantic context in individuals with ASD and language delay was accompanied by strong right hemisphere lateralization at early and N400 latencies; such strong activity was not observed in TD individuals and individuals with ASD without a history of language delay at either latency. These results provide new evidence and support for differential neural mechanisms underlying semantic processing in ASD, and indicate that delayed language acquisition in ASD is associated with different lateralization and processing of language.

Magnetoencephalography and Language

This article provides an overview of research that uses magnetoencephalography to understand the brain basis of human language. The cognitive processes and brain networks that have been implicated in written and spoken language comprehension and production are discussed in relation to different methodologies: we review event-related brain responses, research on the coupling of neural oscillations to speech, oscillatory coupling between brain regions (eg, auditory-motor coupling), and neural decoding approaches in naturalistic language comprehension.

Permutation Statistics for Connectivity Analysis between Regions of Interest in EEG and MEG Data

Connectivity estimates based on electroencephalography (EEG) and magnetoencephalography (MEG) are unique in their ability to provide neurophysiologically meaningful spectral and temporal information non-invasively. This multi-dimensional aspect of the MEG/EEG based connectivity increases the challenges of the analysis and interpretation of the data. Many MEG/EEG studies address this complexity by using a hypothesis-driven approach, which focuses on particular regions of interest (ROI). However, if an effect is distributed unevenly over a large ROI and variable across subjects, it may not be detectable using conventional methods. Here, we propose a novel approach, which enhances the statistical power for weak and spatially discontinuous effects. This results in the ability to identify statistically significant connectivity patterns with spectral, temporal, and spatial specificity while correcting for multiple comparisons using nonparametric permutation methods. We call this new approach the Permutation Statistics for Connectivity Analysis between ROI (PeSCAR). We demonstrate the processing steps with simulated and real human data. The open-source Matlab code implementing PeSCAR are provided online.

The Influence of Visual Uncertainty on Word Surprisal and Processing Effort

A word’s predictability or surprisal, as determined by cloze probabilities or language models (Frank, 2013) is related to processing effort, in that less expected words take more effort to process (Hale, 2001; Lau et al., 2013). A word’s surprisal, however, may also be influenced by the non-linguistic context, such as visual cues: In the visual world paradigm (VWP), anticipatory eye movements suggest that listeners exploit the scene to predict what will be mentioned next (Altmann and Kamide, 1999). How visual context affects surprisal and processing effort, however, remains unclear. Here, we present a series of four studies providing evidence on how visually-determined probabilistic expectations for a spoken target word, as indicated by anticipatory eye movements, predict graded processing effort for that word, as assessed by a pupillometric measure (the Index of Cognitive Activity, ICA). These findings are a clear and robust demonstration that the non-linguistic context can immediately influence both lexical expectations, and surprisal-based processing effort.

Specific lexico-semantic predictions are associated with unique spatial and temporal patterns of neural activity

We used Magnetoencephalography (MEG) in combination with Representational Similarity Analysis to probe neural activity associated with distinct, item-specific lexico-semantic predictions during language comprehension. MEG activity was measured as participants read highly constraining sentences in which the final words could be predicted. Before the onset of the predicted words, both the spatial and temporal patterns of brain activity were more similar when the same words were predicted than when different words were predicted. The temporal patterns localized to the left inferior and medial temporal lobe. These findings provide evidence that unique spatial and temporal patterns of neural activity are associated with item-specific lexico-semantic predictions. We suggest that the unique spatial patterns reflected the prediction of spatially distributed semantic features associated with the predicted word, and that the left inferior/medial temporal lobe played a role in temporally 'binding' these features, giving rise to unique lexico-semantic predictions.

Oscillatory dynamics of cortical functional connections in semantic prediction

An event related potential, known as the N400, has been particularly useful in investigating language processing as it serves as a neural index for semantic prediction. There are numerous studies on the functional segregation of N400 neural sources; however, the oscillatory dynamics of functional connections among the relevant sources has remained elusive. In this study we acquired magnetoencephalography data during a classic N400 paradigm, where the semantic predictability of a fixed target noun was manipulated in simple German sentences. We conducted inter-regional functional connectivity (FC) and time-frequency analysis on known regions of the semantic network, encompassing bilateral temporal, and prefrontal cortices. Increased FC was found in less predicted (LP) nouns compared with highly predicted (HP) nouns in three connections: (a) right inferior frontal gyrus (IFG) and right middle temporal gyrus (MTG) from 0 to 300 ms mainly within the alpha band, (b) left lateral orbitofrontal (LOF) and right IFG around 400 ms within the beta band, and (c) left superior temporal gyrus (STG) and left LOF from 300 to 700 ms in the beta and low gamma bands. Furthermore, gamma spectral power (31-70 Hz) was stronger in HP nouns than in LP nouns in left anterior temporal cortices in earlier time windows (0-200 ms). Our findings support recent theories in language comprehension, suggesting fronto-temporal top-down connections are mainly mediated through beta oscillations while gamma band frequencies are involved in matching between prediction and input.

Relationships between alpha oscillations during speech preparation and the listener N400 ERP to the produced speech

While previous studies separately demonstrate EEG spectral modulations during speech preparation and ERP responses to the listened speech, it is unclear whether these responses are related on a trial-by-trial basis between a speaker and listener. In order to determine whether these responses are related in real-time, Electroencephalography (EEG) responses were measured simultaneously within a speaker and listener using a 24 electrode Mobile EEG system (18 participants; 9 pairs) during a sentence completion task. Each trial consisted of a sentence prompt with an incomplete ending (e.g. “I took my dog for a ____”). The speaker was instructed to fill in the ending with something expected (e.g. “walk”) (40 trials) or unexpected (e.g. “drink”) (40 trials). The other participant listened to the speaker throughout the block. We found that lower alpha band activity was reduced when individuals prepared unexpected sentence endings compared to expected sentence endings. Greater reductions in the speaker’s lower alpha activity during response preparation were correlated with a more negative N400 response in the listener to the unexpected word. These findings demonstrate that alpha suppression and the N400 ERP effect are present within a hyperscanning context and they are correlated between the speaker and listener during sentence completion.

Gamma Oscillatory Activity Related to Language Prediction

Using magnetoencephalography, the current study examined gamma activity associated with language prediction. Participants read high- and low-constraining sentences in which the final word of the sentence was either expected or unexpected. Although no consistent gamma power difference induced by the sentence-final words was found between the expected and unexpected conditions, the correlation of gamma power during the prediction and activation intervals of the sentence-final words was larger when the presented words matched with the prediction compared with when the prediction was violated or when no prediction was available. This suggests that gamma magnitude relates to the match between predicted and perceived words. Moreover, the expected words induced activity with a slower gamma frequency compared with that induced by unexpected words. Overall, the current study establishes that prediction is related to gamma power correlations and a slowing of the gamma frequency.

Neural Correlates of Semantic Prediction and Resolution in Sentence Processing

Most brain-imaging studies of language comprehension focus on activity following meaningful stimuli. Testing adult human participants with high-density EEG, we show that, already before the presentation of a critical word, context-induced semantic predictions are reflected by a neurophysiological index, which we therefore call the semantic readiness potential (SRP). The SRP precedes critical words if a previous sentence context constrains the upcoming semantic content (high-constraint contexts), but not in unpredictable (low-constraint) contexts. Specific semantic predictions were indexed by SRP sources within the motor system—in dorsolateral hand motor areas for expected hand-related words (e.g., “write”), but in ventral motor cortex for face-related words (“talk”). Compared with affirmative sentences, negated ones led to medial prefrontal and more widespread motor source activation, the latter being consistent with predictive semantic computation of alternatives to the negated expected concept. Predictive processing of semantic alternatives in negated sentences is further supported by a negative-going event-related potential at ∼400 ms (N400), which showed the typical enhancement to semantically incongruent sentence endings only in high-constraint affirmative contexts, but not to high-constraint negated ones. These brain dynamics reveal the interplay between semantic prediction and resolution (match vs error) processing in sentence understanding.

SIGNIFICANCE STATEMENT Most neuroscientists agree on the eminent importance of predictive mechanisms for understanding basic as well as higher brain functions. This contrasts with a sparseness of brain measures that directly reflects specific aspects of prediction, as they are relevant in the processing of language and thought. Here we show that when critical words are strongly expected in their sentence context, a predictive brain response reflects meaning features of these anticipated symbols already before they appear. The granularity of the semantic predictions was so fine grained that the cortical sources in sensorimotor and medial prefrontal cortex even distinguished between predicted face- or hand-related action words (e.g., the words “lick” or “pick”) and between affirmative and negated sentence meanings.

The costs of being certain: Brain potential evidence for linguistic preactivation in sentence processing

Prediction in sentence comprehension is often investigated by measuring the amplitude of the N400 ERP component to words that are more or less predictable from their preceding context. The N400-linked to the activation of word-associated semantic information-is reduced for words that are predictable, indicating that preactivation can lead to facilitated processing. We addressed the question whether there is measurable neural activity related to the preactivation of linguistic information before input confirms or disconfirms this prediction. We therefore measured ERPs not only to moderately to highly predictable target words, but also to preceding adverbs. Based on two separate cloze pretests, we quantified the impact of the adverb upon the predictability of the subsequent target word. Using linear mixed-effects analyses, we could show that the N400 amplitude at the target word was inversely related to target cloze value, thus replicating the finding that prediction has a facilitative effect on semantic processing. Crucially, the N400 amplitude at the pretarget adverb was modulated by adverb impact: When adverbs increased the predictability of the following word, the N400 was more negative going. We argue that this effect is related to the preactivation of linguistic information. Our findings indicate that the specification of predictions can lead to additional processes before these predictions are confirmed or disconfirmed and that activation of word-associated information through prediction is highly comparable to activation through actual input.