Predictability and novelty in literal language comprehension: an ERP study

A predictive coding model of the N400

The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.

Brain responses to a lab-evolved artificial language with space-time metaphors

What is the connection between the cultural evolution of a language and the rapid processing response to that language in the brains of individual learners? In an iterated communication study that was conducted previously, participants were asked to communicate temporal concepts such as "tomorrow," "day after," "year," and "past" using vertical movements recorded on a touch screen. Over time, participants developed simple artificial 'languages' that used space metaphorically to communicate in nuanced ways about time. Some conventions appeared rapidly and universally (e.g., using larger vertical movements to convey greater temporal durations). Other conventions required extensive social interaction and exhibited idiosyncratic variation (e.g., using vertical location to convey past or future). Here we investigate whether the brain's response during acquisition of such a language reflects the process by which the language's conventions originally evolved. We recorded participants' EEG as they learned one of these artificial space-time languages. Overall, the brain response to this artificial communication system was language-like, with, for instance, violations to the system's conventions eliciting an N400-like component. Over the course of learning, participants' brain responses developed in ways that paralleled the process by which the language had originally evolved, with early neural sensitivity to violations of a rapidly-evolving universal convention, and slowly developing neural sensitivity to an idiosyncratic convention that required slow social negotiation to emerge. This study opens up exciting avenues of future work to disentangle how neural biases influence learning and transmission in the emergence of structure in language.

Multiple predictions during language comprehension: Friends, foes, or indifferent companions?

To comprehend language, we continually use prior context to pre-activate expected upcoming information, resulting in facilitated processing of incoming words that confirm these predictions. But what are the consequences of disconfirming prior predictions? To address this question, most previous studies have examined unpredictable words appearing in contexts that constrain strongly for a single continuation. However, during natural language processing, it is far more common to encounter contexts that constrain for multiple potential continuations, each with some probability. Here, we ask whether and how pre-activating both higher and lower probability alternatives influences the processing of the lower probability incoming word. One possibility is that, similar to language production, there is continuous pressure to select the higher-probability pre-activated alternative through competitive inhibition. During comprehension, this would result in relative costs in processing the lower probability target. A second possibility is that if the two pre-activated alternatives share semantic features, they mutually enhance each other's pre-activation. This would result in greater facilitation in processing the lower probability target. To distinguish between these accounts, we recorded ERPs as participants read three-sentence scenarios that constrained either for a single word or for two potential continuations - a higher probability expected candidate and a lower probability second-best candidate. We found no evidence that competitive pre-activation between the expected and second-best candidates resulted in costs in processing the second-best target, either during lexico-semantic processing (indexed by the N400) or at later stages of processing (indexed by a later frontal positivity). Instead, we found only benefits of pre-activating multiple alternatives, with evidence of enhanced graded facilitation on lower-probability targets that were semantically related to a higher-probability pre-activated alternative. These findings are consistent with a previous eye-tracking study by Luke and Christianson (2016, Cogn Psychol) using corpus-based materials. They have significant theoretical implications for models of predictive language processing, indicating that routine graded prediction in language comprehension does not operate through the same competitive mechanisms that are engaged in language production. Instead, our results align more closely with hierarchical probabilistic accounts of language comprehension, such as predictive coding.

Conventionality determines the time course of indirect replies comprehension: An ERP study

Indirect language comprehension requires decoding both the literal meaning and the intended meaning of an utterance, in which pragmatic inference is involved. This study tests the role of conventionality in the time course of indirect reply processing by comparing conventional and non-conventional indirect replies with direct reply, respectively. We constructed discourses which consist of a context and a dialogue with one question (e.g., May I buy a necklace for you) and one reply (e.g., I really have too many). The reply utterance was segmented into three phrases and presented orderly for EEG recording, e.g., with the subject as the first phrase (e.g., I), the adverbial as the second phrase (e.g., really), and the predicate as the third phrase (e.g., have too many). Our results showed that for conventional indirect replies, the second phrase elicited a larger anterior negativity, and the third phrase elicited a larger anterior N400 compared with those in direct replies. By contrast, for the non-conventional indirect reply, only the third phrase elicited a larger late negativity than the direct replies. These findings suggest that conventionality determines the time course of the pragmatic inferences for the most relevant interpretation during indirect replies comprehension.

Age-related differences in expectation-based novel word learning

Adult language users can infer the meaning of a previously unfamiliar word from a single exposure to this word in a semantically and thematically constrained context, henceforth, predictive context (Borovsky et al., 2010 Cognition, 116(2), 289-296; Borovsky et al., 2012 Language Learning and Development, 8(3), 278-302). Children use predictive contexts to anticipate upcoming stimuli (Borovsky et al., 2012 Language Learning and Development, 8(3), 278-302; Mani & Huettig, 2012 Journal of Experimental Psychology: Human Perception and Performance, 38(4), 843-847), but the extent to which they rely on prediction to learn novel word forms is unclear (Gambi et al., 2021 Cognition, 211, 104650). Here, we examine children's one-shot learning from predictive contexts using a modified version of the one-shot learning ERP paradigm for children aged 7-13 years. In a first learning phase, we presented audio recordings of expected words and unexpected novel pseudowords in strongly and weakly constraining sentence contexts. In the following priming phase, the same recorded words and pseudowords were used as primes to identical/synonymous, related, and unrelated target words. We measured N400 modulations to the word and pseudoword continuations in the learning phase and to the identical/synonymous, related, or unrelated target words in the priming phase. When initially presented in strongly constraining sentences, novel pseudowords primed synonymous targets equally well as word primes of the same intended meaning. This pattern was particularly pronounced in older children. Our findings suggest that, around early adolescence, children can use single exposures to constraining contexts to infer the meaning of novel words and to integrate these novel words in their lexicons.

Information gain modulates brain activity evoked by reading

The human brain processes language to optimise efficient communication. Studies have shown extensive evidence that the brain’s response to language is affected both by lower-level features, such as word-length and frequency, and syntactic and semantic violations within sentences. However, our understanding on cognitive processes at discourse level remains limited: How does the relationship between words and the wider topic one is reading about affect language processing? We propose an information theoretic model to explain cognitive resourcing. In a study in which participants read sentences from Wikipedia entries, we show information gain, an information theoretic measure that quantifies the specificity of a word given its topic context, modulates word-synchronised brain activity in the EEG. Words with high information gain amplified a slow positive shift in the event related potential. To show that the effect persists for individual and unseen brain responses, we furthermore show that a classifier trained on EEG data can successfully predict information gain from previously unseen EEG. The findings suggest that biological information processing seeks to maximise performance subject to constraints on information capacity.

The N1-N2-LPC Pattern in Processing Advertising Pictorial Metaphors: An ERP Study

We investigated what the temporal processing is of advertising pictorial metaphors. After presenting “a word of product” and “its advertising pictures,” the experiment instructed participants to make a follow-up true–false judgment considering what the picture intended to suggest. A repeated-measures ANOVAs for a 2 (picture type: metaphor, non-metaphor) × 2 (prime–target condition: congruent, incongruent) × 3 (electrode site: Fz, Cz, Pz) experimental condition was conducted on three components, N1 (100–150 ms), N2 (200–300 ms), and LPC (400–600 ms and 600–1,000 ms). The results show that metaphor pictures elicited larger amplitude in N1 (broadly distributed), N2 (frontally biased) and LPC (parietally biased), roughly reflecting an entire process with an initial response to visual onsets, an early recognition of semantic violations and a prolonged reanalysis process of semantic integration. We argue that, different than verbal metaphors, this faster processing occurred due to the involvement of visual pathway.

The relationship between brain reaction and English reading tests for non-native English speakers

This research analyzed the brain activity of non-native English speakers while engaged in English reading tests. The brain wave event-related potentials (ERPs) of participants were used to analyze the difference between making correct and incorrect choices on English reading test items. Three English reading tests of differing levels were designed and 20 participants, 10 males and 10 females whose ages ranged from 20 to 24, voluntarily participated in the experiment. Experimental results were analyzed by performing independent t-tests on the ERPs of participants for gender, difficulty level, and correct versus wrong options. Participants who chose incorrect options elicited a larger N600, verifying results found in the literature. Another interesting result was found: For incorrectly answered items, different areas of brain showing a significant difference in ERPs between the chosen and non-chosen options corresponded to gender differences; for males, this area was located in the right hemisphere whereas for females, it was located in the left. Experimental results imply that non-native English speaking males and females employ different areas of the brain to comprehend the meaning of difficult items.

On the violation of causal, emotional, and locative inferences: An event-related potentials study

Previous event-related potential studies have demonstrated the online generation of inferences during reading for comprehension tasks. The present study contrasted the brainwave patterns of activity to the fulfilment or violation of various types of inferences (causal, emotional, locative). Relative to inference congruent sentence endings, a typical centro-parietal N400 was elicited for the violation of causal and locative inferences. This N400 effect was initially absent for emotional inferences, most likely due to their lower cloze probability. Between 500 and 750ms, a larger frontal positivity (pN400FP) was elicited by inference incongruent sentence endings in the causal condition. In emotional sentences, both inference congruent and incongruent endings exerted this frontally distributed late positivity. For the violation of locative inferences, the larger positivity was only marginally significant over left posterior scalp locations. Thus, not all inference eliciting sentences evoked a similar pattern of ERP responses. We interpret and discuss our results in line with recent views on what the N400, the P600 and the pN400FP brainwave potentials index.

Disentangling Metaphor from Context: An ERP Study

A large body of electrophysiological literature showed that metaphor comprehension elicits two different event-related brain potential responses, namely the so-called N400 and P600 components. Yet most of these studies test metaphor in isolation while in natural conversation metaphors do not come out of the blue but embedded in linguistic and extra-linguistic context. This study aimed at assessing the role of context in the metaphor comprehension process. We recorded EEG activity while participants were presented with metaphors and equivalent literal expressions in a minimal context (Experiment 1) and in a supportive context where the word expressing the ground between the metaphor's topic and vehicle was made explicit (Experiment 2). The N400 effect was visible only in minimal context, whereas the P600 was visible both in the absence and in the presence of contextual cues. These findings suggest that the N400 observed for metaphor is related to contextual aspects, possibly indexing contextual expectations on upcoming words that guide lexical access and retrieval, while the P600 seems to reflect truly pragmatic interpretative processes needed to make sense of a metaphor and derive the speaker's meaning, also in the presence of contextual cues. In sum, previous information in the linguistic context biases toward a metaphorical interpretation but does not suppress interpretative pragmatic mechanisms to establish the intended meaning.

How vertical hand movements impact brain activity elicited by literally and metaphorically related words: an ERP study of embodied metaphor

Embodied metaphor theory suggests abstract concepts are metaphorically linked to more experientially basic ones and recruit sensorimotor cortex for their comprehension. To test whether words associated with spatial attributes reactivate traces in sensorimotor cortex, we recorded EEG from the scalp of healthy adults as they read words while performing a concurrent task involving either upward- or downward- directed arm movements. ERPs were time-locked to words associated with vertical space—either literally (ascend, descend) or metaphorically (inspire, defeat)—as participants made vertical movements that were either congruent or incongruent with the words. Congruency effects emerged 200–300 ms after word onset for literal words, but not until after 500 ms post-onset for metaphorically related words. Results argue against a strong version of embodied metaphor theory, but support a role for sensorimotor simulation in concrete language.

Examining the N400 semantic context effect item-by-item: relationship to corpus-based measures of word co-occurrence

With increasing availability of digital text, there has been an explosion of computational methods designed to turn patterns of word co-occurrence in large text corpora into numerical scores expressing the "semantic distance" between any two words. The success of such methods is typically evaluated by how well they predict human judgments of similarity. Here, I examine how well corpus-based methods predict amplitude of the N400 component of the event-related potential (ERP), an online measure of lexical processing in brain electrical activity. ERPs elicited by the second words of 303 word pairs were analyzed at the level of individual items. Three corpus-based measures (mutual information, distributional similarity, and latent semantic analysis) were compared to a traditional measure of free association strength. In a regression analysis, corpus-based and free association measures each explained some of the variance in N400 amplitude, suggesting that these may tap distinct aspects of word relationships. Lexical factors of concreteness of word meaning, word frequency, number of semantic associates, and orthographic similarity also explained variance in N400 amplitude at the single-item level.

Is the comprehension of idiomatic sentences indeed impaired in paranoid Schizophrenia? A window into semantic processing deficits

Schizophrenia patients have been reported to be more impaired in comprehending non-literal than literal language since early studies on proverbs. Preference for literal rather than figurative interpretations continues to be documented. The main aim of this study was to establish whether patients are indeed able to use combinatorial semantic processing to comprehend literal sentences and both combinatorial analysis, and retrieval of pre-stored meanings to comprehend idiomatic sentences. The study employed a sentence continuation task in which subjects were asked to decide whether a target word was a sensible continuation of a previous sentence fragment to investigate idiomatic and literal sentence comprehension in patients with paranoid schizophrenia. Patients and healthy controls were faster in accepting sensible continuations than in rejecting non-sensible ones in both literal and idiomatic sentences. Patients were as accurate as controls in comprehending literal and idiomatic sentences, but they were overall slower than controls in all conditions. Once the contribution of cognitive covariates was partialled out, the response times (RTs) to sensible idiomatic continuations of patients did not significantly differ from those of controls. This suggests that the state of residual schizophrenia did not contribute to slower processing of sensible idioms above and beyond the cognitive deficits that are typically associated with schizophrenia.

Prospective and retrospective semantic processing: prediction, time, and relationship strength in event-related potentials

Semantic context effects have variously been attributed to prospective processing - predictions about upcoming words - or to retrospective appreciation of relationships after reading both context and target. In two experiments, we altered the core variable distinguishing prospective from retrospective processing, namely time. Word pairs varying in strength of relationship were presented sequentially, to allow time for anticipation of the second word, or simultaneously. For both sorts of presentation, the amplitude of the N400 component of the event-related potential was graded from Unrelated to Moderate/Weak to Strong associates. Strong associates showed a temporal advantage over weaker associates - an earlier context effect - only during sequential presentation. Spatial distributions of the N400 context effects also differed for simultaneous versus sequential presentation.

Beyond the N400: complementary access to early neural correlates of novel metaphor comprehension using combined electrophysiological and haemodynamic measurements

The simultaneous application of different neuroimaging methods combining high temporal and spatial resolution can uniquely contribute to current issues and open questions in the field of pragmatic language perception. In the present study, comprehension of novel metaphors was investigated using near-infrared spectroscopy (NIRS) combined with the simultaneous acquisition of electroencephalography (EEG)/event-related potentials (ERPs). For the first time, we investigated the effects of figurative language on early electrophysiological markers (P200, N400) and their functional relationship to cortical haemodynamic responses within the language network (Broca's area, Wernicke's area). To this end, 20 healthy subjects judged 120 sentences with respect to their meaningfulness, whereby phrases were either literal, metaphoric, or meaningless. Our results indicated a metaphor-specific P200 reduction and a linear increase of N400 amplitudes from literal over metaphoric to meaningless sentences. Moreover, there were metaphor related effects on haemodynamic responses accessed with NIRS, especially within the left lateral frontal cortex (Broca's area). Significant correlations between electrophysiological and haemodynamic responses indicated that P200 reductions during metaphor comprehension were associated with an increased recruitment of neural activity within left Wernicke's area, indicating a link between variations in neural activity and haemodynamic changes within Wernicke's area. This link may reflect processes related to interindividual differences regarding the ability to classify novel metaphors. The present study underlines the usefulness of simultaneous NIRS measurements in language paradigms - especially for investigating the functional significance of neurophysiological markers that have so far been rarely examined - as these measurements are easily and efficiently realizable and allow for a complementary examination of neural activity and associated metabolic changes in cortical areas.

Prediction in the service of comprehension: modulated early brain responses to omitted speech segments

Speech signals are often compromised by disruptions originating from external (e.g., masking noise) or internal (e.g., inaccurate articulation) sources. Speech comprehension thus entails detecting and replacing missing information based on predictive and restorative neural mechanisms. The present study targets predictive mechanisms by investigating the influence of a speech segment's predictability on early, modality-specific electrophysiological responses to this segment's omission. Predictability was manipulated in simple physical terms in a single-word framework (Experiment 1) or in more complex semantic terms in a sentence framework (Experiment 2). In both experiments, final consonants of the German words Lachs ([laks], salmon) or Latz ([lats], bib) were occasionally omitted, resulting in the syllable La ([la], no semantic meaning), while brain responses were measured with multi-channel electroencephalography (EEG). In both experiments, the occasional presentation of the fragment La elicited a larger omission response when the final speech segment had been predictable. The omission response occurred ∼125-165 msec after the expected onset of the final segment and showed characteristics of the omission mismatch negativity (MMN), with generators in auditory cortical areas. Suggestive of a general auditory predictive mechanism at work, this main observation was robust against varying source of predictive information or attentional allocation, differing between the two experiments. Source localization further suggested the omission response enhancement by predictability to emerge from left superior temporal gyrus and left angular gyrus in both experiments, with additional experiment-specific contributions. These results are consistent with the existence of predictive coding mechanisms in the central auditory system, and suggestive of the general predictive properties of the auditory system to support spoken word recognition.

Association of abnormal semantic processing with delusion-like ideation in frequent cannabis users: an electrophysiological study

RATIONALE

Frequent cannabis use is a risk marker for schizophrenia and delusions, but the neurocognitive mechanisms of this relationship remain unclear.

OBJECTIVES

We sought evidence that cannabis users have deficits in processing relationships between meaningful stimuli, similar to abnormalities reported in schizophrenia, and that these deficits are associated with delusion-like ideation. We used the N400 event-related brain potential (ERP) waveform as a neurophysiological probe of activation of concepts in semantic memory. We hypothesized that cannabis users would exhibit larger (more negative) than normal N400 amplitudes in response to stimuli meaningfully related to a preceding prime-reflecting deficient activation of concepts related to the prime. We further hypothesized that the magnitude of this abnormality would correlate with severity of delusion-like ideation.

METHODS

We recorded ERPs in 24 frequent cannabis users and 24 non-using comparison participants who viewed prime words followed by targets which were either words related or unrelated to the prime or pronounceable nonwords. The participants' task was to indicate whether the target was a word. Delusion-like ideation was measured via the Schizotypal Personality Questionnaire.

RESULTS

Contrary to our hypothesis, cannabis users exhibited smaller than normal N400s to both related and unrelated targets. These abnormalities correlated with delusion-like ideation in cannabis users only.

CONCLUSIONS

The results are consistent with a generalized abnormality of activation within semantic memory neural networks in cannabis users. Further research is needed to investigate whether such an abnormality plays a role in the development of delusion-like ideation in cannabis users.

So that's what you meant! Event-related potentials reveal multiple aspects of context use during construction of message-level meaning

Factors that modulate the influence of contextual information on semantic processing in language comprehension have been thoroughly investigated with the N400 component of the event-related potential (ERP), a direct measure of initial contact with semantic memory. Although context has a strong and immediate impact on processing, multiple mechanisms contribute to the construction of message-level representations during normal comprehension. Some of these may be engaged after or concurrent with the formation of an initial meaning representation, and can then serve to revise or reshape meaning. In this study, ERPs were recorded while participants read plausible sentences that continuously varied in the amount of contextual constraint for the sentence-final word, defined via extensive norming data including the range of possible alternative completions for the contexts. Consistent with numerous past studies, the amplitude of the N400 was graded with expectancy, as amplitudes decreased with increasing constraint. Additionally, a left-lateralized, broad, slow negativity onsetting around 400-500 ms was largest for sentences with moderately strong constraint. Within this range of constraint, the negativity was larger for sentences with fewer alternative completions compared to those with many different ones. The timing and scalp distribution of the effect resemble brain responses linked to engagement of working memory resources, ambiguity resolution, and comprehension of jokes. Similar to cases of "frame-shifting" in non-literal language, this effect may reflect processing associated with reinterpretation or reconsideration of contextual material when multiple interpretations of a sentence were likely.

Age-related changes in the impact of contextual strength on multiple aspects of sentence comprehension

Contextual information influences multiple aspects of language comprehension extended over time. To determine how age-related changes impact normal comprehension, effects of contextual strength were examined with event-related potentials. Increased contextual constraint facilitated semantic processing (reduced N400s). Effects were smaller and delayed for older adults, and sensitivity to contextual information was diminished for weak contexts. Both groups elicited a later left-lateralized frontal negativity associated with reinterpretation of context when multiple interpretations of a sentence were likely. Older adults evidenced the frontal negativity over a wider range of constraint. The change in the use of contextual information across age is attributed to decreased reliance on predictive processing for older adults. Thus, age-related changes in comprehension lead to differential engagement of processing resources over time for older adults.