Neural evidence for Bayesian trial-by-trial adaptation on the N400 during semantic priming

Unraveling the mechanism of semantic object-based attention: The role of top-down search strategies

There is an ongoing debate between the sensory enhancement account and the attentional prioritization account concerning the mechanism underlying object-based attention. This debate remains unresolved because of the predominant use of geometric objects in previous studies, which made it difficult to experimentally dissociate the two accounts due to the prominent boundaries of these objects. The current study investigated the mechanism underlying semantic object-based attention by utilizing Chinese two-character words with different word frequencies and the event-related potentials (ERPs) technique to elucidate the ongoing debate. The behavioral results showed that the semantic object effect was observed only in the high-frequency condition. The ERP results revealed the following: (1) the N1 component was larger for the high-frequency condition compared with the low-frequency condition. However, there was no significant difference in amplitude between the N1 component elicited by invalid same object locations and invalid different object locations, irrespective of whether it occurred in high- or low-frequency conditions. (2) The P3 component elicited by invalid same object locations was larger than that elicited by invalid different object locations only in the high-frequency condition. (3) The N400 amplitude elicited by invalid same object locations was smaller than that elicited by invalid different object locations only in the high-frequency condition. These results suggest that in the absence of obvious object boundaries, the production of object-based attention is primarily driven by search strategies. Attentional prioritization, rather than sensory enhancement, is the dominant mechanism underlying object-based attention.

Cross-linguistic syntactic priming as rational expectation for syntactic repetition in the bilingual environment

Recent research suggests that syntactic priming in language comprehension-the facilitated processing of repeated syntactic structures-arises from the expectation for syntactic repetition due to rational adaptation to the linguistic environment. To further evaluate the generalizability of this expectation adaptation account in cross-linguistic syntactic priming and explore the influence of second language (L2) proficiency, we conducted a self-paced reading study with Chinese L2 learners of English by utilizing the sentential complement-direct object (SC-DO) ambiguity. The results showed that participants exposed to clusters of SC structures subsequently processed repetitions of this structure more rapidly (i.e., larger priming effects) than those exposed to the same number of SC structures but spaced in time, despite the prime and target being in two different languages (Chinese and English). Furthermore, this difference in priming strength was more pronounced for participants with higher L2 (English) proficiency. These findings demonstrate that cross-linguistic syntactic priming is consistent with the expectation for syntactic repetition that rationally adapts to syntactic clustering properties in surrounding bilingual environments, and such adaptation is enhanced as L2 proficiency increases. Taken together, our study extends the expectation adaptation account to cross-linguistic syntactic priming and integrates the role of L2 proficiency, which can shed new light on the mechanisms underlying syntactic priming, bilingual shared syntactic representations and expectation-based sentence processing.

A predictive coding model of the N400

The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.

The Interaction of Context Constraints and Predictive Validity during Sentence Reading

Words are not processed in isolation; instead, they are commonly embedded in phrases and sentences. The sentential context influences the perception and processing of a word. However, how this is achieved by brain processes and whether predictive mechanisms underlie this process remain a debated topic. Here, we employed an experimental paradigm in which we orthogonalized sentence context constraints and predictive validity, which was defined as the ratio of congruent to incongruent sentence endings within the experiment. While recording electroencephalography, participants read sentences with three levels of sentential context constraints (high, medium, and low). Participants were also separated into two groups that differed in their ratio of valid congruent to incongruent target words that could be predicted from the sentential context. For both groups, we investigated modulations of alpha power before, and N400 amplitude modulations after target word onset. The results reveal that the N400 amplitude gradually decreased with higher context constraints and cloze probability. In contrast, alpha power was not significantly affected by context constraint. Neither the N400 nor alpha power were significantly affected by changes in predictive validity.

Context-specific effects of violated expectations: ERP evidence

A complete understanding of the predictive processing effect in sentence comprehension needs to understand both the facilitation effect of successful prediction and the cost associated with disconfirmed predictions. The current study compares the predictive processing effect across two types of contexts in Mandarin Chinese: the classifier-noun vs. verb-noun phrases, when controlling for the degree of contextual constraints and cloze probability of the target nouns across the two contexts. The two contexts showed similar N400 patterns for expected target nouns, indicative of an identical facilitation effect of confirmed contextual expectation. But in the post-N400 time window, the processing cost associated with the unexpected words differed between the two contexts. Additional differences between the two contexts were also revealed by the neural oscillation patterns obtained prior to the target noun. The differences between the classifier vs. verb contexts shed new light on the revision mechanism that deals with disconfirmed expectations.

Multiple predictions during language comprehension: Friends, foes, or indifferent companions?

To comprehend language, we continually use prior context to pre-activate expected upcoming information, resulting in facilitated processing of incoming words that confirm these predictions. But what are the consequences of disconfirming prior predictions? To address this question, most previous studies have examined unpredictable words appearing in contexts that constrain strongly for a single continuation. However, during natural language processing, it is far more common to encounter contexts that constrain for multiple potential continuations, each with some probability. Here, we ask whether and how pre-activating both higher and lower probability alternatives influences the processing of the lower probability incoming word. One possibility is that, similar to language production, there is continuous pressure to select the higher-probability pre-activated alternative through competitive inhibition. During comprehension, this would result in relative costs in processing the lower probability target. A second possibility is that if the two pre-activated alternatives share semantic features, they mutually enhance each other's pre-activation. This would result in greater facilitation in processing the lower probability target. To distinguish between these accounts, we recorded ERPs as participants read three-sentence scenarios that constrained either for a single word or for two potential continuations - a higher probability expected candidate and a lower probability second-best candidate. We found no evidence that competitive pre-activation between the expected and second-best candidates resulted in costs in processing the second-best target, either during lexico-semantic processing (indexed by the N400) or at later stages of processing (indexed by a later frontal positivity). Instead, we found only benefits of pre-activating multiple alternatives, with evidence of enhanced graded facilitation on lower-probability targets that were semantically related to a higher-probability pre-activated alternative. These findings are consistent with a previous eye-tracking study by Luke and Christianson (2016, Cogn Psychol) using corpus-based materials. They have significant theoretical implications for models of predictive language processing, indicating that routine graded prediction in language comprehension does not operate through the same competitive mechanisms that are engaged in language production. Instead, our results align more closely with hierarchical probabilistic accounts of language comprehension, such as predictive coding.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Brain Signatures of Embodied Semantics and Language: A Consensus Paper

According to embodied theories (including embodied, embedded, extended, enacted, situated, and grounded approaches to cognition), language representation is intrinsically linked to our interactions with the world around us, which is reflected in specific brain signatures during language processing and learning. Moving on from the original rivalry of embodied vs. amodal theories, this consensus paper addresses a series of carefully selected questions that aim at determining when and how rather than whether motor and perceptual processes are involved in language processes. We cover a wide range of research areas, from the neurophysiological signatures of embodied semantics, e.g., event-related potentials and fields as well as neural oscillations, to semantic processing and semantic priming effects on concrete and abstract words, to first and second language learning and, finally, the use of virtual reality for examining embodied semantics. Our common aim is to better understand the role of motor and perceptual processes in language representation as indexed by language comprehension and learning. We come to the consensus that, based on seminal research conducted in the field, future directions now call for enhancing the external validity of findings by acknowledging the multimodality, multidimensionality, flexibility and idiosyncrasy of embodied and situated language and semantic processes.

What we do (not) know about the mechanisms underlying adaptive speech perception: A computational framework and review

Speech from unfamiliar talkers can be difficult to comprehend initially. These difficulties tend to dissipate with exposure, sometimes within minutes or less. Adaptivity in response to unfamiliar input is now considered a fundamental property of speech perception, and research over the past two decades has made substantial progress in identifying its characteristics. The mechanisms underlying adaptive speech perception, however, remain unknown. Past work has attributed facilitatory effects of exposure to any one of three qualitatively different hypothesized mechanisms: (1) low-level, pre-linguistic, signal normalization, (2) changes in/selection of linguistic representations, or (3) changes in post-perceptual decision-making. Direct comparisons of these hypotheses, or combinations thereof, have been lacking. We describe a general computational framework for adaptive speech perception (ASP) that-for the first time-implements all three mechanisms. We demonstrate how the framework can be used to derive predictions for experiments on perception from the acoustic properties of the stimuli. Using this approach, we find that-at the level of data analysis presently employed by most studies in the field-the signature results of influential experimental paradigms do not distinguish between the three mechanisms. This highlights the need for a change in research practices, so that future experiments provide more informative results. We recommend specific changes to experimental paradigms and data analysis. All data and code for this study are shared via OSF, including the R markdown document that this article is generated from, and an R library that implements the models we present.

Decoding semantic relatedness and prediction from EEG: A classification method comparison

Machine-learning (ML) decoding methods have become a valuable tool for analyzing information represented in electroencephalogram (EEG) data. However, a systematic quantitative comparison of the performance of major ML classifiers for the decoding of EEG data in neuroscience studies of cognition is lacking. Using EEG data from two visual word-priming experiments examining well-established N400 effects of prediction and semantic relatedness, we compared the performance of three major ML classifiers that each use different algorithms: support vector machine (SVM), linear discriminant analysis (LDA), and random forest (RF). We separately assessed the performance of each classifier in each experiment using EEG data averaged over cross-validation blocks and using single-trial EEG data by comparing them with analyses of raw decoding accuracy, effect size, and feature importance weights. The results of these analyses demonstrated that SVM outperformed the other ML methods on all measures and in both experiments.

Understanding the Effects of Constraint and Predictability in ERP

Intuitively, strongly constraining contexts should lead to stronger probabilistic representations of sentences in memory. Encountering unexpected words could therefore be expected to trigger costlier shifts in these representations than expected words. However, psycholinguistic measures commonly used to study probabilistic processing, such as the N400 event-related potential (ERP) component, are sensitive to word predictability but not to contextual constraint. Some research suggests that constraint-related processing cost may be measurable via an ERP positivity following the N400, known as the anterior post-N400 positivity (PNP). The PNP is argued to reflect update of a sentence representation and to be distinct from the posterior P600, which reflects conflict detection and reanalysis. However, constraint-related PNP findings are inconsistent. We sought to conceptually replicate Federmeier et al. (2007) and Kuperberg et al. (2020), who observed that the PNP, but not the N400 or the P600, was affected by constraint at unexpected but plausible words. Using a pre-registered design and statistical approach maximising power, we demonstrated a dissociated effect of predictability and constraint: strong evidence for predictability but not constraint in the N400 window, and strong evidence for constraint but not predictability in the later window. However, the constraint effect was consistent with a P600 and not a PNP, suggesting increased conflict between a strong representation and unexpected input rather than greater update of the representation. We conclude that either a simple strong/weak constraint design is not always sufficient to elicit the PNP, or that previous PNP constraint findings could be an artifact of smaller sample size.

Strategic Processing of Gender Stereotypes in Sentence Comprehension: An ERP Study

Gender stereotypes are often involved in language comprehension. This study investigated whether and to what extent their processing is under strategic control, by examining both proportion and order effects related to gender stereotypes for role nouns. We manipulated stereotypical gender consistencies, as in “Li’s daughter/son was a nurse…”, the relative proportions of gender-consistent and gender-inconsistent sentences (80%:20% and 50%:50% for high-proportion and equal-proportion sessions, respectively), and a between-participant factor of session order (high-proportion sessions preceding equal-proportion sessions and a reversed order for the high–equal and equal–high groups, respectively). Linear mixed-effect models revealed a larger N400 and a larger late negativity for stereotypically inconsistent compared to consistent sentences for the high–equal group only. These results indicate that even if sentence contexts have already determined the gender of target role characters, gender stereotypes for role nouns are still activated when the first half of the experiment facilitates their activation. The analyses of trial-by-trial dynamics showed that the N400 effects gradually decreased throughout equal-proportion sessions for the equal–high group. Our findings suggest that the processing of gender stereotypes can be under strategic control. In addition, readers may develop other strategies based on sentence contexts, when the processing strategy based on cue validity is not available.

Parafoveal words can modulate sentence meaning: Electrophysiological evidence from an RSVP-with-flanker task

During natural reading, readers can take up some visual information from not-yet-fixated words to the right of the current fixation and it is well-established that this parafoveal preview facilitates the subsequent foveal processing of the word. However, the extraction and integration of word meaning from parafoveal words and their possible influence on the semantic content of the sentence are controversial. In the current study, we recorded event-related potentials (ERPs) in the RSVP-with-flankers paradigm to test whether and how updates of sentential meaning, based only on parafoveal information, may influence the subsequent foveal processing. In Chinese sentences, the congruency of parafoveal and foveal target words with the sentence was orthogonally manipulated. In contrast to previous research, we also controlled for potentially confounding effects of parafoveal-to-foveal repetition priming (identity preview effects) on the N400. Crucially, we found that the classic effect of foveal congruency on the N400 component only appeared when the word in preview had been congruent with sentence meaning; in contrast, there was no N400 as a function of foveal incongruency when the preview word had also been incongruent. These results indicate that sentence meaning rapidly adapts to parafoveal preview, altering the semantic context for the subsequently fixated word. We also show that correct parafoveal preview generally attenuates the N400 once a word is fixated, regardless of congruency. Taken together, our findings underline the highly generative and adaptive framework of language comprehension.

Cognitive control mediates age-related changes in flexible anticipatory processing during listening comprehension

Effective listening comprehension not only requires processing local linguistic input, but also necessitates incorporating contextual cues available in the global communicative environment. Local sentence processing can be facilitated by pre-activation of likely upcoming input, or predictive processing. Recent evidence suggests that young adults can flexibly adapt local predictive processes based on cues provided by the global communicative environment, such as the reliability of specific speakers. Whether older comprehenders can also flexibly adapt to global contextual cues is currently unknown. Moreover, it is unclear whether the underlying mechanisms supporting local predictive processing differ from those supporting adaptation to global contextual cues. Critically, it is unclear whether these mechanisms change as a function of typical aging. We examined the flexibility of prediction in young and older adults by presenting sentences from speakers whose utterances were typically more or less predictable (i.e., reliable speakers who produced expected words 80% of the time, versus unreliable speakers who produced expected words 20% of the time). For young listeners, global speaker reliability cues modulated neural effects of local predictability on the N400. In contrast, older adults, on average, did not show global modulation of local processing. Importantly, however, cognitive control (i.e., Stroop interference effects) mediated age-related reductions in sensitivity to the reliability of the speaker. Both young and older adults with high cognitive control showed greater N400 effects of predictability during sentences produced by a reliable speaker, suggesting that cognitive control is required to regulate the strength of top-down predictions based on global contextual information. Critically, cognitive control predicted sensitivity to global speaker-specific information but not local predictability cues, suggesting that predictive processing in local sentence contexts may be supported by separable neural mechanisms from adaptation of prediction as a function of global context. These results have important implications for interpreting age-related change in predictive processing, and for drawing more generalized conclusions regarding domain-general versus language-specific accounts of prediction.

The N400 ERP component reflects an error-based implicit learning signal during language comprehension

The functional significance of the N400 evoked-response component is still actively debated. An increasing amount of theoretical and computational modelling work is built on the interpretation of the N400 as a prediction error. In neural network modelling work, it was proposed that the N400 component can be interpreted as the change in a probabilistic representation of meaning that drives the continuous adaptation of an internal model of the statistics of the environment. These results imply that increased N400 amplitudes should correspond to greater adaptation, which can be measured via implicit memory. To investigate this model derived hypothesis, the current study manipulated expectancy in a sentence reading task to influence N400 amplitudes and subsequently presented the previously expected vs. unexpected words in a perceptual identification task to measure implicit memory. As predicted, reaction times in the perceptual identification task were significantly faster for previously unexpected words that induced larger N400 amplitudes in the previous sentence reading task. Additionally, it could be demonstrated that this adaptation seems to specifically depend on the process underlying N400 amplitudes, as participants with larger N400 differences during sentence reading also exhibited a larger implicit memory benefit in the perceptual identification task. These findings support the interpretation of the N400 as an implicit learning signal driving adaptation in language processing.

How 'rational' is semantic prediction? A critique and re-analysis of

In a recent article in Cognition, Delaney-Busch et al. (2019) claim evidence for 'rational', Bayesian adaptation of semantic predictions, using ERP data from Lau, Holcomb, and Kuperberg (2013). Participants read associatively related and unrelated prime-target word pairs in a first block with only 10% related trials and a second block with 50%. Related words elicited smaller N400s than unrelated words, and this difference was strongest in the second block, suggesting greater engagement in predictive processing. Using a rational adaptor model, Delaney-Busch et al. argue that the stronger N400 reduction for related words in the second block developed as a function of the number of related trials, and concluded therefore that participants predicted related words more strongly when their predictions were fulfilled more often. In this critique, I discuss two critical flaws in their analyses, namely the confounding of prediction effects with those of lexical frequency and the neglect of data from the first block. Re-analyses suggest a different picture: related words by themselves did not yield support for their conclusion, and the effect of relatedness gradually strengthened in othe two blocks in a similar way. Therefore, the N400 did not yield evidence that participants rationally adapted their semantic predictions. Within the framework proposed by Delaney-Busch et al., presumed semantic predictions may even be thought of as 'irrational'. While these results yielded no evidence for rational or probabilistic prediction, they do suggest that participants became increasingly better at predicting target words from prime words.

Adaptation in Predictive Prosodic Processing in Bilinguals

Native language listeners engage in predictive processing in many processing situations and adapt their predictive processing to the statistics of the input. In contrast, second language listeners engage in predictive processing in fewer processing situations. The current study uses eye-tracking data from two experiments in bilinguals’ native language (L1) and second language (L2) to explore their predictive processing based on contrastive pitch accent cues, and their adaptation in the face of prediction errors. The results of the first experiment show inhibition effects for unpredicted referents in both the L1 and the L2 that can be modeled with a Bayesian adaptation model, suggesting that bilinguals adapt their prediction in the face of prediction errors in a way that is compatible with the model. In contrast, the results of the second experiment, after a training phase that increased the predictive validity of the cue, show inhibition effects for unpredicted referents only in the L1, but not in the L2. In addition, the Bayesian adaptation model significantly predicts only the L1, but not the L2 data. The results are discussed with respect to adaptation to the statistical properties of the input.

Rational Adaptation in Lexical Prediction: The Influence of Prediction Strength

Recent studies indicate that the processing of an unexpected word is costly when the initial, disconfirmed prediction was strong. This penalty was suggested to stem from commitment to the strongly predicted word, requiring its inhibition when disconfirmed. Additional studies show that comprehenders rationally adapt their predictions in different situations. In the current study, we hypothesized that since the disconfirmation of strong predictions incurs costs, it would also trigger adaptation mechanisms influencing the processing of subsequent (potentially) strong predictions. In two experiments (in Hebrew and English), participants made speeded congruency judgments on two-word phrases in which the first word was either highly constraining (e.g., “climate,” which strongly predicts “change”) or not (e.g., “vegetable,” which does not have any highly probable completion). We manipulated the proportion of disconfirmed predictions in highly constraining contexts between participants. The results provide additional evidence of the costs associated with the disconfirmation of strong predictions. Moreover, they show a reduction in these costs when participants experience a high proportion of disconfirmed strong predictions throughout the experiment, indicating that participants adjust the strength of their predictions when strong prediction is discouraged. We formulate a Bayesian adaptation model whereby prediction failure cost is weighted by the participant’s belief (updated on each trial) about the likelihood of encountering the expected word, and show that it accounts for the trial-by-trial data.

An ERP index of real-time error correction within a noisy-channel framework of human communication

Recent evidence suggests that language processing is well-adapted to noise in the input (e.g., spelling or speech errors, misreading or mishearing) and that comprehenders readily correct the input via rational inference over possible intended sentences given probable noise corruptions. In the current study, we probed the processing of noisy linguistic input, asking whether well-studied ERP components may serve as useful indices of this inferential process. In particular, we examined sentences where semantic violations could be attributed to noise-for example, in "The storyteller could turn any incident into an amusing antidote", where the implausible word "antidote" is orthographically and phonologically close to the intended "anecdote". We found that the processing of such sentences-where the probability that the message was corrupted by noise exceeds the probability that it was produced intentionally and perceived accurately-was associated with a reduced (less negative) N400 effect and an increased P600 effect, compared to semantic violations which are unlikely to be attributed to noise ("The storyteller could turn any incident into an amusing hearse"). Further, the magnitudes of these ERP effects were correlated with the probability that the comprehender retrieved a plausible alternative. This work thus adds to the growing body of literature that suggests that many aspects of language processing are optimized for dealing with noise in the input, and opens the door to electrophysiologic investigations of the computations that support the processing of imperfect input.

Selective adaptation in sentence comprehension: Evidence from event-related brain potentials

In this study, two event-related potential experiments were conducted to investigate whether readers adapt their expectations to morphosyntactically (Experiment 1) or semantically (Experiment 2) anomalous sentences when they are repeatedly exposed to them. To address this issue, we experimentally manipulated the probability of occurrence of grammatical sentences and syntactically and semantically anomalous sentences through experiments. For the low probability block, anomalous sentences were presented less frequently than grammatical sentences (with a ratio of 1 to 4), while they were presented as frequently as grammatical sentences in the equal probability block. Experiment 1 revealed a smaller P600 effect for morphosyntactic violations in the equal probability block than in the low probability block. Linear mixed-effects models were used to examine how the size of the P600 effect changed as the experiment went along. The results showed that the smaller P600 effect of the equal probability block resulted from an amplitude's decline in morphosyntactically violated sentences over the course of the experiment, suggesting an adaptation to morphosyntactic violations. In Experiment 2, semantically anomalous sentences elicited a larger N400 effect than their semantically natural counterparts regardless of probability manipulation. Little evidence was found in favour of adaptation to semantic violations in that the processing cost associated with the N400 did not decrease over the course of the experiment. Therefore, a dynamic aspect of language-processing system was demonstrated in this study. We will discuss why the language-processing system shows a selective adaptation to morphosyntactic violations.

Impairments in Probabilistic Prediction and Bayesian Learning Can Explain Reduced Neural Semantic Priming in Schizophrenia

It has been proposed that abnormalities in probabilistic prediction and dynamic belief updating explain the multiple features of schizophrenia. Here, we used electroencephalography (EEG) to ask whether these abnormalities can account for the well-established reduction in semantic priming observed in schizophrenia under nonautomatic conditions. We isolated predictive contributions to the neural semantic priming effect by manipulating the prime's predictive validity and minimizing retroactive semantic matching mechanisms. We additionally examined the link between prediction and learning using a Bayesian model that probed dynamic belief updating as participants adapted to the increase in predictive validity. We found that patients were less likely than healthy controls to use the prime to predictively facilitate semantic processing on the target, resulting in a reduced N400 effect. Moreover, the trial-by-trial output of our Bayesian computational model explained between-group differences in trial-by-trial N400 amplitudes as participants transitioned from conditions of lower to higher predictive validity. These findings suggest that, compared with healthy controls, people with schizophrenia are less able to mobilize predictive mechanisms to facilitate processing at the earliest stages of accessing the meanings of incoming words. This deficit may be linked to a failure to adapt to changes in the broader environment. This reciprocal relationship between impairments in probabilistic prediction and Bayesian learning/adaptation may drive a vicious cycle that maintains cognitive disturbances in schizophrenia.

The N400 indexes acquisition of novel emotion concepts via conceptual combination

The ability to learn new emotion concepts is adaptive and socially valuable as it communicates culturally held understandings about values, goals, and experiences. Yet, little work has examined the underlying mechanisms that allow for new emotion concepts and words to be integrated into the conceptual system. One such mechanism may be conceptual combination, or the ability to form novel concepts by dynamically combining previously acquired conceptual knowledge. In this study, we used event-related potentials (ERPs) to investigate the electrophysiological correlates of novel emotion concept acquisition via conceptual combination. Participants were briefly trained on 30 novel emotion combinations, each consisting of two English emotion words (the components; e.g., "sadness + fatigue") and a pseudoword (the target; e.g., "despip"). Participants then completed a semantic congruency task while ERPs were recorded. On each trial, two components were presented serially, followed by a target; participants judged whether the target was a valid combination of the preceding components. Targets could be correct or incorrect trained pseudowords, or new untrained pseudowords. Furthermore, components could be presented in reversed order (e.g., "fatigue" then "sadness") or as synonyms (e.g., "exhaustion" for "fatigue"). Consistent with our main hypotheses, we found a main effect of target, such that the correct combinations showed reduced N400 amplitudes when compared to both incorrect and untrained pseudowords. Critically, this effect held regardless of how the preceding components were presented, suggesting deeper semantic learning. These results extend prior findings on conceptual combination and novel word learning, and are congruent with predictive processing accounts of brain function.

I know what you're probably going to say: Listener adaptation to variable use of uncertainty expressions

Pragmatic theories of utterance interpretation share the assumption that listeners reason about alternative utterances that a speaker could have produced, but didn't. For such reasoning to be successful, listeners must have precise expectations about a speaker's production choices. This is at odds with the considerable variability across speakers that exists at all levels of linguistic representation. This tension can be reconciled by listeners adapting to the statistics of individual speakers. While linguistic adaptation is increasingly widely attested, semantic/pragmatic adaptation is underexplored. Moreover, what kind of representations listeners update during semantic/pragmatic adaptation - estimates of the speaker's lexicon, or estimates of the speaker's utterance preferences - remains poorly understood. In this work, we investigate semantic/pragmatic adaptation in the domain of uncertainty expressions like might and probably. In a series of web-based experiments, we find 1) that listeners vary in their expectations about a generic speaker's use of uncertainty expressions; 2) that listeners rapidly update their expectations about the use of uncertainty expressions after brief exposure to a speaker with a specific usage of uncertainty expressions; and 3) that listeners' interpretations of uncertainty expressions change after being exposed to a specific speaker. We present a novel computational model of semantic/pragmatic adaptation based on Bayesian belief updating and show, through a series of model comparisons, that semantic/pragmatic adaptation is best captured by listeners updating their beliefs both about the speaker's lexicon and their utterance preferences. This work has implications for both semantic theories of uncertainty expressions and psycholinguistic theories of adaptation: it highlights the need for dynamic semantic representations and suggests that listeners integrate their general linguistic knowledge with speaker-specific experiences to arrive at more precise interpretations.

Characterizing the Dynamics of Learning in Repeated Reference Games

The language we use over the course of conversation changes as we establish common ground and learn what our partner finds meaningful. Here we draw upon recent advances in natural language processing to provide a finer-grained characterization of the dynamics of this learning process. We release an open corpus (>15,000 utterances) of extended dyadic interactions in a classic repeated reference game task where pairs of participants had to coordinate on how to refer to initially difficult-to-describe tangram stimuli. We find that different pairs discover a wide variety of idiosyncratic but efficient and stable solutions to the problem of reference. Furthermore, these conventions are shaped by the communicative context: words that are more discriminative in the initial context (i.e., that are used for one target more than others) are more likely to persist through the final repetition. Finally, we find systematic structure in how a speaker's referring expressions become more efficient over time: Syntactic units drop out in clusters following positive feedback from the listener, eventually leaving short labels containing open-class parts of speech. These findings provide a higher resolution look at the quantitative dynamics of ad hoc convention formation and support further development of computational models of learning in communication.

The neurocognitive basis of metamemory: Using the N400 to study the contribution of fluency to judgments of learning

Metamemory is crucial for monitoring, evaluating, and optimizing memory performance. The basis of metamemory, however, is a matter of considerable debate. In the present study, we examined the contribution of processing fluency-the ease of processing information during learning-to metamemory judgments. We recorded event-related potentials (ERPs) while participants studied related and unrelated word pairs across two study-test cycles in a judgment of learning (JOL) task. In the first study-test cycle, related pairs were associated with better cued recall, higher JOLs, and a reduced N400 amplitude than unrelated pairs. Crucially, between- and within subject correlational analyses indicated that reduced N400 amplitudes, indexing more fluent processing, were associated with higher JOLs. Furthermore, single-trial N400 mediated a small but significant portion of the relatedness effect on JOLs. In the second study-test cycle, relatedness still increased recall and JOLs. However, related and unrelated pairs did not differ in N400 amplitude. Rather, unrelated pairs elicited a parietal positivity in a later time window that partially mediated the relatedness effect on JOLs. Together, these results suggest that processing fluency, indexed by the N400, contributes to the relatedness effect on JOLs when novel word pairs are learned, but not when previously studied pairs are relearned. Our results also imply that aspects of fluency not captured by the N400 and/or explicit beliefs about memory contribute to JOLs. This study demonstrates the utility of ERPs in gaining new insights into the neurocognitive mechanisms of metamemory.

(Early) context effects on event-related potentials over natural inputs

Language understanding requires the integration of the input with preceding context. Event-related potentials (ERPs) have contributed significantly to our understanding of what contextual information is accessed and when. Much of this research has, however, been limited to experimenter-designed stimuli with highly atypical lexical and context statistics. This raises questions about the extent to which previous findings generalize to everyday language processing of natural stimuli with typical linguistic statistics. We ask whether context can affect ERPs over natural stimuli early, before the N400 time window. We re-analyzed a data set of ERPs over ~700 visually presented content words in sentences from English novels. To increase power, we employed linear mixed effects regression simultaneously modeling random variance by subject and by item. To reduce concerns about Type I error inflation common to any type of time series analysis, we introduced a simple approach to model and discount auto-correlations at multiple, empirically determined, time lags. We compared this approach to Bonferroni correction. Planned follow-up analyses used Generalized Additive Mixture Models to assess the linearity of contextual effects, including lexical surprisal, found within the N400 time window. We found that contextual information affects ERPs in both early (~200ms after word onset) and late (N400) time windows, supporting a cascading, interactive account of lexical access.

fMRI evidence that left posterior temporal cortex contributes to N400 effects of predictability independent of congruity

Previous electrophysiological work argues that predictability and semantic incongruity rapidly impact comprehension, as indicated by modulation of the N400 component between ~300 and 500 ms. An ongoing question is whether effects of predictability in fact reflect pre-activation in long-term memory as opposed to modulating the kind of integration processes triggered by incongruity. Using fMRI, we compared the impact of predictability and incongruity in adjective-noun phrases, in regions identified with lexical and phrasal localizer scans. We found that predictability impacted activity in left posterior middle temporal gyrus (pMTG), while incongruity impacted activity in left precentral gyrus. Together with parallel data from ERP, these data are consistent with the hypothesis that left pMTG activity is a key contributor to N400 effects of predictability and that the relevant mechanism is reduced activation of stored lexical representations. We tentatively suggest that the left precentral region may play a role in reanalysis when incongruity is encountered.

Context-Based Facilitation in Visual Word Recognition: Evidence for Visual and Lexical But Not Pre-Lexical Contributions

Word familiarity and predictive context facilitate visual word processing, leading to faster recognition times and reduced neuronal responses. Previously, models with and without top-down connections, including lexical-semantic, pre-lexical (e.g., orthographic/phonological), and visual processing levels were successful in accounting for these facilitation effects. Here we systematically assessed context-based facilitation with a repetition priming task and explicitly dissociated pre-lexical and lexical processing levels using a pseudoword (PW) familiarization procedure. Experiment 1 investigated the temporal dynamics of neuronal facilitation effects with magnetoencephalography (MEG; N = 38 human participants), while experiment 2 assessed behavioral facilitation effects (N = 24 human participants). Across all stimulus conditions, MEG demonstrated context-based facilitation across multiple time windows starting at 100 ms, in occipital brain areas. This finding indicates context-based facilitation at an early visual processing level. In both experiments, we furthermore found an interaction of context and lexical familiarity, such that stimuli with associated meaning showed the strongest context-dependent facilitation in brain activation and behavior. Using MEG, this facilitation effect could be localized to the left anterior temporal lobe at around 400 ms, indicating within-level (i.e., exclusively lexical-semantic) facilitation but no top-down effects on earlier processing stages. Increased pre-lexical familiarity (in PWs familiarized utilizing training) did not enhance or reduce context effects significantly. We conclude that context-based facilitation is achieved within visual and lexical processing levels. Finally, by testing alternative hypotheses derived from mechanistic accounts of repetition suppression, we suggest that the facilitatory context effects found here are implemented using a predictive coding mechanism.