Simulating the N400 ERP component as semantic network error: Insights from a feature-based connectionist attractor model of word meaning

The differential illusion memory for high-associated abstract concepts (DIM-HA) effect

A vast body of evidence has shown that concrete concepts are processed faster and more accurately than abstract concepts in a variety of cognitive tasks. This phenomenon is widely known as the concreteness effect, and explanations for its occurrence seem to reflect differences in processing and organization for both types of representations. While there is considerable evidence to support this concreteness effect, the nature of these differences is still controversial. In developing an explanation, we have proposed a relatively different approach from a false memory perspective using the Deese-Roediger-McDermott paradigm. To explore the implications of the association in creating false memories, we explore behavioral and electrophysiologically the false memory effect, where targets were manipulated according to their association strength and their concreteness. Results showed that false recognition rates differed significantly between concrete and abstract critical words when they were associated strongly with their respective lists, which led to a higher proportion of abstract false alarms both in behavioral and electrophysiological experiments. The principal outcome, which was called the DIM-HA effect, was discussed in terms of theories of associative activation and qualitatively different representation.

Prediction in reading: A review of predictability effects, their theoretical implications, and beyond

Historically, prediction during reading has been considered an inefficient and cognitively expensive processing mechanism given the inherently generative nature of language, which allows upcoming text to unfold in an infinite number of possible ways. This article provides an accessible and comprehensive review of the psycholinguistic research that, over the past 40 or so years, has investigated whether readers are capable of generating predictions during reading, typically via experiments on the effects of predictability (i.e., how well a word can be predicted from its prior context). Five theoretically important issues are addressed: What is the best measure of predictability? What is the functional relationship between predictability and processing difficulty? What stage(s) of processing does predictability affect? Are predictability effects ubiquitous? What processes do predictability effects actually reflect? Insights from computational models of reading about how predictability manifests itself to facilitate the reading of text are also discussed. This review concludes by arguing that effects of predictability can, to a certain extent, be taken as demonstrating evidence that prediction is an important but flexible component of real-time language comprehension, in line with broader predictive accounts of cognitive functioning. However, converging evidence, especially from concurrent eye-tracking and brain-imaging methods, is necessary to refine theories of prediction.

The prediction potential indexes the meaning and communicative function of upcoming utterances

Prediction has a fundamental role in language processing. However, predictions can be made at different levels, and it is not always clear whether speech sounds, morphemes, words, meanings, or communicative functions are anticipated during dialogues. Previous studies reported specific brain signatures of communicative pragmatic function, in particular enhanced brain responses immediately after encountering an utterance used to request an object from a partner, but relatively smaller ones when the same utterance was used for naming the object. The present experiment now investigates whether similar neuropragmatic signatures emerge in recipients before the onset of upcoming utterances carrying different predictable communicative functions. Trials started with a context question and object pictures displayed on the screen, raising the participant's expectation that words from a specific semantic category (food or tool) would subsequently be used to either name or request one of the objects. Already 600 msec before utterance onset, a larger prediction potential was observed when a request was anticipated relative to naming expectation. As this result is congruent with the neurophysiological difference previously observed right after the critical utterance, the anticipatory brain activity may index predictions about the social-communicative function of upcoming utterances. In addition, we also found that the predictable semantic category of the upcoming word was likewise reflected in the anticipatory brain potential. Thus, the neurophysiological characteristics of the prediction potential can capture different types of upcoming linguistic information, including semantic and pragmatic aspects of an upcoming utterance and communicative action.

Flexible Conceptual Representations

A view that has been gaining prevalence over the past decade is that the human conceptual system is malleable, dynamic, context-dependent, and task-dependent, that is, flexible. Within the flexible conceptual representation framework, conceptual representations are constructed ad hoc, forming a different, idiosyncratic instantiation upon each occurrence. In this review, we scrutinize the neurocognitive literature to better understand the nature of this flexibility. First, we identify some key characteristics of these representations. Next, we consider how these flexible representations are constructed by addressing some of the open questions in this framework: We review the age-old question of how to reconcile flexibility with the apparent need for shareable stable definitions to anchor meaning and come to mutual understanding, as well as some newer questions we find critical, namely, the nature of relations among flexible representations, the role of feature saliency in activation, and the viability of all-or-none feature activations. We suggest replacing the debate about the existence of a definitional stable core that is obligatorily activated with a question of the degree and probability of activation of the information constituting a conceptual representation. We rely on published works to suggest that (1) prior featural salience matters, (2) feature activation may be graded, and (3) Bayesian updating of prior information according to current demands offers a viable account of how flexible representations are constructed. This proposal provides a theoretical mechanism for incorporating a changing momentary context into a constructed representation, while still preserving some of the concept's constituent meaning.

Error-based Implicit Learning in Language: The Effect of Sentence Context and Constraint in a Repetition Paradigm

Prediction errors drive implicit learning in language, but the specific mechanisms underlying these effects remain debated. This issue was addressed in an EEG study manipulating the context of a repeated unpredictable word (repetition of the complete sentence or repetition of the word in a new sentence context) and sentence constraint. For the manipulation of sentence constraint, unexpected words were presented either in high-constraint (eliciting a precise prediction) or low-constraint sentences (not eliciting any specific prediction). Repetition-induced reduction of N400 amplitudes and of power in the alpha/beta frequency band was larger for words repeated with their sentence context as compared with words repeated in a new low-constraint context, suggesting that implicit learning happens not only at the level of individual items but additionally improves sentence-based predictions. These processing benefits for repeated sentences did not differ between constraint conditions, suggesting that sentence-based prediction update might be proportional to the amount of unpredicted semantic information, rather than to the precision of the prediction that was violated. In addition, the consequences of high-constraint prediction violations, as reflected in a frontal positivity and increased theta band power, were reduced with repetition. Overall, our findings suggest a powerful and specific adaptation mechanism that allows the language system to quickly adapt its predictions when unexpected semantic information is processed, irrespective of sentence constraint, and to reduce potential costs of strong predictions that were violated.

A predictive coding model of the N400

The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.

Explaining the Sentence Superiority Effect and N400s Elicited by Words and Short Sentences with OB1-Reader

Research into reading has benefitted from the emergence of powerful computational models that account for reading behavior at different levels. Such models become more powerful when the underlying anatomy, architecture or 'physiology' can be linked to the behavior of interest. OB1-reader is a reading model that simulates the processes underlying reading in the human brain. Previous studies showed that OB1-reader can account for various phenomena in the word recognition and text reading literatures. Here we aim to extend OB1's scope, by simulating behavioral performance and evoked EEG activity for two experimental word-recognition tasks: a flanker task in which unrelated flankers generated less accurate responses combined with a larger N400, and a sentence reading task in which words were recognized more accurately at central positions and within intact sentences, than at peripheral positions and in scrambled sentences. OB1 simulated several behavioral findings in both paradigms, including the so-called sentence superiority effect. Moreover, virtual event-related potentials (ERPs) generated from node activity in OB1 were compared to human ERPs. More lexical activity in OB1 predicted the size of the N400 component of human readers in both experiments, but not the N250.

Tracking Lexical and Semantic Prediction Error Underlying the N400 Using Artificial Neural Network Models of Sentence Processing

Recent research has shown that the internal dynamics of an artificial neural network model of sentence comprehension displayed a similar pattern to the amplitude of the N400 in several conditions known to modulate this event-related potential. These results led Rabovsky et al. (2018) to suggest that the N400 might reflect change in an implicit predictive representation of meaning corresponding to semantic prediction error. This explanation stands as an alternative to the hypothesis that the N400 reflects lexical prediction error as estimated by word surprisal (Frank et al., 2015). In the present study, we directly model the amplitude of the N400 elicited during naturalistic sentence processing by using as predictor the update of the distributed representation of sentence meaning generated by a sentence gestalt model (McClelland et al., 1989) trained on a large-scale text corpus. This enables a quantitative prediction of N400 amplitudes based on a cognitively motivated model, as well as quantitative comparison of this model to alternative models of the N400. Specifically, we compare the update measure from the sentence gestalt model to surprisal estimated by a comparable language model trained on next-word prediction. Our results suggest that both sentence gestalt update and surprisal predict aspects of N400 amplitudes. Thus, we argue that N400 amplitudes might reflect two distinct but probably closely related sub-processes that contribute to the processing of a sentence.

Feats: A database of semantic features for early produced noun concepts

Semantic feature production norms have several desirable characteristics that have supported models of representation and processing in adults. However, several key challenges have limited the use of semantic feature norms in studies of early language acquisition. First, existing norms provide uneven and inconsistent coverage of early-acquired concepts that are typically produced and assessed in children under the age of three, which is a time of tremendous growth of early vocabulary skills. Second, it is difficult to assess the degree to which young children may be familiar with normed features derived from these adult-generated datasets. Third, it has been difficult to adopt standard methods to generate semantic network models of early noun learning. Here, we introduce Feats-a tool that was designed to make headway on these challenges by providing a database, the Language Learning and Meaning Acquisition (LLaMA) lab Noun Norms that extends a widely used set of feature norms McRae et al. Behavior Research Methods 37, 547-559, (2005) to include full coverage of noun concepts on a commonly used early vocabulary assessment. Feats includes several tools to facilitate exploration of features comprising early-acquired nouns, assess the developmental appropriateness of individual features using toddler-accessibility norms, and extract semantic network statistics for individual vocabulary profiles. We provide a tutorial overview of Feats. We additionally validate our approach by presenting an analysis of an overlapping set of concepts collected across prior and new data collection methods. Furthermore, using network graph analyses, we show that the extended set of norms provides novel, reliable results given their enhanced coverage.

Effects of anxiety state on N400 event-related brain potential response to unexpected semantic stimuli

Emotional states can influence how people use meaningful context to make predictions about what comes next. To measure whether state anxiety influences such prediction, we used the N400 event-related brain potential (ERP) response to semantic stimuli, whose amplitude is smaller (less negative) when the stimulus is more predicted based on preceding context. Participants (n = 28) were randomized to one of two groups, who underwent either an "anxious-uncertainty" procedure previously shown to increase anxiety, or a control procedure. Both before and after this procedure, participants' ERPs were recorded while they viewed category definitions (e.g., "a type of fruit"), each followed by a target word that was either a high-typicality category exemplar ("apple"), low-typicality exemplar ("cherry"), or non-exemplar ("clamp") of the category. Participants' task was to respond by pressing one of two buttons to indicate whether the target represented a member of the category. As expected, based on previous work, overall, N400 amplitudes were largest (most negative) in response to non-exemplars, intermediate to low-typicality exemplars, and smallest to high-typicality exemplars. N400 amplitudes were larger to non-exemplars after the anxious-uncertainty procedure than after the control procedure. N400 amplitudes to both types of exemplars did not differ after the anxious-uncertainty procedure versus the control procedure. The results are consistent with participants devoting more neural resources to processing contextually unexpected items under anxious states, rather than anxiety facilitating processing of expected items.

Why is a flamingo named as pelican and asparagus as celery? Understanding the relationship between targets and errors in a speeded picture naming task

Speakers sometimes make word production errors, such as mistakenly saying pelican instead of flamingo. This study explored which properties of an error influence the likelihood of its selection over the target word. Analysing real-word errors in speeded picture naming, we investigated whether, relative to the target, naming errors were more typical representatives of the semantic category, were associated with more semantic features, and/or were semantically more closely related to the target than its near semantic neighbours were on average. Results indicated that naming errors tended to be more typical category representatives and possess more semantic features than the targets. Moreover, while not being the closest semantic neighbours, errors were largely near semantic neighbours of the targets. These findings suggest that typicality, number of semantic features, and semantic similarity govern activation levels in the production system, and we discuss possible mechanisms underlying these effects in the context of word production theories.

Insights from simultaneous EEG-fMRI and patient data illuminate the role of the anterior medial temporal lobe in N400 generation

The N400, a negative event-related potential (ERP) peaking approximately 400 ms after stimulus onset, is known to reflect the processing of semantic information. While scalp recordings have contributed to understanding the psychological processes underlying the N400, they have been limited in identifying its neural basis. However, recent intracranial ERP recordings and fMRI studies have shed light on the crucial role of the anterior medial temporal lobe (AMTL) in semantic information processing. These findings suggest that the N400 partially represents activity in the AMTL structures. To investigate the neural underpinnings of the N400 effect, we simultaneously recorded ERPs and event-related fMRI during a semantic priming paradigm in a sample of 12 young, healthy subjects. Additionally, we collected ERPs and structural brain data from older healthy adults and patients with amnestic mild cognitive impairment (aMCI), a population characterized by neurodegenerative changes in the AMTL. In our fMRI results, we identified bilateral loci in the AMTL as the global maxima. Employing an EEG-informed fMRI analysis, we explored trial-to-trial fluctuations in semantic processing by linking single-trial N400 amplitudes to the Blood Oxygen Level Dependent (BOLD) signal. This approach provided the first direct evidence linking the N400 recorded at the scalp level to the corresponding BOLD signal in the AMTL. Consistent with these findings, patients with aMCI exhibited a diminished N400 effect compared to healthy older adults. Furthermore, voxel-based morphometry analysis revealed a correlation between the magnitude of the N400 effect and the integrity of the AMTL. By integrating data from simultaneous EEG-fMRI, and patient studies, our research advances our understanding of the neural substrate of the N400 and highlights the critical involvement of the AMTL in semantic processing.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Bayesian Surprise Predicts Human Event Segmentation in Story Listening

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error.

Better Together: Integrating Multivariate with Univariate Methods, and MEG with EEG to Study Language Comprehension

We used MEG and EEG to examine the effects of Plausibility (anomalous vs. plausible) and Animacy (animate vs. inanimate) on activity to incoming words during language comprehension. We conducted univariate event-related and multivariate spatial similarity analyses on both datasets. The univariate and multivariate results converged in their time course and sensitivity to Plausibility. However, only the spatial similarity analyses detected effects of Animacy. The MEG and EEG findings largely converged between 300-500ms, but diverged in their univariate and multivariate responses to the anomalies between 600-1000ms. We interpret the full set of results within a predictive coding framework. In addition to the theoretical significance of these findings, we discuss the methodological implications of the convergence and divergence between the univariate and multivariate results, as well as between the MEG and EEG results. We argue that a deeper understanding of language processing can be achieved by integrating different analysis approaches and techniques.

Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations

Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip-gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract-concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb-noun thematic role relations and noun-noun event-based relations that play important roles in sentence comprehension. Interestingly, Skip-gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event-based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event-based relations.

Predictive coding across the left fronto-temporal hierarchy during language comprehension

We used magnetoencephalography (MEG) and event-related potentials (ERPs) to track the time-course and localization of evoked activity produced by expected, unexpected plausible, and implausible words during incremental language comprehension. We suggest that the full pattern of results can be explained within a hierarchical predictive coding framework in which increased evoked activity reflects the activation of residual information that was not already represented at a given level of the fronto-temporal hierarchy ("error" activity). Between 300 and 500 ms, the three conditions produced progressively larger responses within left temporal cortex (lexico-semantic prediction error), whereas implausible inputs produced a selectively enhanced response within inferior frontal cortex (prediction error at the level of the event model). Between 600 and 1,000 ms, unexpected plausible words activated left inferior frontal and middle temporal cortices (feedback activity that produced top-down error), whereas highly implausible inputs activated left inferior frontal cortex, posterior fusiform (unsuppressed orthographic prediction error/reprocessing), and medial temporal cortex (possibly supporting new learning). Therefore, predictive coding may provide a unifying theory that links language comprehension to other domains of cognition.

Are they really stronger? Comparing effects of semantic variables in speeded deadline and standard picture naming

Investigations of effects of semantic variables on picture naming have often been inconclusive, with some studies reporting significant and others non-significant effects. One potential explanation may relate to the specific naming tasks used: While most previous studies have used standard picture naming, others have used speeded naming that requires participants to prioritise naming speed over accuracy. Speeded naming has been suggested to cause enhanced effects of item-inherent word characteristics due to disruptions of cognitive control and resulting modulations of responsiveness to input. Consequently, this study investigated whether effects are stronger in speeded compared to standard picture naming, focusing on six feature-based semantic variables: number of semantic features, intercorrelational density, number of near semantic neighbours, semantic similarity, typicality, and distinctiveness. The results showed few differences in the variables' effects between the two naming tasks: In the naming latency analysis, the inhibitory effect of distinctiveness was stronger in the speeded naming task, while in the accuracy analysis the effect of number of semantic features was stronger in the standard naming task. These findings cannot, therefore, be exclusively accounted for by increased responsiveness to input in speeded naming and we discuss possible underlying mechanisms. We conclude that, while some differences in effects of semantic variables between previous studies may have been caused by the specific naming task used, differences between studies more likely depend on statistical power and control of other influential variables in the experiment.

Investigating a neural language model's replicability of psycholinguistic experiments: A case study of NPI licensing

The recent success of deep learning neural language models such as Bidirectional Encoder Representations from Transformers (BERT) has brought innovations to computational language research. The present study explores the possibility of using a language model in investigating human language processes, based on the case study of negative polarity items (NPIs). We first conducted an experiment with BERT to examine whether the model successfully captures the hierarchical structural relationship between an NPI and its licensor and whether it may lead to an error analogous to the grammatical illusion shown in the psycholinguistic experiment (Experiment 1). We also investigated whether the language model can capture the fine-grained semantic properties of NPI licensors and discriminate their subtle differences on the scale of licensing strengths (Experiment 2). The results of the two experiments suggest that overall, the neural language model is highly sensitive to both syntactic and semantic constraints in NPI processing. The model's processing patterns and sensitivities are shown to be very close to humans, suggesting their role as a research tool or object in the study of language.

Very early and late form-to-meaning computations during visual word recognition as revealed by electrophysiology

We used a large-scale data-driven approach to investigate the role of word form in accessing semantics. By using distributional semantic methods and taking advantage of an ERP lexical decision mega-study, we investigated the exact time dynamic of semantic access from printed words as driven by orthography-semantics consistency (OSC) and phonology-semantics consistency (PSC). Generalized Additive Models revealed very early and late OSC-by-PSC interactions, visible at 100 and 400 msec, respectively. This pattern suggests that, during visual word recognition: a) meaning is accessed by means of two distinct and interactive paths - the orthography-to-meaning and the orthography-to-phonology-to-meaning path -, which mutually contribute to recognition since early stages; b) the system may exploit a dual mechanism for semantic access, with early and late effects associated to a fast-coarse and a slow-fine grained semantic analysis, respectively. The results also highlight the high sensitivity of the visual word recognition system to arbitrary form-meaning relations.

Electrophysiological evidence for a subject-first strategy in visually situated auditory sentence processing in Korean

This study investigated a subject-first strategy in prediction mechanism in visually situated sentence processing in Korean, using event-related potentials (ERPs). According to the subject-first strategy, parsers tend to generate sentences conforming to canonical sentence word order (i.e., SOV in Korean), subject-first sentence, mapping conceptually more prominent referent such as agent of the event on the subject position of the sentence. Therefore, in the predictive mechanism of language comprehension, the subject is pre-activated and anticipated for the first NP of the sentence at the initial phase of bottom-up language processing. This study tested this subject-first strategy in Korean by examining brain responses to object-initial sentences (OV) compared with subject-initial sentences (SV) under the context of clear thematic role relations set by a visual image. The results of an ERP experiment with 30 native Korean speakers identified neural effects for object-initial sentences compared with subject-initial sentences at the NP and Verb, reflecting a conflict between the pre-activated representation in the parser's mind and the encountered bottom-up input. An N400 effect was elicited at the NP, as early as at the noun, not at the following object case marker. Late frontal positivity (LFP) was also found in the sentence-final verb, proving the processing difficulty of non-canonical object-initial sentences compared with canonical subject-initial sentences. These results indicate that Korean native speakers build linguistic representation conforming to a canonical sentence in SOV language in the predictive mechanism supporting subject-first strategy but revise the predicted event structure rapidly upon newly encountering input.

Can you touch the N400? The interactive effects of body-object interaction and task demands on N400 amplitudes and decision latencies

The effects of semantic richness on N400 amplitudes remain unclear. Some studies have reported semantic richness evoking greater N400s, whereas others have reported the opposite effect. Moreover, N400 effects of some semantic richness variables, such as body-object interaction (BOI), have yet to be demonstrated. BOI quantifies the degree to which a word's referent is easy to interact with; words such as bicycle are high-BOI whereas words such as butterfly are low-BOI. We examined BOI effects on N400 amplitudes and decision latencies in two semantic tasks. We found that in a touchable/untouchable task, low-BOI words (e.g., butterfly) evoked greater N400s than high-BOI words (e.g., bicycle), but there was no difference in decision latencies. Conversely, in a concrete/abstract task, high and low-BOI words evoked similar N400s, but decision latencies were shorter for high-BOI than for low-BOI words. Our results show that semantic richness upstream and downstream effects are dissociable and task dependent.

Investigating the influence of semantic factors on word retrieval: Reservations, results and recommendations

There is consensus that word retrieval starts with activation of semantic representations. However, in adults without language impairment, relatively little attention has been paid to the effects of the semantic attributes of to-be-retrieved words. This paper, therefore, addresses the question of which item-inherent semantic factors influence word retrieval. Specifically, it reviews the literature on a selection of these factors: imageability, concreteness, number of semantic features, typicality, intercorrelational density, featural distinctiveness, concept distinctiveness, animacy, semantic neighbourhood density, semantic similarity, operativity, valence, and arousal. It highlights several methodological challenges in this field, and has a focus on the insights from studies with people with aphasia where the effects of these variables are more prevalent. The paper concludes that further research simultaneously examining the effects of different semantic factors that are likely to affect lexical co-activation, and the interaction of these variables, would be fruitful, as would suitably scaled computational modelling of these effects in unimpaired language processing and in language impairment. Such research would enable the refinement of theories of semantic processing and word production, and potentially have implications for diagnosis and treatment of semantic and lexical impairments.

Connecting and considering: Electrophysiology provides insights into comprehension

The ability to rapidly and systematically access knowledge stored in long-term memory in response to incoming sensory information-that is, to derive meaning from the world-lies at the core of human cognition. Research using methods that can precisely track brain activity over time has begun to reveal the multiple cognitive and neural mechanisms that make this possible. In this article, I delineate how a process of connecting affords an effortless, continuous infusion of meaning into human perception. In a relatively invariant time window, uncovered through studies using the N400 component of the event-related potential, incoming sensory information naturally induces a graded landscape of activation across long-term semantic memory, creating what might be called "proto-concepts". Connecting can be (but is not always) followed by a process of further considering those activations, wherein a set of more attentionally demanding "active comprehension" mechanisms mediate the selection, augmentation, and transformation of the initial semantic representations. The result is a limited set of more stable bindings that can be arranged in time or space, revised as needed, and brought to awareness. With this research, we are coming closer to understanding how the human brain is able to fluidly link sensation to experience, to appreciate language sequences and event structures, and, sometimes, to even predict what might be coming up next.

A Theoretical Framework for a Hybrid View of the N400

In this study, we present a novel theoretical account of the N400 event-related potential (ERP) component. Hybrid views interpret this ERP component in terms of two cognitive operations: (i) access of information, which is related to predictions (predictability component), and (ii) integration of information, which is related to plausibility (plausibility component). Though there is an empirical evidence for this view, what has been left open so far is how these two operations can be defined. In our approach, both components are related to categorization. The critical word and the argument position it is related to are associated with categories that have a graded structure. This graded structure is defined in terms of weights both on attributes and values of features belonging to a category. The weights, in turn, are defined using probability distributions. The predictability component is defined in terms of the information gain with respect to non mismatched features between the two categories. The plausibility component is defined as the difference in the degree of typicality between the two categories. Finally, the N400 amplitude is defined as a function of both components.

The N400 ERP component reflects an error-based implicit learning signal during language comprehension

The functional significance of the N400 evoked-response component is still actively debated. An increasing amount of theoretical and computational modelling work is built on the interpretation of the N400 as a prediction error. In neural network modelling work, it was proposed that the N400 component can be interpreted as the change in a probabilistic representation of meaning that drives the continuous adaptation of an internal model of the statistics of the environment. These results imply that increased N400 amplitudes should correspond to greater adaptation, which can be measured via implicit memory. To investigate this model derived hypothesis, the current study manipulated expectancy in a sentence reading task to influence N400 amplitudes and subsequently presented the previously expected vs. unexpected words in a perceptual identification task to measure implicit memory. As predicted, reaction times in the perceptual identification task were significantly faster for previously unexpected words that induced larger N400 amplitudes in the previous sentence reading task. Additionally, it could be demonstrated that this adaptation seems to specifically depend on the process underlying N400 amplitudes, as participants with larger N400 differences during sentence reading also exhibited a larger implicit memory benefit in the perceptual identification task. These findings support the interpretation of the N400 as an implicit learning signal driving adaptation in language processing.

Is prediction nothing more than multi-scale pattern completion of the future?

While the notion of the brain as a prediction machine has been extremely influential and productive in cognitive science, there are competing accounts of how best to model and understand the predictive capabilities of brains. One prominent framework is of a "Bayesian brain" that explicitly generates predictions and uses resultant errors to guide adaptation. We suggest that the prediction-generation component of this framework may involve little more than a pattern completion process. We first describe pattern completion in the domain of visual perception, highlighting its temporal extension, and show how this can entail a form of prediction in time. Next, we describe the forward momentum of entrained dynamical systems as a model for the emergence of predictive processing in non-predictive systems. Then, we apply this reasoning to the domain of language, where explicitly predictive models are perhaps most popular. Here, we demonstrate how a connectionist model, TRACE, exhibits hallmarks of predictive processing without any representations of predictions or errors. Finally, we present a novel neural network model, inspired by reservoir computing models, that is entirely unsupervised and memoryless, but nonetheless exhibits prediction-like behavior in its pursuit of homeostasis. These explorations demonstrate that brain-like systems can get prediction "for free," without the need to posit formal logical representations with Bayesian probabilities or an inference machine that holds them in working memory.

Effects of semantic variables on word production in aphasia

Words differ in the complexity of their semantic representations and their relationships to other words and these differences can be operationalised as a variety of semantic variables. The research presented here investigates how word production in aphasia is influenced by six feature-based semantic variables (number of near semantic neighbours, semantic similarity, number of semantic features, typicality, intercorrelational density, and distinctiveness). Previous research has reported inconsistent findings for some of the semantic variables, while others have not been previously studied in aphasia. Spoken picture naming data from a large group of individuals with aphasia with mixed spoken word production impairments (n = 175) and a sub-group who produced few phonological errors (n = 60) was analysed. We examined effects of the semantic variables on overall naming accuracy and on the occurrence of different error types (semantic errors overall, coordinate errors, omissions), while controlling for other psycholinguistic variables using generalised linear mixed effects models and Bayesian correlations. Across analyses, number of semantic features was the most important predictor with a facilitatory main effect on naming accuracy in the sub-group analysis. Number of semantic features, along with typicality and semantic similarity, also predicted error types and in some analyses these effects depended on the integrity of semantic processing. Effects of the semantic variables and their theoretical explanations and implications are discussed in light of previous research and models of word production.

The N400 indexes acquisition of novel emotion concepts via conceptual combination

The ability to learn new emotion concepts is adaptive and socially valuable as it communicates culturally held understandings about values, goals, and experiences. Yet, little work has examined the underlying mechanisms that allow for new emotion concepts and words to be integrated into the conceptual system. One such mechanism may be conceptual combination, or the ability to form novel concepts by dynamically combining previously acquired conceptual knowledge. In this study, we used event-related potentials (ERPs) to investigate the electrophysiological correlates of novel emotion concept acquisition via conceptual combination. Participants were briefly trained on 30 novel emotion combinations, each consisting of two English emotion words (the components; e.g., "sadness + fatigue") and a pseudoword (the target; e.g., "despip"). Participants then completed a semantic congruency task while ERPs were recorded. On each trial, two components were presented serially, followed by a target; participants judged whether the target was a valid combination of the preceding components. Targets could be correct or incorrect trained pseudowords, or new untrained pseudowords. Furthermore, components could be presented in reversed order (e.g., "fatigue" then "sadness") or as synonyms (e.g., "exhaustion" for "fatigue"). Consistent with our main hypotheses, we found a main effect of target, such that the correct combinations showed reduced N400 amplitudes when compared to both incorrect and untrained pseudowords. Critically, this effect held regardless of how the preceding components were presented, suggesting deeper semantic learning. These results extend prior findings on conceptual combination and novel word learning, and are congruent with predictive processing accounts of brain function.

Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity

With strong and valid predictions, grasping a message is easy, whereas more demanding processing is required in the absence of robust expectations. We here demonstrate that brain correlates of the interplay between prediction and perception mechanisms in the understanding of meaningful sentences. Sentence fragments that strongly predict subsequent words induced anticipatory brain activity preceding the expected words; this potential was absent if context did not strongly predict subsequent words. Subjective reports of certainty about upcoming words and objective corpus-based measures correlated with the size of the anticipatory signal, thus establishing its status as a semantic prediction potential (SPP). Crucially, there was an inverse correlation between the SPP and the N400 brain response. The main cortical generators of SPP and N400 were found in inferior prefrontal cortex and posterior temporal cortex, respectively. Interestingly, sentence meaning was reflected by both measures, with additional category-specific sources of SPPs and N400s falling into parieto-temporo-occipital (visual) and frontocentral (sensorimotor) areas for animal- and tool-related words, respectively. These results show that the well-known brain index of semantic comprehension, N400, has an antecedent with different brain localization but similar semantic discriminatory function. We discuss whether N400 dynamics may causally depend on mechanisms underlying SPP size and sources.

Strategic and Non-Strategic Semantic Expectations Hierarchically Modulate Neural Processing

Perception is facilitated by a hierarchy of expectations generated from context and prior knowledge. In auditory processing, violations of local (within-trial) expectations elicit a mismatch negativity (MMN), while violations of global (across-trial) expectations elicit a later positive component (P300). This result is taken as evidence of prediction errors ascending through the expectation hierarchy. However, in language comprehension, there is no evidence that violations of semantic expectations across local-global levels similarly elicit a sequence of hierarchical error signals, thus drawing into question the putative link between event-related potentials (ERPs) and prediction errors. We investigated the neural basis of such hierarchical expectations of semantics in a word-pair priming paradigm. By manipulating the overall proportion of related or unrelated word-pairs across the task, we created two global contexts that differentially encouraged strategic use of primes. Across two experiments, we replicated behavioral evidence of greater priming in the high validity context, reflecting strategic expectations of upcoming targets based on "global" context. In our preregistered EEG analyses, we observed a "local" prediction error ERP effect (i.e., semantic priming) ∼250 ms post-target, which, in exploratory analyses, was followed 100 ms later by a signal that interacted with the global context. However, the later effect behaved in an apredictive manner, i.e., was most extreme for fulfilled expectations, rather than violations. Our results are consistent with interpretations of early ERPs as reflections of prediction error and later ERPs as processes related to conscious access and in support of task demands.

A Comprehension- or a Production-Based Monitor? Response to Roelofs (2020)

Roelofs (2020) has put forth a rebuttal of the criticisms raised against comprehension-based monitoring and has also raised a number of objections against production-based monitors. In this response, I clarify that the model defended by Roelofs is not a comprehension-based monitor, but belongs to a class of monitoring models which I refer to as production-perception models. I review comprehension-based and production-perception models, highlight the strength of each, and point out the differences between them. I then discuss the limitations of both for monitoring production at higher levels, which has been the motivation for production-based monitors. Next, I address the specific criticisms raised by Roelofs (2020) in light of the current evidence. I end by presenting several lines of arguments that preclude a single monitoring mechanism as meeting all the demands of monitoring in a task as complex as communication. A more fruitful avenue is perhaps to focus on what theories are compatible with the nature of representations at specific levels of the production system and with specific aims of monitoring in language production.

To catch a Snitch: Brain potentials reveal variability in the functional organization of (fictional) world knowledge during reading

We harnessed the temporal sensitivity of event-related brain potentials (ERPs) alongside individual differences in Harry Potter (HP) knowledge to investigate the extent to which the availability and timing of information relevant for real-time written word processing are influenced by variation in domain knowledge. We manipulated meaningful (category, event) relationships between sentence fragments about HP stories and their sentence final words. During word-by-word reading, N400 amplitudes to (a) linguistically supported and (b) unsupported but meaningfully related, but not to (c) unsupported, unrelated sentence endings varied with HP domain knowledge. Single-trial analyses revealed that only the N400s to linguistically supported (but not to either type of unsupported) sentence-final words varied as a function of whether individuals knew (or could remember) the correct (supported) ending for each HP "fact." We conclude that the quick availability of information relevant for word understanding in sentences is a function of individuals' knowledge of both specific facts and the domain to which the facts belong. During written sentence processing, as domain knowledge increases, it is clearly evident that individuals can make use of the relevant knowledge systematically organized around themes, events, and categories in that domain, to the extent they have it.

Predictability modulates neurocognitive semantic processing of non-verbal narratives

Predictability is known to modulate semantic processing in language, but it is unclear to what extent this applies for other modalities. Here we ask whether similar cognitive processes are at play in predicting upcoming events in a non-verbal visual narrative. Typically developing adults viewed comics sequences in which a target panel was highly predictable (“high cloze”), less predictable (“low cloze”), or incongruent with the preceding narrative context (“anomalous”) during EEG recording. High and low predictable sequences were determined by a pretest where participants assessed “what happened next?”, resulting in cloze probability scores for sequence outcomes comparable to those used to measure predictability in sentence processing. Through both factorial and correlational analyses, we show a significant modulation of neural responses by cloze such that N400 effects are diminished as a target panel in a comic sequence becomes more predictable. Predictability thus appears to play a similar role in non-verbal comprehension of sequential images as in language comprehension, providing further evidence for the domain generality of semantic processing in the brain.

Neural habituation enhances novelty detection: an EEG study of rapidly presented words

Huber and O'Reilly (2003) proposed that neural habituation aids perceptual processing, separating neural responses to currently viewed objects from recently viewed objects. However, synaptic depression has costs, producing repetition deficits. Prior work confirmed the transition from repetition benefits to deficits with increasing duration of a prime object, but the prediction of enhanced novelty detection was not tested. The current study examined this prediction with a same/different word priming task, using support vector machine (SVM) classification of EEG data, ERP analyses focused on the N400, and dynamic neural network simulations fit to behavioral data to provide a priori predictions of the ERP effects. Subjects made same/different judgements to a response word in relation to an immediately preceding brief target word; prime durations were short (50ms) or long (400ms), and long durations decreased P100/N170 responses to the target word, suggesting that this manipulation increased habituation. Following long duration primes, correct "different" judgments of primed response words increased, evidencing enhanced novelty detection. An SVM classifier predicted trial-by-trial behavior with 66.34% accuracy on held-out data, with greatest predictive power at a time pattern consistent with the N400. The habituation model was augmented with a maintained semantics layer (i.e., working memory) to generate behavior and N400 predictions. A second experiment used response-locked ERPs, confirming the model's assumption that residual activation in working memory is the basis of novelty decisions. These results support the theory that neural habituation enhances novelty detection, and the model assumption that the N400 reflects updating of semantic information in working memory.

Religiosity is associated with less prediction of the typical: An event-related brain potential study

Why are some people more religious than others? According to one hypothesis, people who strongly seek definitive explanations for situations with incomplete information are more likely to be religious. According to a different hypothesis, individuals with smaller "prediction error" responses to unexpected stimuli are more likely to discount evidence contradicting religious beliefs, predisposing them to maintain such beliefs. We sought neurophysiological evidence for these hypotheses using the N400 event-related potential (ERP), which is smaller to more contextually expected stimuli, reflecting prediction of probable completions for meaningful situations. We recorded ERPs from participants viewing category definitions followed by high-typicality category exemplar (HTE), low-typicality exemplar (LTE), or non-exemplar (NE) words. As expected, N400s were largest for NEs, intermediate for LTEs, and smallest for HTEs. Religiosity correlated with smaller N400 amplitude differences between HTEs and both LTEs and NEs. Less strong prediction of probable stimuli based on prior information may predispose to religiosity.

The Neural Time Course of Semantic Ambiguity Resolution in Speech Comprehension

Semantically ambiguous words challenge speech comprehension, particularly when listeners must select a less frequent (subordinate) meaning at disambiguation. Using combined magnetoencephalography (MEG) and EEG, we measured neural responses associated with distinct cognitive operations during semantic ambiguity resolution in spoken sentences: (i) initial activation and selection of meanings in response to an ambiguous word and (ii) sentence reinterpretation in response to subsequent disambiguation to a subordinate meaning. Ambiguous words elicited an increased neural response approximately 400-800 msec after their acoustic offset compared with unambiguous control words in left frontotemporal MEG sensors, corresponding to sources in bilateral frontotemporal brain regions. This response may reflect increased demands on processes by which multiple alternative meanings are activated and maintained until later selection. Disambiguating words heard after an ambiguous word were associated with marginally increased neural activity over bilateral temporal MEG sensors and a central cluster of EEG electrodes, which localized to similar bilateral frontal and left temporal regions. This later neural response may reflect effortful semantic integration or elicitation of prediction errors that guide reinterpretation of previously selected word meanings. Across participants, the amplitude of the ambiguity response showed a marginal positive correlation with comprehension scores, suggesting that sentence comprehension benefits from additional processing around the time of an ambiguous word. Better comprehenders may have increased availability of subordinate meanings, perhaps due to higher quality lexical representations and reflected in a positive correlation between vocabulary size and comprehension success.

Quasi-compositional mapping from form to meaning: a neural network-based approach to capturing neural responses during human language comprehension

We argue that natural language can be usefully described as quasi-compositional and we suggest that deep learning-based neural language models bear long-term promise to capture how language conveys meaning. We also note that a successful account of human language processing should explain both the outcome of the comprehension process and the continuous internal processes underlying this performance. These points motivate our discussion of a neural network model of sentence comprehension, the Sentence Gestalt model, which we have used to account for the N400 component of the event-related brain potential (ERP), which tracks meaning processing as it happens in real time. The model, which shares features with recent deep learning-based language models, simulates N400 amplitude as the automatic update of a probabilistic representation of the situation or event described by the sentence, corresponding to a temporal difference learning signal at the level of meaning. We suggest that this process happens relatively automatically, and that sometimes a more-controlled attention-dependent process is necessary for successful comprehension, which may be reflected in the subsequent P600 ERP component. We relate this account to current deep learning models as well as classic linguistic theory, and use it to illustrate a domain general perspective on some specific linguistic operations postulated based on compositional analyses of natural language. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Speed-Accuracy Tradeoffs in Brain and Behavior: Testing the Independence of P300 and N400 Related Processes in Behavioral Responses to Sentence Categorization

Although the N400 was originally discovered in a paradigm designed to elicit a P300 (Kutas and Hillyard, 1980), its relationship with the P300 and how both overlapping event-related potentials (ERPs) determine behavioral profiles is still elusive. Here we conducted an ERP (N = 20) and a multiple-response speed-accuracy tradeoff (SAT) experiment (N = 16) on distinct participant samples using an antonym paradigm (The opposite of black is white/nice/yellow with acceptability judgment). We hypothesized that SAT profiles incorporate processes of task-related decision-making (P300) and stimulus-related expectation violation (N400). We replicated previous ERP results (Roehm et al., 2007): in the correct condition (white), the expected target elicits a P300, while both expectation violations engender an N400 [reduced for related (yellow) vs. unrelated targets (nice)]. Using multivariate Bayesian mixed-effects models, we modeled the P300 and N400 responses simultaneously and found that correlation between residuals and subject-level random effects of each response window was minimal, suggesting that the components are largely independent. For the SAT data, we found that antonyms and unrelated targets had a similar slope (rate of increase in accuracy over time) and an asymptote at ceiling, while related targets showed both a lower slope and a lower asymptote, reaching only approximately 80% accuracy. Using a GLMM-based approach (Davidson and Martin, 2013), we modeled these dynamics using response time and condition as predictors. Replacing the predictor for condition with the averaged P300 and N400 amplitudes from the ERP experiment, we achieved identical model performance. We then examined the piecewise contribution of the P300 and N400 amplitudes with partial effects (see Hohenstein and Kliegl, 2015). Unsurprisingly, the P300 amplitude was the strongest contributor to the SAT-curve in the antonym condition and the N400 was the strongest contributor in the unrelated condition. In brief, this is the first demonstration of how overlapping ERP responses in one sample of participants predict behavioral SAT profiles of another sample. The P300 and N400 reflect two independent but interacting processes and the competition between these processes is reflected differently in behavioral parameters of speed and accuracy.

A Central Component of the N1 Event-Related Brain Potential Could Index the Early and Automatic Inhibition of the Actions Systematically Activated by Objects

Stimuli of the environment, like objects, systematically activate the actions they are associated to. These activations occur extremely fast. Nevertheless, behavioral data reveal that, in most cases, these activations are then automatically inhibited, around 100 ms after the occurrence of the stimulus. We thus tested whether this early inhibition could be indexed by a central component of the N1 event-related brain potential (ERP). To achieve that goal, we looked at whether this ERP component is larger in tasks that could increase the inhibition and in trials where reaction times (RTs) happen to be long. The illumination of a real space bar of a keyboard out of the dark was used as a stimulus. To maximize the modulation of the inhibition, the task participants had to perform was manipulated across blocks. A look-only task and a count task were used to increase inhibition and an immediate press task was used to decrease it. ERPs of the two block-conditions where presses had to be prevented and where the largest central N1s were predicted were compared to those elicited in the press task, differentiating the ERPs to the third of the trials where presses were the slowest from the ERPs to the third of the trials with the fastest presses. Despite larger negativities due to lateralized readiness potentials (LRPs) and despite greater attention likely in immediate press-trials, central N1s were found to be minimal for the fastest presses, intermediate for the slowest ones and maximal for the two no-press conditions. These results thus provide a strong support for the idea that the central N1 indexes an early and short lasting automatic inhibition of the actions systematically activated by objects. They also confirm that the strength of this automatic inhibition spontaneously fluctuates across trials and tasks. On the other hand, just before N1s, parietal P1s were found larger for fastest presses. They might thus index the initial activation of these actions. Finally, consistent with the idea that N300s index late inhibition processes, that occur preferentially when the task requires them, these ERPs were quasi absent for fast presses trials and much larger in the three other conditions.

Modeling early lexico-semantic network development: Perceptual features matter most

What aspects of word meaning are important in early word learning and lexico-semantic network development? Adult lexico-semantic systems flexibly encode multiple types of semantic features, including functional, perceptual, taxonomic, and encyclopedic. However, various theoretical accounts of lexical development differ on whether and how these semantic properties of word meanings are initially encoded into young children's emerging lexico-semantic networks. Whereas some accounts highlight the importance of early perceptual versus conceptual properties, others posit that thematic or functional aspects of word meaning are primary relative to taxonomic knowledge. We seek to shed light on these debates with 2 modeling studies that explore patterns in early word learning using a large database of early vocabulary in 5,450 children, and a newly developed set of semantic features of early acquired nouns. In Study 1, we ask whether semantic properties of early acquired words relate to order in which these words are typically learned; Study 2 models normative lexico-semantic noun-feature network development compared to random network growth. Both studies provide converging evidence that perceptual properties of word meanings play a key role in early word learning and lexico-semantic network development. The findings lend support to theoretical accounts of language learning that highlight the importance of the child's perceptual experience. (PsycINFO Database Record (c) 2019 APA, all rights reserved).