Thinking ahead: the role and roots of prediction in language comprehension

Prediction in reading: A review of predictability effects, their theoretical implications, and beyond

Historically, prediction during reading has been considered an inefficient and cognitively expensive processing mechanism given the inherently generative nature of language, which allows upcoming text to unfold in an infinite number of possible ways. This article provides an accessible and comprehensive review of the psycholinguistic research that, over the past 40 or so years, has investigated whether readers are capable of generating predictions during reading, typically via experiments on the effects of predictability (i.e., how well a word can be predicted from its prior context). Five theoretically important issues are addressed: What is the best measure of predictability? What is the functional relationship between predictability and processing difficulty? What stage(s) of processing does predictability affect? Are predictability effects ubiquitous? What processes do predictability effects actually reflect? Insights from computational models of reading about how predictability manifests itself to facilitate the reading of text are also discussed. This review concludes by arguing that effects of predictability can, to a certain extent, be taken as demonstrating evidence that prediction is an important but flexible component of real-time language comprehension, in line with broader predictive accounts of cognitive functioning. However, converging evidence, especially from concurrent eye-tracking and brain-imaging methods, is necessary to refine theories of prediction.

Oral reading promotes predictive processing in Chinese sentence reading: eye movement evidence

Background

Fluent sentence reading is widely acknowledged to depend on top-down contextual prediction, wherein sentential and contextual cues guide the pre-activation of linguistic representations before encountering stimuli, facilitating subsequent comprehension. The Prediction-by-Production hypothesis posits an explanation for predictive processes in language comprehension, suggesting that prediction during comprehension involves processes associated with language production. However, there is a lack of eye movement evidence supporting this hypothesis within sentence reading contexts. Thus, we manipulated reading mode and word predictability to examine the influence of language production on predictive processing.

Methods

Participants engaged in silent or oral reading of sentences containing either high or low-predictable target words. Eye movements were recorded using the Eyelink1000 eye tracker.

Results

The findings revealed a higher skipping rate and shorter fixation times for high-predictable words compared to low-predictable ones, and for silent compared to oral reading. Notably, interactive effects were observed in the time measures (FFD, SFD, GD) during first-pass reading, indicating that word predictability effects were more pronounced during oral reading than silent reading.

Discussion

The observed pattern of results suggests that the activation of the production system enhances predictive processing during the early lexical access, providing empirical support for the Prediction-by-Production hypothesis in eye movement sentence reading situations, extending the current understanding of the timing and nature of predictions in reading comprehension.

I know how you'll say it: evidence of speaker-specific speech prediction

Most models of language comprehension assume that the linguistic system is able to pre-activate phonological information. However, the evidence for phonological prediction is mixed and controversial. In this study, we implement a paradigm that capitalizes on the fact that foreign speakers usually make phonological errors. We investigate whether speaker identity (native vs. foreign) is used to make specific phonological predictions. Fifty-two participants were recruited to read sentence frames followed by a last spoken word which was uttered by either a native or a foreign speaker. They were required to perform a lexical decision on the last spoken word, which could be either semantically predictable or not. Speaker identity (native vs. foreign) may or may not be cued by the face of the speaker. We observed that the face cue is effective in speeding up the lexical decision when the word is predictable, but it is not effective when the word is not predictable. This result shows that speech prediction takes into account the phonological variability between speakers, suggesting that it is possible to pre-activate in a detailed and specific way the phonological representation of a predictable word.

Event-related potentials and behavioral correlates of emotional recognition memory in late pregnancy

PURPOSE

Research on cognitive and emotional functions during pregnancy challenges the prevalent perception of cognitive decline in pregnant women. This study investigates the behavioral and neural dynamics of cognitive-affective processing in third-trimester pregnant women, comparing them with non-pregnant controls.

METHODS

Using a 64-channel EEG-ERP system, we recorded brain activity as participants engaged in an emotional word recognition task. This task involved initially viewing a sequence of emotional and neutral words, followed by a recognition test where participants identified each word as 'new' or 'previously seen'.

RESULTS

Contrary to widespread beliefs about diminished recognition ability during late pregnancy, our results revealed no significant differences in error rates between groups. However, pregnant participants demonstrated slower reaction times. In terms of neural responses, pregnant women exhibited increased amplitudes in the N1, P2, and N400 ERP components, suggesting that they may require additional brain resources compared with non-pregnant individuals to process perceptual information. A significant interaction was observed between pregnancy status and the emotional valence of stimuli. Pregnant women showed heightened N1 and N400 responses to negative words, indicating increased sensitivity to stimuli potentially representing threat. This enhanced response was not observed for positive or neutral words. Furthermore, there was an amplified N1 response to 'new' words, but not to 'old' words.

CONCLUSION

These findings suggest that late pregnancy is characterized by heightened responsiveness to new and particularly negative stimuli, potentially leading to a more cautious behavioral approach. Heightened vigilance and sensitivity could offer evolutionary advantages, optimizing fetal development and enhancing maternal well-being.

Exploring Pragmatic Factors on the Logical Relationships of Conditional Reasoning: A Study of Counterfactual and Hypothetical Conditionals

Previous theories have established the mental model activation of processing different types of conditionals, stating that counterfactual conditionals expressing events that contradict known facts (e.g., "If it had rained, then they would not go to the park.") are considered to trigger two mental models: (1) a hypothetical but factually wrong model (e.g., "rain" and "did not go to the park") and (2) a corresponding real-world model (e.g., "did not rain" and "went to the park"). This study aimed to investigate whether pragmatic factors differentially influence readers' comprehension and distinction between counterfactual and hypothetical conditional sentences in Mandarin Chinese. Participants were required to read and judge the comprehensibility of Chinese hypothetical and counterfactual conditionals, which were different in temporal indicators (past vs. future temporal indicators) in the antecedent. Different polarities (with vs. without negators) and different moving directions (different directional verbs: lai2 [come] vs. qu4 [go]) in the consequent were also manipulated. Linear mixed-effects models (LMEM) revealed that hypothetical conditionals (with future temporal indicators) were more comprehensible than counterfactual conditionals (with past temporal indicators). The semantic similarities within the subordinate clause revealed future temporal indicators had higher lexical-semantic co-occurrence than past indicators, suggesting that temporal indicators impact comprehension partly through lexical semantics in the premise, and hypothetical conditionals are more easily processed. However, the semantic similarity analysis of the main and the subordinate clauses showed no effect of temporal indicators, suggesting that lexical-semantic co-occurrence across clauses may not substantially contribute to the distinction between hypothetical conditionals and counterfactual conditionals. In conclusion, this study offers insights into the comprehension of Chinese conditional sentences by shedding light on the pragmatic factors influencing the activation of different mental models.

Understanding words in context: A naturalistic EEG study of children's lexical processing

When listening to speech, adults rely on context to anticipate upcoming words. Evidence for this comes from studies demonstrating that the N400, an event-related potential (ERP) that indexes ease of lexical-semantic processing, is influenced by the predictability of a word in context. We know far less about the role of context in children's speech comprehension. The present study explored lexical processing in adults and 5-10-year-old children as they listened to a story. ERPs time-locked to the onset of every word were recorded. Each content word was coded for frequency, semantic association, and predictability. In both children and adults, N400s reflect word predictability, even when controlling for frequency and semantic association. These findings suggest that both adults and children use top-down constraints from context to anticipate upcoming words when listening to stories.

Neural Tracking of Speech Acoustics in Noise Is Coupled with Lexical Predictability as Estimated by Large Language Models

Adults heard recordings of two spatially separated speakers reading newspaper and magazine articles. They were asked to listen to one of them and ignore the other, and EEG was recorded to assess their neural processing. Machine learning extracted neural sources that tracked the target and distractor speakers at three levels: the acoustic envelope of speech (delta- and theta-band modulations), lexical frequency for individual words, and the contextual predictability of individual words estimated by GPT-4 and earlier lexical models. To provide a broader view of speech perception, half of the subjects completed a simultaneous visual task, and the listeners included both native and non-native English speakers. Distinct neural components were extracted for these levels of auditory and lexical processing, demonstrating that native English speakers had greater target-distractor separation compared with non-native English speakers on most measures, and that lexical processing was reduced by the visual task. Moreover, there was a novel interaction of lexical predictability and frequency with auditory processing; acoustic tracking was stronger for lexically harder words, suggesting that people listened harder to the acoustics when needed for lexical selection. This demonstrates that speech perception is not simply a feedforward process from acoustic processing to the lexicon. Rather, the adaptable context-sensitive processing long known to occur at a lexical level has broader consequences for perception, coupling with the acoustic tracking of individual speakers in noise.

The role of general cognitive skills in integrating visual and linguistic information during sentence comprehension: individual differences across the lifespan

Individuals exhibit massive variability in general cognitive skills that affect language processing. This variability is partly developmental. Here, we recruited a large sample of participants (N = 487), ranging from 9 to 90 years of age, and examined the involvement of nonverbal processing speed (assessed using visual and auditory reaction time tasks) and working memory (assessed using forward and backward Digit Span tasks) in a visual world task. Participants saw two objects on the screen and heard a sentence that referred to one of them. In half of the sentences, the target object could be predicted based on verb-selectional restrictions. We observed evidence for anticipatory processing on predictable compared to non-predictable trials. Visual and auditory processing speed had main effects on sentence comprehension and facilitated predictive processing, as evidenced by an interaction. We observed only weak evidence for the involvement of working memory in predictive sentence comprehension. Age had a nonlinear main effect (younger adults responded faster than children and older adults), but it did not differentially modulate predictive and non-predictive processing, nor did it modulate the involvement of processing speed and working memory. Our results contribute to delineating the cognitive skills that are involved in language-vision interactions.

Registered report: Age-preserved semantic memory and the CRUNCH effect manifested as differential semantic control networks: An fMRI study

Semantic memory representations are generally well maintained in aging, whereas semantic control is thought to be more affected. To explain this phenomenon, this study tested the predictions of the Compensation-Related Utilization of Neural Circuits Hypothesis (CRUNCH), focusing on task demands in aging as a possible framework. The CRUNCH effect would manifest itself in semantic tasks through a compensatory increase in neural activation in semantic control network regions but only up to a certain threshold of task demands. This study compares 39 younger (20-35 years old) with 39 older participants (60-75 years old) in a triad-based semantic judgment task performed in an fMRI scanner while manipulating task demand levels (low versus high) through semantic distance. In line with the CRUNCH predictions, differences in neurofunctional activation and behavioral performance (accuracy and response times) were expected in younger versus older participants in the low- versus high-demand conditions, which should be manifested in semantic control Regions of Interest (ROIs). Our older participants had intact behavioral performance, as proposed in the literature for semantic memory tasks (maintained accuracy and slower response times (RTs)). Age-invariant behavioral performance in the older group compared to the younger one is necessary to test the CRUNCH predictions. The older adults were also characterized by high cognitive reserve, as our neuropsychological tests showed. Our behavioral results confirmed that our task successfully manipulated task demands: error rates, RTs and perceived difficulty increased with increasing task demands in both age groups. We did not find an interaction between age group and task demand, or a statistically significant difference in activation between the low- and high-demand conditions for either RTs or accuracy. As for brain activation, we did not find the expected age group by task demand interaction, or a significant main effect of task demand. Overall, our results are compatible with some neural activation in the semantic network and the semantic control network, largely in frontotemporoparietal regions. ROI analyses demonstrated significant effects (but no interactions) of task demand in the left and right inferior frontal gyrus, the left posterior middle temporal gyrus, the posterior inferior temporal gyrus and the prefrontal gyrus. Overall, our test did not confirm the CRUNCH predictions.

The Costs (and Benefits?) of Effortful Listening for Older Adults: Insights from Simultaneous Electrophysiology, Pupillometry, and Memory

Although the impact of acoustic challenge on speech processing and memory increases as a person ages, older adults may engage in strategies that help them compensate for these demands. In the current preregistered study, older adults (n = 48) listened to sentences-presented in quiet or in noise-that were high constraint with either expected or unexpected endings or were low constraint with unexpected endings. Pupillometry and EEG were simultaneously recorded, and subsequent sentence recognition and word recall were measured. Like young adults in prior work, we found that noise led to increases in pupil size, delayed and reduced ERP responses, and decreased recall for unexpected words. However, in contrast to prior work in young adults where a larger pupillary response predicted a recovery of the N400 at the cost of poorer memory performance in noise, older adults did not show an associated recovery of the N400 despite decreased memory performance. Instead, we found that in quiet, increases in pupil size were associated with delays in N400 onset latencies and increased recognition memory performance. In conclusion, we found that transient variation in pupil-linked arousal predicted trade-offs between real-time lexical processing and memory that emerged at lower levels of task demand in aging. Moreover, with increased acoustic challenge, older adults still exhibited costs associated with transient increases in arousal without the corresponding benefits.

Predictive language processing: integrating comprehension and production, and what atypical populations can tell us

Predictive processing, a crucial aspect of human cognition, is also relevant for language comprehension. In everyday situations, we exploit various sources of information to anticipate and therefore facilitate processing of upcoming linguistic input. In the literature, there are a variety of models that aim at accounting for such ability. One group of models propose a strict relationship between prediction and language production mechanisms. In this review, we first introduce very briefly the concept of predictive processing during language comprehension. Secondly, we focus on models that attribute a prominent role to language production and sensorimotor processing in language prediction ("prediction-by-production" models). Contextually, we provide a summary of studies that investigated the role of speech production and auditory perception on language comprehension/prediction tasks in healthy, typical participants. Then, we provide an overview of the limited existing literature on specific atypical/clinical populations that may represent suitable testing ground for such models-i.e., populations with impaired speech production and auditory perception mechanisms. Ultimately, we suggest a more widely and in-depth testing of prediction-by-production accounts, and the involvement of atypical populations both for model testing and as targets for possible novel speech/language treatment approaches.

A predictive coding model of the N400

The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.

Neurophysiological signatures of prediction in language: A critical review of anticipatory negativities

Recent event-related potential (ERP) studies in language comprehension converge in finding anticipatory negativities preceding words or word segments that can be pre-activated based on either sentence contexts or phonological cues. We review these findings from different paradigms in the light of evidence from other cognitive domains in which slow negative potentials have long been associated with anticipatory processes and discuss their potential underlying mechanisms. We propose that this family of anticipatory negativities captures common mechanisms associated with the pre-activation of linguistic information both within words and within sentences. Future studies could utilize these anticipatory negativities in combination with other, well-established ERPs, to simultaneously track prediction-related processes emerging at different time intervals (before and after the perception of pre-activated input) and with distinct time courses (shorter-lived and longer-lived cognitive operations).

Used to Be a Dime, Now It's a Dollar: Revised Speech Perception in Noise Key Word Predictability Revisited 40 Years On

PURPOSE

Almost 40 years after its development, in this article, we reexamine the relevance and validity of the ubiquitously used Revised Speech Perception in Noise (R-SPiN) sentence corpus. The R-SPiN corpus includes "high-context" and "low-context" sentences and has been widely used in the field of hearing research to examine the benefit derived from semantic context across English-speaking listeners, but research investigating age differences has yielded somewhat inconsistent findings. We assess the appropriateness of the corpus for use today in different English-language cultures (i.e., British and American) as well as for older and younger adults.

METHOD

Two hundred forty participants, including older (60-80 years) and younger (19-31 years) adult groups in the the United Kingdom and United States, completed a cloze task consisting of R-SPiN sentences with the final word removed. Cloze, as a measure of predictability, and entropy, as a measure of response uncertainty, were compared between culture and age groups.

RESULTS

Most critically, of the 200 "high-context" stimuli, only around half were assessed as highly predictable for older adults (United Kingdom: 109; United States: 107); and fewer still, for younger adults (United Kingdom: 75; United States: 81). We also found dominant responses to these "high-context" stimuli varied between cultures, with U.S. responses being more likely to match the original R-SPiN target.

CONCLUSIONS

Our findings highlight the issue of incomplete transferability of corpus items across English-language cultures as well as diminished equivalency for older and younger adults. By identifying relevant items for each population, this work could facilitate the interpretation of inconsistent findings in the literature, particularly relating to age effects.

Dissociating the pre-activation of word meaning and form during sentence comprehension: Evidence from EEG representational similarity analysis

During language comprehension, the processing of each incoming word is facilitated in proportion to its predictability. Here, we asked whether anticipated upcoming linguistic information is actually pre-activated before new bottom-up input becomes available, and if so, whether this pre-activation is limited to the level of semantic features, or whether extends to representations of individual word-forms (orthography/phonology). We carried out Representational Similarity Analysis on EEG data while participants read highly constraining sentences. Prior to the onset of the expected target words, sentence pairs predicting semantically related words (financial "bank" - "loan") and form-related words (financial "bank" - river "bank") produced more similar neural patterns than pairs predicting unrelated words ("bank" - "lesson"). This provides direct neural evidence for item-specific semantic and form predictive pre-activation. Moreover, the semantic pre-activation effect preceded the form pre-activation effect, suggesting that top-down pre-activation is propagated from higher to lower levels of the linguistic hierarchy over time.

Effects of anxiety state on N400 event-related brain potential response to unexpected semantic stimuli

Emotional states can influence how people use meaningful context to make predictions about what comes next. To measure whether state anxiety influences such prediction, we used the N400 event-related brain potential (ERP) response to semantic stimuli, whose amplitude is smaller (less negative) when the stimulus is more predicted based on preceding context. Participants (n = 28) were randomized to one of two groups, who underwent either an "anxious-uncertainty" procedure previously shown to increase anxiety, or a control procedure. Both before and after this procedure, participants' ERPs were recorded while they viewed category definitions (e.g., "a type of fruit"), each followed by a target word that was either a high-typicality category exemplar ("apple"), low-typicality exemplar ("cherry"), or non-exemplar ("clamp") of the category. Participants' task was to respond by pressing one of two buttons to indicate whether the target represented a member of the category. As expected, based on previous work, overall, N400 amplitudes were largest (most negative) in response to non-exemplars, intermediate to low-typicality exemplars, and smallest to high-typicality exemplars. N400 amplitudes were larger to non-exemplars after the anxious-uncertainty procedure than after the control procedure. N400 amplitudes to both types of exemplars did not differ after the anxious-uncertainty procedure versus the control procedure. The results are consistent with participants devoting more neural resources to processing contextually unexpected items under anxious states, rather than anxiety facilitating processing of expected items.

Characterizing directional dynamics of semantic prediction based on inter-regional temporal generalization

The event-related potential/field component N400(m) has been widely used as a neural index for semantic prediction. It has long been hypothesized that feedback information from inferior frontal areas plays a critical role in generating the N400. However, due to limitations in causal connectivity estimation, direct testing of this hypothesis has remained difficult. Here, magnetoencephalography (MEG) data was obtained during a classic N400 paradigm where the semantic predictability of a fixed target noun was manipulated in simple German sentences. To estimate causality, we implemented a novel approach based on machine learning and temporal generalization to estimate the effect of inferior frontal gyrus (IFG) on temporal areas. In this method, a support vector machine (SVM) classifier is trained on each time point of the neural activity in IFG to classify less predicted (LP) and highly predicted (HP) nouns and then tested on all time points of superior/middle temporal sub-regions activity (and vice versa, to establish spatio-temporal evidence for or against causality). The decoding accuracy was significantly above chance level when the classifier was trained on IFG activity and tested on future activity in superior and middle temporal gyrus (STG/MTG). The results present new evidence for a model predictive speech comprehension where predictive IFG activity is fed back to shape subsequent activity in STG/MTG, implying a feedback mechanism in N400 generation. In combination with the also observed strong feedforward effect from left STG/MTG to IFG, our findings provide evidence of dynamic feedback and feedforward influences between IFG and temporal areas during N400 generation.

Individual word representations dissociate from linguistic context along a cortical unimodal to heteromodal gradient

Language comprehension involves multiple hierarchical processing stages across time, space, and levels of representation. When processing a word, the sensory input is transformed into increasingly abstract representations that need to be integrated with the linguistic context. Thus, language comprehension involves both input-driven as well as context-dependent processes. While neuroimaging research has traditionally focused on mapping individual brain regions to the distinct underlying processes, recent studies indicate that whole-brain distributed patterns of cortical activation might be highly relevant for cognitive functions, including language. One such pattern, based on resting-state connectivity, is the 'principal cortical gradient', which dissociates sensory from heteromodal brain regions. The present study investigated the extent to which this gradient provides an organizational principle underlying language function, using a multimodal neuroimaging dataset of functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) recordings from 102 participants during sentence reading. We found that the brain response to individual representations of a word (word length, orthographic distance, and word frequency), which reflect visual; orthographic; and lexical properties, gradually increases towards the sensory end of the gradient. Although these properties showed opposite effect directions in fMRI and MEG, their association with the sensory end of the gradient was consistent across both neuroimaging modalities. In contrast, MEG revealed that properties reflecting a word's relation to its linguistic context (semantic similarity and position within the sentence) involve the heteromodal end of the gradient to a stronger extent. This dissociation between individual word and contextual properties was stable across earlier and later time windows during word presentation, indicating interactive processing of word representations and linguistic context at opposing ends of the principal gradient. To conclude, our findings indicate that the principal gradient underlies the organization of a range of linguistic representations while supporting a gradual distinction between context-independent and context-dependent representations. Furthermore, the gradient reveals convergent patterns across neuroimaging modalities (similar location along the gradient) in the presence of divergent responses (opposite effect directions).

It takes a village: A multi-brain approach to studying multigenerational family communication

Grandparents play a critical role in child rearing across the globe. Yet, there is a shortage of neurobiological research examining the relationship between grandparents and their grandchildren. We employ multi-brain neurocomputational models to simulate how changes in neurophysiological processes in both development and healthy aging affect multigenerational inter-brain coupling - a neural marker that has been linked to a range of socio-emotional and cognitive outcomes. The simulations suggest that grandparent-child interactions may be paired with higher inter-brain coupling than parent-child interactions, raising the possibility that the former may be more advantageous under certain conditions. Critically, this enhancement of inter-brain coupling for grandparent-child interactions is more pronounced in tri-generational interactions that also include a parent, which may speak to findings that grandparent involvement in childrearing is most beneficial if the parent is also an active household member. Together, these findings underscore that a better understanding of the neurobiological basis of cross-generational interactions is vital, and that such knowledge can be helpful in guiding interventions that consider the whole family. We advocate for a community neuroscience approach in developmental social neuroscience to capture the diversity of child-caregiver relationships in real-world settings.

Good enough processing: what have we learned in the 20 years since Ferreira et al. (2002)?

Traditionally, language processing has been thought of in terms of complete processing of the input. In contrast to this, Ferreira and colleagues put forth the idea of good enough processing. The proposal was that during everyday processing, ambiguities remain unresolved, we rely on heuristics instead of full analyses, and we carry out deep processing only if we need to for the task at hand. This idea has gathered substantial traction since its conception. In the current work, I review the papers that have tested the three key claims of good enough processing: ambiguities remain unresolved and underspecified, we use heuristics to parse sentences, and deep processing is only carried out if required by the task. I find mixed evidence for these claims and conclude with an appeal to further refinement of the claims and predictions of the theory.

The Impact of Linguistic Prediction Violations on Downstream Recognition Memory and Sentence Recall

Predicting upcoming words during language comprehension not only affects processing in the moment but also has consequences for memory, although the source of these memory effects (e.g., whether driven by lingering pre-activations, re-analysis following prediction violations, or other mechanisms) remains underspecified. Here, we investigated downstream impacts of prediction on memory in two experiments. First, we recorded EEG as participants read strongly and weakly constraining sentences with expected, unexpected but plausible, or semantically anomalous endings ("He made a holster for his gun / father / train") and were tested on their recognition memory for the sentence endings. Participants showed similar rates of false alarms for predicted but never presented sentence endings whether the prediction violation was plausible or anomalous, suggesting that these arise from pre-activation of the expected words during reading. During sentence reading, especially in strongly constraining sentences, plausible prediction violations elicited an anterior positivity; anomalous endings instead elicited a posterior positivity, whose amplitude was predictive of later memory for those anomalous words. ERP patterns at the time of recognition differentiated plausible and anomalous sentence endings: Words that had been plausible prediction violations elicited enhanced late positive complex amplitudes, suggesting greater episodic recollection, whereas anomalous sentence endings elicited greater N1 amplitudes, suggesting attentional tagging. In a follow-up behavioral study, a separate group of participants read the same sentence stimuli and were tested for sentence-level recall. We found that recall of full sentences was impaired when sentences ended with a prediction violation. Taken together, the results suggest that prediction violations draw attention and affect encoding of the violating word, in a manner that depends on plausibility, and that this, in turn, may impair future memory of the gist of the sentence.

Conditionals in context: Brain signatures of prediction in discourse processing

Comprehenders are known to generate expectations about upcoming linguistic input at the sentence and discourse level. However, most previous studies on prediction focused mainly on word-induced brain activity rather than examining neural activity preceding a critical stimulus in discourse processing, where prediction actually takes place. In this EEG study, participants were presented with multiple sentences resembling a discourse including conditional sentences with either only if or if, which are characterized by different semantics, triggering stronger or weaker predictions about the possible continuation of the presented discourses, respectively. Results revealed that discourses including only if, as compared to discourses with bare if, triggered an increased predictive neural activity before the expected critical word, resembling the readiness potential. Moreover, word-induced P300 brain responses were found to be enhanced by unpredictable discourse continuations and reduced in predictable discourse continuations. Intriguingly, brain responses preceding and following the critical word were found to be correlated, which yields evidence for predictive activity modulating word-induced processing on the discourse level. These findings shed light on the predictive nature of neural processes at the discourse level, critically advancing our understanding of the functional interconnection between discourse understanding and prediction processes in brain and mind.

Temporal Shift Length and Antecedent Occurrence Likelihood Modulate Counterfactual Conditional Comprehension: Evidence from Event-Related Potentials

Counterfactual conditionals posit hypothetical scenarios in which antecedent events contradict reality. This study examined whether and how the processing difficulty of Chinese counterfactual conditionals (yaobushi, equivalent to if it had not been for in English) can be affected by the length of temporal shifts of the events across clauses and the likelihood of the antecedent occurrence. Participants read Chinese counterfactuals that contained either long (e.g., qunian-xianzai [last year-right now]) or short temporal shifts (e.g., zuotian-xianzai [yesterday-right now]) within highly likely (e.g., sign up for school activity) or less likely contexts (e.g., sign up for Arctic scientific research). ERP results revealed a significant N400 interaction between the temporal shift length and antecedent likelihood on the temporal indicators in the consequent and the sentence-ending verbs. Specifically, the less likely events elicited larger negativity than highly likely events with short temporal shifts on the temporal indicator. On the sentence-ending word, the long temporal shift elicited enlarged negativity than the short temporal shift when the antecedent was highly likely. These findings have two key implications regarding the interplay of implied causality and falsity constraints during counterfactual comprehension. First, salient falsity constraints can override effects of causal coherence on processing. Second, greater negativity for unlikely antecedents suggests that counterfactual markers concurrently activate factual and hypothetical representations.

Modulating Complex Sentence Processing in Aphasia Through Attention and Semantic Networks

PURPOSE

Lexical processing impairments such as delayed and reduced activation of lexical-semantic information have been linked to syntactic processing disruptions and sentence comprehension deficits in individuals with aphasia (IWAs). Lexical-level deficits can also preclude successful lexical encoding during sentence processing and amplify the processing costs of similarity-based interference during syntactic retrieval. We investigate whether two manipulations to engage attention and pre-activate semantic features of a target (to-be-retrieved) noun will (a) boost lexical activation during initial lexical encoding and (b) facilitate syntactic dependency linking through improved resolution of interference in IWAs and neurologically unimpaired age-matched controls (AMCs).

METHOD

Eye-tracking-while-listening with a visual world paradigm was used to investigate whether semantic and attentional manipulations modulated initial lexical processing and downstream syntactic retrieval of the direct-object noun in object-relative sentences.

RESULTS

In the attention and semantic manipulations, the AMC group showed no changes in initial lexical access levels; however, gaze patterns revealed clear facilitations in dependency linking and interference resolution. In the IWA group, the attentional cue increased and maintained activation of N1 with modest facilitations in dependency linking. In the semantic condition, IWA results showed a greater degree of facilitation during dependency linking.

CONCLUSIONS

The results suggest that attention and semantic activation are parameters that may be manipulated to strengthen encoding of lexical representations to facilitate retrieval (i.e., dependency linking) and mitigate similarity-based interference. In IWAs, these manipulations may help to reduce lexical processing deficits that can preclude successful encoding.

Multiple predictions during language comprehension: Friends, foes, or indifferent companions?

To comprehend language, we continually use prior context to pre-activate expected upcoming information, resulting in facilitated processing of incoming words that confirm these predictions. But what are the consequences of disconfirming prior predictions? To address this question, most previous studies have examined unpredictable words appearing in contexts that constrain strongly for a single continuation. However, during natural language processing, it is far more common to encounter contexts that constrain for multiple potential continuations, each with some probability. Here, we ask whether and how pre-activating both higher and lower probability alternatives influences the processing of the lower probability incoming word. One possibility is that, similar to language production, there is continuous pressure to select the higher-probability pre-activated alternative through competitive inhibition. During comprehension, this would result in relative costs in processing the lower probability target. A second possibility is that if the two pre-activated alternatives share semantic features, they mutually enhance each other's pre-activation. This would result in greater facilitation in processing the lower probability target. To distinguish between these accounts, we recorded ERPs as participants read three-sentence scenarios that constrained either for a single word or for two potential continuations - a higher probability expected candidate and a lower probability second-best candidate. We found no evidence that competitive pre-activation between the expected and second-best candidates resulted in costs in processing the second-best target, either during lexico-semantic processing (indexed by the N400) or at later stages of processing (indexed by a later frontal positivity). Instead, we found only benefits of pre-activating multiple alternatives, with evidence of enhanced graded facilitation on lower-probability targets that were semantically related to a higher-probability pre-activated alternative. These findings are consistent with a previous eye-tracking study by Luke and Christianson (2016, Cogn Psychol) using corpus-based materials. They have significant theoretical implications for models of predictive language processing, indicating that routine graded prediction in language comprehension does not operate through the same competitive mechanisms that are engaged in language production. Instead, our results align more closely with hierarchical probabilistic accounts of language comprehension, such as predictive coding.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

How does linguistic context influence word learning?

While there are well-known demonstrations that children can use distributional information to acquire multiple components of language, the underpinnings of these achievements are unclear. In the current paper, we investigate the potential pre-requisites for a distributional learning model that can explain how children learn their first words. We review existing literature and then present the results of a series of computational simulations with Vector Space Models, a type of distributional semantic model used in Computational Linguistics, which we evaluate against vocabulary acquisition data from children. We focus on nouns and verbs, and we find that: (i) a model with flexibility to adjust for the frequency of events provides a better fit to the human data, (ii) the influence of context words is very local, especially for nouns, and (iii) words that share more contexts with other words are harder to learn.

Expectation adaptation for rare cadences in music: Item order matters in repetition priming

Humans make predictions about future events in many domains, including when they listen to music. Previous accounts of harmonic expectation in music have emphasised the role of implicit musical knowledge acquired in the long term through the mechanism of statistical learning. However, it is not known whether listeners can adapt their expectations for unusual harmonies in the short term through repetition priming, and whether the extent of any short-term adaptation depends on the unfolding statistical structure of the music. To explore these possibilities, we presented 150 participants with phrases from Bach chorales that ended with a cadence that was either a priori likely or unlikely based on the long-term statistical structure of the corpus of chorales. While holding the 50-50 incidence of likely vs. unlikely cadences constant, we manipulated the order in which these phrases were presented such that the local probability of hearing an unlikely cadence changed throughout the experiment. For each phrase, participants provided two judgements: (a) a prospective rating of how confident they were in their expectations for the cadence, and (b) a retrospective rating of how well the presented cadence matched their expectations. While confidence ratings increased over the course of the experiment, the rate of change decreased as the local probability of an unexpected cadence increased. Participants' expectations favoured likely cadences over unlikely cadences on average, but their expectation ratings for unlikely cadences increased at a faster rate over the course of the experiment than for likely cadences, particularly when the local probability of hearing an unlikely cadence was high. Thus, despite entrenched long-term statistics about cadences, listeners can indeed adapt to unusual musical harmonies and are sensitive to the local statistical structure of the musical environment. We suggest that this adaptation is an instance of Bayesian belief updating, a domain-general process that accounts for expectation adaptation in multiple domains.

Ignoring the alternatives: The N400 is sensitive to stimulus preactivation alone

The N400 component of the event-related brain potential is a neural signal of processing difficulty. In the language domain, it is widely believed to be sensitive to the degree to which a given word or its semantic features have been preactivated in the brain based on the preceding context. However, it has also been shown that the brain often preactivates many words in parallel. It is currently unknown whether the N400 is also affected by the preactivations of alternative words other than the stimulus that is actually presented. This leaves a weak link in the derivation chain-how can we use the N400 to understand the mechanisms of preactivation if we do not know what it indexes? This study directly addresses this gap. We estimate the extent to which all words in a lexicon are preactivated in a given context using the predictions of contemporary large language models. We then directly compare two competing possibilities: that the amplitude of the N400 is sensitive only to the extent to which the stimulus is preactivated, and that it is also sensitive to the preactivation states of the alternatives. We find evidence of the former. This result allows for better grounded inferences about the mechanisms underlying the N400, lexical preactivation in the brain, and language processing more generally.

Bayesian Surprise Predicts Human Event Segmentation in Story Listening

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error.

Older Adults Show Intraindividual Variation in the Use of Predictive Processing

BACKGROUND

The use of prediction can aid language comprehension through preactivation of relevant word features. However, predictions can be wrong, and it has been proposed that resolving the mismatch between the predicted and presented item requires cognitive resources. Older adults tend not to predict and instead rely more on passive comprehension. Here, we tested, using an intraindividual approach, whether older adults consistently use this less demanding processing strategy while reading or whether they attempt to predict on some trials.

METHODS

We used a cross-task conflict paradigm. Younger and older participants self-paced to read sentences that ended with either an expected or unexpected word. Each sentence was then followed by a flanker stimulus that could be congruent or incongruent. We examined responses within and across the two tasks.

RESULTS

Unexpected words were in general read as quickly as expected words, indicating that typical processing of these words was similar. However, for both younger and older adults, there was a greater proportion of very slow trials for unexpected words, revealing different processing on a subset of trials. Critically, in older adults, these slowly read unexpected words engaged control, as seen in speeded responses to incongruent flanker stimuli.

CONCLUSION

Using a cross-task conflict paradigm, we showed that older adults are able to predict and engage cognitive resources to cope with prediction violations, but do not opt to use these processes consistently.

A common neural code for meaning in discourse production and comprehension

How does the brain code the meanings conveyed by language? Neuroimaging studies have investigated this by linking neural activity patterns during discourse comprehension to semantic models of language content. Here, we applied this approach to the production of discourse for the first time. Participants underwent fMRI while producing and listening to discourse on a range of topics. We used a distributional semantic model to quantify the similarity between different speech passages and identified where similarity in neural activity was predicted by semantic similarity. When people produced discourse, speech on similar topics elicited similar activation patterns in a widely distributed and bilateral brain network. This network was overlapping with, but more extensive than, the regions that showed similarity effects during comprehension. Critically, cross-task neural similarities between comprehension and production were also predicted by similarities in semantic content. This result suggests that discourse semantics engages a common neural code that is shared between comprehension and production. Effects of semantic similarity were bilateral in all three RSA analyses, even while univariate activation contrasts in the same data indicated left-lateralised BOLD responses. This indicates that right-hemisphere regions encode semantic properties even when they are not activated above baseline. We suggest that right-hemisphere regions play a supporting role in processing the meaning of discourse during both comprehension and production.

Spatiotemporal evidence uncovers differential neural activity patterns in cognitive and affective conflict control

Studies have shown that there are overlapping neural bases for cognitive and affective conflict control, but whether the neural activity patterns caused by the two types of conflict are similar remains to be explored. The present study utilizes electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI) to temporally and spatially analyze the differences between cognitive and affective conflict control. We employ a semantic conflict task which includes blocks of cognitive and affective judgements primed by conflicting and non-conflicting contexts. The results showed a typical neural conflict effect in the cognitive judgment blocks as reflected by greater amplitudes of P2, N400, and the late positive potential (LPP), as well as greater activation of the left pre-supplementary motor area (pre-SMA) and the right inferior frontal gyrus (IFG) in the conflict condition relative to the non-conflict condition. These patterns did not emerge in the affective judgments, but instead, showed reversed effects of the LPP and in the left SMA. Taken together, these findings suggest that cognitive and affective conflict control result in different neural activity patterns.

Decoding semantic relatedness and prediction from EEG: A classification method comparison

Machine-learning (ML) decoding methods have become a valuable tool for analyzing information represented in electroencephalogram (EEG) data. However, a systematic quantitative comparison of the performance of major ML classifiers for the decoding of EEG data in neuroscience studies of cognition is lacking. Using EEG data from two visual word-priming experiments examining well-established N400 effects of prediction and semantic relatedness, we compared the performance of three major ML classifiers that each use different algorithms: support vector machine (SVM), linear discriminant analysis (LDA), and random forest (RF). We separately assessed the performance of each classifier in each experiment using EEG data averaged over cross-validation blocks and using single-trial EEG data by comparing them with analyses of raw decoding accuracy, effect size, and feature importance weights. The results of these analyses demonstrated that SVM outperformed the other ML methods on all measures and in both experiments.

Predicting coarse-grained semantic features in language comprehension: evidence from ERP representational similarity analysis and Chinese classifier

Existing studies demonstrate that comprehenders can predict semantic information during language comprehension. Most evidence comes from a highly constraining context, in which a specific word is likely to be predicted. One question that has been investigated less is whether prediction can occur when prior context is less constraining for predicting specific words. Here, we aim to address this issue by examining the prediction of animacy features in low-constraining context, using electroencephalography (EEG), in combination with representational similarity analysis (RSA). In Chinese, a classifier follows a numeral and precedes a noun, and classifiers constrain animacy features of upcoming nouns. In the task, native Chinese Mandarin speakers were presented with either animate-constraining or inanimate-constraining classifiers followed by congruent or incongruent nouns. EEG amplitude analysis revealed an N400 effect for incongruent conditions, reflecting the difficulty of semantic integration when an incompatible noun is encountered. Critically, we quantified the similarity between patterns of neural activity following the classifiers. RSA results revealed that the similarity between patterns of neural activity following animate-constraining classifiers was greater than following inanimate-constraining classifiers, before the presentation of the nouns, reflecting pre-activation of animacy features of nouns. These findings provide evidence for the prediction of coarse-grained semantic feature of upcoming words.

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

A fronto-temporal brain network has long been implicated in language comprehension. However, this network's role in language production remains debated. In particular, it remains unclear whether all or only some language regions contribute to production, and which aspects of production these regions support. Across 3 functional magnetic resonance imaging experiments that rely on robust individual-subject analyses, we characterize the language network's response to high-level production demands. We report 3 novel results. First, sentence production, spoken or typed, elicits a strong response throughout the language network. Second, the language network responds to both phrase-structure building and lexical access demands, although the response to phrase-structure building is stronger and more spatially extensive, present in every language region. Finally, contra some proposals, we find no evidence of brain regions-within or outside the language network-that selectively support phrase-structure building in production relative to comprehension. Instead, all language regions respond more strongly during production than comprehension, suggesting that production incurs a greater cost for the language network. Together, these results align with the idea that language comprehension and production draw on the same knowledge representations, which are stored in a distributed manner within the language-selective network and are used to both interpret and generate linguistic utterances.

Predictive Sentence Processing at Speed: Evidence from Online Mouse Cursor Tracking

Three online mouse cursor-tracking experiments investigated predictive sentence processing at speed. Participants viewed visual arrays with objects like a bike and kite while hearing predictive sentences like, "What the man will ride, which is shown on this page, is the bike," or non-predictive sentences like, "What the man will spot, which is shown on this page, is the bike." Based on the selectional restrictions of "ride" (i.e., vs. "spot"), participants made mouse cursor movements to the bike before hearing the noun "bike." Compellingly, this effect was observed at speech rates of ∼3 (Experiment 1), ∼6 (Experiment 2), and ∼9 (Experiment 3) syllables/s. While prior research suggests striking limits on prediction, these results highlight temporal dynamics that may impact comprehenders' ability to preactivate information when hearing impressively rapid speech. Implications for theories of sentence processing are discussed.

How do L2 learners deal with a “dead” language? A psycholinguistic study on sentence processing in Latin

Many decades of research have shown that sentence processing works in an highly incremental and predictive fashion (Marslen-Wilson, 1975) – in the L1 but also (maybe to a lower extent) in the L2. But whereas almost all studies on L2 sentence processing focus on modern languages, it is fully unclear how a language like Latin that fundamentally differs in the way it is taught and used is processed. Thus, the current study focuses on the question if proficient L2 learners of Latin show any evidence for incremental and predictive processing when reading Latin sentences. In a Rapid Serial Visual Presentation (RSVP) task 25 advanced learners of Latin read 384 easy three-word Latin sentences that were manipulated among the factors animacy, argument order and verb position. The results indicate that the participants used the word order and animacy information to incorporate the arguments into the sentence structure on-line and to predict upcoming arguments on the basis of verb information. These findings are interpreted as the first evidence for incremental and predictive processing when reading Latin sentences.

Neural tracking of linguistic and acoustic speech representations decreases with advancing age

BACKGROUND

Older adults process speech differently, but it is not yet clear how aging affects different levels of processing natural, continuous speech, both in terms of bottom-up acoustic analysis and top-down generation of linguistic-based predictions. We studied natural speech processing across the adult lifespan via electroencephalography (EEG) measurements of neural tracking.

GOALS

Our goals are to analyze the unique contribution of linguistic speech processing across the adult lifespan using natural speech, while controlling for the influence of acoustic processing. Moreover, we also studied acoustic processing across age. In particular, we focus on changes in spatial and temporal activation patterns in response to natural speech across the lifespan.

METHODS

52 normal-hearing adults between 17 and 82 years of age listened to a naturally spoken story while the EEG signal was recorded. We investigated the effect of age on acoustic and linguistic processing of speech. Because age correlated with hearing capacity and measures of cognition, we investigated whether the observed age effect is mediated by these factors. Furthermore, we investigated whether there is an effect of age on hemisphere lateralization and on spatiotemporal patterns of the neural responses.

RESULTS

Our EEG results showed that linguistic speech processing declines with advancing age. Moreover, as age increased, the neural response latency to certain aspects of linguistic speech processing increased. Also acoustic neural tracking (NT) decreased with increasing age, which is at odds with the literature. In contrast to linguistic processing, older subjects showed shorter latencies for early acoustic responses to speech. No evidence was found for hemispheric lateralization in neither younger nor older adults during linguistic speech processing. Most of the observed aging effects on acoustic and linguistic processing were not explained by age-related decline in hearing capacity or cognition. However, our results suggest that the effect of decreasing linguistic neural tracking with advancing age at word-level is also partially due to an age-related decline in cognition than a robust effect of age.

CONCLUSION

Spatial and temporal characteristics of the neural responses to continuous speech change across the adult lifespan for both acoustic and linguistic speech processing. These changes may be traces of structural and/or functional change that occurs with advancing age.

Functional neuroanatomy of lexical access in contextually and visually guided spoken word production

Lexical access is commonly studied using bare picture naming, which is visually guided, but in real-life conversation, lexical access is more commonly contextually guided. In this fMRI study, we examined the underlying functional neuroanatomy of contextually and visually guided lexical access, and its consistency across sessions. We employed a context-driven picture naming task with fifteen healthy speakers reading incomplete sentences (word-by-word) and subsequently naming the picture depicting the final word. Sentences provided either a constrained or unconstrained lead-in setting for the picture to be named, thereby approximating lexical access in natural language use. The picture name could be planned either through sentence context (constrained) or picture appearance (unconstrained). This procedure was repeated in an equivalent second session two to four weeks later with the same sample to test for test-retest consistency. Picture naming times showed a strong context effect, confirming that constrained sentences speed up production of the final word depicted as an image. fMRI results showed that the areas common to contextually and visually guided lexical access were left fusiform and left inferior frontal gyrus (both consistently active across-sessions), and middle temporal gyrus. However, non-overlapping patterns were also found, notably in the left temporal and parietal cortices, suggesting a different neural circuit for contextually versus visually guided lexical access.

Patterns of hemispheric asymmetry provide evidence dissociating the semantic and syntactic P600

To understand how neural networks in the left (LH) and right (RH) cerebral hemispheres contribute to different aspects of language comprehension, in two experiments we measured event-related potentials (ERPs) as right-handed participants read sentences, some of which contained morphosyntactic and thematic role violations. Replicating prior work (Kuperberg et al., 2006), in Experiment 1 thematic role violations elicited both an N400 and a (semantic) P600 effect. Morphosyntactic violations elicited effects that differed as a function of participants' familial sinistrality (the presence [FS+] or absence [FS-] of a left-handed biological relative): FS+ participants showed a (syntactic) P600 effect whereas FS- participants showed a biphasic N400 and P600 response. To assess whether this difference reflects different underlying patterns of lateralization, in Experiment 2 target words were presented using visual half-field (VF) presentation. Indeed, for morphosyntactic violations, the FS- group elicited an asymmetric pattern, showing a P600 effect only with LH-biased presentation and an N400 effect in both VFs (cf. Lee and Federmeier, 2015). In contrast, FS+ participants showed a bilateral (N400-only) response pattern. This provides further evidence of FS-based differences in hemispheric contributions to syntactic processing. Strikingly, we found that, when lateralized, thematic role violations did not elicit a P600 effect, suggesting that this effect requires contributions from both hemispheres. The different response patterns for morphosyntactic and thematic role animacy violations across FS and VF also point to a processing difference in the comprehension mechanisms underlying the semantic and syntactic P600, which had heretofore been assumed to be variants of the same component.

Prediction error cost exists in the reading processing of Chinese native speakers and advanced Chinese L2 learners

This study applies the paradigm of self-paced reading to examine the Context Predictability Effect in the processing of Chinese and detect whether there is a prediction error cost. Context constraint strength (constraining and neutral) and word predictability (predictable and unpredictable) were strictly manipulated. The statistical results suggest that: (1) There is a Context Predictability Effect for Chinese native speakers in reading processing, which is consistent with most previous studies; (2) There is also a Context Predictability Effect for advanced Chinese L2 learners; (3) Both Chinese native speakers and Chinese L2 learners have a prediction error cost in reading processing, a finding different from those of much previous research. (4) Chinese L2 learners are significantly slower than Chinese native speakers when they conduct predictive reading processing. This paper is very enlightening in that it identifies the existence of a prediction error cost in Chinese L2 processing by means of behavioral experiments, providing evidence for the hypothesis of Lexical Prediction. In a strongly predictive setting, when encountering a plausible but unpredictable word, the brain must expend extra effort to suppress, revise, or reanalyze the material, and this may account for the prediction error cost.

Context effects in language comprehension: The role of emotional state and attention on semantic and syntactic processing

Semantics and syntax are core components of language. The prevailing view was that processing of word meaning and syntactic processing happens in isolation from other systems. In light of proofed interactions between language and other systems, especially with perception, action and emotion, this view became untenable. This article reviews Event-related potential studies conducted at the Donders Centre for Cognition exploring the interplay between language comprehension and a person's emotional state. The research program was aimed at an investigation of the online effects of emotional state on semantic processing and syntactic processing. To this aim we manipulated mood via film fragments (happy vs. sad) before participants read neutral sentences while their EEG was recorded. In Part 1, it is shown that mood impacts online semantic processing (as indicated by N400) and the processing of syntactic violations (as indicated by P600). Part 2 was directed at a further determination of the mechanisms underlying these interactions. The role of heuristics was examined by investigating the effects of mood on the P600 to semantic reversals. The results revealed that mood affects heuristic processing. The next step consisted of an assessment of the role of attention, in the mood-by-semantics and mood-by-syntax interaction. This was accomplished by recording EEG while manipulating attention via task next to emotional state. Participants performed a semantic or syntactic judgment task vs. a letter-size judgment task. The main ERP results were as follows: (i) attention interacts with the mood effect on semantic processing and syntactic processing, respectively, (ii) the effects of mood on semantic processing and syntactic processing are reliable, and (iii) the mood effects on semantic processing are not fixed but context-dependent. In Part 3 the effects of mood on the processing of script knowledge and general world knowledge are presented. Part 4 closes with a discussion of the mechanisms involved in the mood-by-language interactions and recommendations for future research. Regarding the underlying mechanism we propose that heuristics based on semantic expectancies or syntactic expectancies play a key role in the mood-by-language interactions. The results support the view that language takes place in continuous interaction with other (non-language) systems.

The Angular Gyrus as a Hub for Modulation of Language-related Cortex by Distinct Prefrontal Executive Control Regions

It has become clear in recent years that reading, while relying on domain-specific language processing regions, also involves regions that implement executive processes more broadly. Such executive control is generally considered to be implemented by prefrontal regions, which exert control via connectivity that allows them to modulate processing in target brain regions. The present study examined whether three previously identified and distinct executive control regions in the pFC [Wang, K., Banich, M. T., Reineberg, A. E., Leopold, D. R., Willcutt, E. G., Cutting, L. E., et al. Left posterior prefrontal regions support domain-general executive processes needed for both reading and math. Journal of Neuropsychology, 14, 467-495, 2020] show similar patterns of functional connectivity (FC) during a reading comprehension task as compared with a symbol identification condition. Our FC results in a sample of adolescents (n = 120) suggest all three regions commonly show associations with activity in "classic" left hemisphere reading areas, including the angular and supramarginal gyri, yet each exhibits differential connectivity as well. In particular, precentral regions show differential FC to parietal portions of the dorsal language stream, the inferior frontal junction shows differential FC to middle temporal regions of the right hemisphere and other regions involved in semantic processing, and portions of the inferior frontal gyrus show differential FC to an extensive set of right hemisphere prefrontal regions. These results suggest that prefrontal control over language-related regions occurs in a coordinated yet discrete manner.

Word prediction using closely and moderately related verbs in Down syndrome

People with Down syndrome (DS) have several difficulties in language learning, and one of the areas most affected is language production. Theoretical frameworks argue that prediction depends on the production system. Yet, people with DS can predict upcoming nouns using semantically related verbs. Possibly, prediction skills in people with DS are driven by their associative mechanism rather than by the prediction mechanism based on the production system. This study explores prediction mechanisms in people with DS and their relationship with production skills. Three groups were evaluated in a preferential-looking task: young adults, children with DS, and a typically developing control group paired by sex and mental age. Participants saw two images, a target and a distractor. They also heard a sentence in one of the three conditions: with a verb that was closely related to the object (e.g., "The woman read the book"), with a verb that was moderately related to the object (e.g., "My uncle waited for the bus"), or with a verb that was unrelated to the object (e.g., "My sister threw a broom"). Their productive vocabulary was then measured. In the young adult and typically developing groups, the results showed prediction in sentences with highly and moderately related verbs. Participants with DS, however, showed prediction skills only in the highly related context. There was no influence of chronological age, mental age, or production on prediction skills. These results indicate that people with DS base prediction mainly on associative mechanisms and they have difficulty in generating top-down predictions.

Is the Mind Inherently Predicting? Exploring Forward and Backward Looking in Language Processing

Prediction is one characteristic of the human mind. But what does it mean to say the mind is a "prediction machine" and inherently forward looking as is frequently claimed? In natural languages, many contexts are not easily predictable in a forward fashion. In English, for example, many frequent verbs do not carry unique meaning on their own but instead, rely on another word or words that follow them to become meaningful. Upon reading take a the processor often cannot easily predict walk as the next word. But the system can "look back" and integrate walk more easily when it follows take a (e.g., as opposed to *make|get|have a walk). In the present paper, we provide further evidence for the importance of both forward and backward-looking in language processing. In two self-paced reading tasks and an eye-tracking reading task, we found evidence that adult English native speakers' sensitivity to word forward and backward conditional probability significantly predicted reading times over and above psycholinguistic predictors of reading latencies. We conclude that both forward and backward-looking (prediction and integration) appear to be important characteristics of language processing. Our results thus suggest that it makes just as much sense to call the mind an "integration machine" which is inherently backward 'looking.'

The Effect of Code-Switching Experience on the Neural Response Elicited to a Sentential Code Switch

Switching between languages, or codeswitching, is a cognitive ability that multilinguals can perform with ease. This study investigates whether codeswitching during sentence reading affects early access to meaning, as indexed by the robust brain response called the N400. We hypothesize that the brain prioritizes the meaning of the word during comprehension with codeswitching costs emerging at a different stage of processing. Event-related potentials (ERPs) were recorded while Spanish-English balanced bilinguals (n = 24) read Spanish sentences containing a target noun that could create a semantic violation, codeswitch or both. Self-reported frequency of daily codeswitching was used as a regressor to determine if the cost of reading a switch is modulated by codeswitching experience. A robust N400 to semantic violations was followed by a late positive component (LPC). Codeswitches modulated the left anterior negativity (LAN) and LPC, but not the N400, with codeswitched semantic violations resulting in a sub-additive interaction. Codeswitching experience modulated the LPC, but not the N400. The results suggest that early access to semantic memory during comprehension happens independent of the language in which the words are presented. Codeswitching affects a separate stage of comprehension with switching experience modulating the brain's response to experiencing a language switch.

The cognitive hearing science perspective on perceiving, understanding, and remembering language: The ELU model

The review gives an introductory description of the successive development of data patterns based on comparisons between hearing-impaired and normal hearing participants' speech understanding skills, later prompting the formulation of the Ease of Language Understanding (ELU) model. The model builds on the interaction between an input buffer (RAMBPHO, Rapid Automatic Multimodal Binding of PHOnology) and three memory systems: working memory (WM), semantic long-term memory (SLTM), and episodic long-term memory (ELTM). RAMBPHO input may either match or mismatch multimodal SLTM representations. Given a match, lexical access is accomplished rapidly and implicitly within approximately 100-400 ms. Given a mismatch, the prediction is that WM is engaged explicitly to repair the meaning of the input - in interaction with SLTM and ELTM - taking seconds rather than milliseconds. The multimodal and multilevel nature of representations held in WM and LTM are at the center of the review, being integral parts of the prediction and postdiction components of language understanding. Finally, some hypotheses based on a selective use-disuse of memory systems mechanism are described in relation to mild cognitive impairment and dementia. Alternative speech perception and WM models are evaluated, and recent developments and generalisations, ELU model tests, and boundaries are discussed.

Evoking the N400 Event-related Potential (ERP) Component Using a Publicly Available Novel Set of Sentences with Semantically Incongruent or Congruent Eggplants (Endings)

During speech comprehension, the ongoing context of a sentence is used to predict sentence outcome by limiting subsequent word likelihood. Neurophysiologically, violations of context-dependent predictions result in amplitude modulations of the N400 event-related potential (ERP) component. While N400 is widely used to measure semantic processing and integration, no publicly-available auditory stimulus set is available to standardize approaches across the field. Here, we developed an auditory stimulus set of 442 sentences that utilized the semantic anomaly paradigm, provided cloze probability for all stimuli, and was developed for both children and adults. With 20 neurotypical adults, we validated that this set elicits robust N400′s, as well as two additional semantically-related ERP components: the recognition potential (~250 ms) and the late positivity component (~600 ms). This stimulus set (https://doi.org/10.5061/dryad.9ghx3ffkg) and the 20 high-density (128-channel) electrophysiological datasets (https://doi.org/10.5061/dryad.6wwpzgmx4) are made publicly available to promote data sharing and reuse. Future studies that use this stimulus set to investigate sentential semantic comprehension in both control and clinical populations may benefit from the increased comparability and reproducibility within this field of research.