Influences of semantic and syntactic context on open- and closed-class words

Language style (mis)matching: Consuming entertainment media from someone unlike you is linked to positive attitudes

Prior work suggests people often match with conversational partners by using a common rate of style words (e.g., articles, pronouns). Indeed, such language style matching (LSM) has associated positively with downstream social and psychological dynamics like cooperation, liking, and well-being. To what degree is LSM predictive of positive attitudes in entertainment media settings? The current two-study paper addressed this question by collecting participants' writing style with three diverse prompts, and then having them consume a random selection of TED talks (Study 1) and videotaped podcast narratives (Study 2). The evidence suggested less LSM was associated with positive attitudes (e.g., an interest in watching another video by the speaker, feeling connected to the speaker). Mediation analyses revealed the negative relationship between LSM and positive attitudes was explained by one's novelty need satisfaction. Implications for LSM research, plus character construction and narrative development in media psychology, are discussed.

Brain imaging and electrophysiological markers of anaphoric reference during speech production

Pronouns create cohesive links in discourse by referring to previously mentioned elements. Here, we focus on pronominalization during speech production in three experiments employing ERP and fMRI methodologies. Participants were asked to produce two short sentences describing a man or woman using an object. In the second sentence, they were instructed to use a pronoun to refer to the same person and a noun to refer to a different person. The first ERP experiment revealed that noun conditions elicited more negative ERPs starting at 220 ms, with significant differences in early and later time windows, particularly in the left hemisphere. The second ERP experiment showed divergence at 280 ms, with significant differences between 300 and 400 ms at midline electrodes, again indicating more negative ERPs for nouns. The fMRI experiment identified greater activations for nouns than pronouns in regions like the superior temporal gyrus (STG) and cerebellar vermis, suggesting higher working memory load and lexical retrieval demands for nouns compared to pronouns. Moreover, pronouns elicited an enhanced centro-parietal positivity, indicating increased attentional demands. These findings suggest that while noun processing requires greater working memory and lexical retrieval, pronoun processing engages more attentional resources. This study advances our understanding of the neural mechanisms underlying pronominalization during speech production, highlighting distinct neural responses for nouns and pronouns.

Exploring the activation of target words in adults who stutter with and without conscious intention to speak: ERP evidence

INTRODUCTION

The primary aim was to determine whether the activation of target words unfolds differently in adults who stutter (AWS) versus typically-fluent adults (TFA) preparing to name pictures. A secondary aim was to explore the influence of conscious intention to speak on this process.

METHOD

18 AWS and 18 TFA completed a picture naming task in which pictures - named at a delay - were followed by auditory probe words that were identical, or unrelated, to the target picture labels. A subset of those participants (15 AWS and 15 TFA) completed a second task in which pictures - judged conceptually at a delay - were followed by auditory probe words that directly named the pictures or were unrelated. Event-related potentials (ERPs) were recorded from probe word onset in each task. It was predicted that the N400 ERP component - an index of lexical processing - would be attenuated to Identity versus Unrelated trials. Between-groups differences in lexical activation (if any) were predicted to result in between-groups differences in the amplitude, latency and/or scalp topography of N400 priming effects.

RESULTS

N400 priming was detected in both tasks for both Groups. In the picture naming task, the N400 priming effect had a more broadly-distributed scalp topography in TFA versus AWS. No between-groups differences were detected in the N400 priming effect in the conceptual judgment task.

CONCLUSIONS

A between-groups difference in the scalp topography of the N400 priming effect in the picture naming task points to a between-groups difference in intention-driven lexical access. Discussion considers how the top-down process of intention - if diminished among AWS - might impact lexical selection and speech motor control.

Prediction in reading: A review of predictability effects, their theoretical implications, and beyond

Historically, prediction during reading has been considered an inefficient and cognitively expensive processing mechanism given the inherently generative nature of language, which allows upcoming text to unfold in an infinite number of possible ways. This article provides an accessible and comprehensive review of the psycholinguistic research that, over the past 40 or so years, has investigated whether readers are capable of generating predictions during reading, typically via experiments on the effects of predictability (i.e., how well a word can be predicted from its prior context). Five theoretically important issues are addressed: What is the best measure of predictability? What is the functional relationship between predictability and processing difficulty? What stage(s) of processing does predictability affect? Are predictability effects ubiquitous? What processes do predictability effects actually reflect? Insights from computational models of reading about how predictability manifests itself to facilitate the reading of text are also discussed. This review concludes by arguing that effects of predictability can, to a certain extent, be taken as demonstrating evidence that prediction is an important but flexible component of real-time language comprehension, in line with broader predictive accounts of cognitive functioning. However, converging evidence, especially from concurrent eye-tracking and brain-imaging methods, is necessary to refine theories of prediction.

Neurophysiological signatures of prediction in language: A critical review of anticipatory negativities

Recent event-related potential (ERP) studies in language comprehension converge in finding anticipatory negativities preceding words or word segments that can be pre-activated based on either sentence contexts or phonological cues. We review these findings from different paradigms in the light of evidence from other cognitive domains in which slow negative potentials have long been associated with anticipatory processes and discuss their potential underlying mechanisms. We propose that this family of anticipatory negativities captures common mechanisms associated with the pre-activation of linguistic information both within words and within sentences. Future studies could utilize these anticipatory negativities in combination with other, well-established ERPs, to simultaneously track prediction-related processes emerging at different time intervals (before and after the perception of pre-activated input) and with distinct time courses (shorter-lived and longer-lived cognitive operations).

Strong Prediction: Language Model Surprisal Explains Multiple N400 Effects

Theoretical accounts of the N400 are divided as to whether the amplitude of the N400 response to a stimulus reflects the extent to which the stimulus was predicted, the extent to which the stimulus is semantically similar to its preceding context, or both. We use state-of-the-art machine learning tools to investigate which of these three accounts is best supported by the evidence. GPT-3, a neural language model trained to compute the conditional probability of any word based on the words that precede it, was used to operationalize contextual predictability. In particular, we used an information-theoretic construct known as surprisal (the negative logarithm of the conditional probability). Contextual semantic similarity was operationalized by using two high-quality co-occurrence-derived vector-based meaning representations for words: GloVe and fastText. The cosine between the vector representation of the sentence frame and final word was used to derive contextual cosine similarity estimates. A series of regression models were constructed, where these variables, along with cloze probability and plausibility ratings, were used to predict single trial N400 amplitudes recorded from healthy adults as they read sentences whose final word varied in its predictability, plausibility, and semantic relationship to the likeliest sentence completion. Statistical model comparison indicated GPT-3 surprisal provided the best account of N400 amplitude and suggested that apparently disparate N400 effects of expectancy, plausibility, and contextual semantic similarity can be reduced to variation in the predictability of words. The results are argued to support predictive coding in the human language network.

[The EEG N400 component as a marker of language acquisition and processing in cochlear implant users]

Language processing can be measured objectively using late components in the evoked brain potential. The most established component in this area of research is the N400 component, a negativity that peaks at about 400 ms after stimulus onset with a centro-parietal maximum. It reflects semantic processing. Its presence, as well as its temporal and quantitative expression, allows to conclude about the quality of processing. It is therefore suitable for measuring speech comprehension in special populations, such as cochlear implant (CI) users. The following is an overview of the use of the N400 component as a tool for studying language processes in CI users. We present studies with adult CI users, where the N400 reflects the quality of speech comprehension with the new hearing device and we present studies with children where the emergence of the N400 component reflects the acquisition of their very first vocabulary.

Anaphoric distance dependencies in visual narrative structure and processing

Linguistic syntax has often been claimed as uniquely complex due to features like anaphoric relations and distance dependencies. However, visual narratives of sequential images, like those in comics, have been argued to use sequencing mechanisms analogous to those in language. These narrative structures include "refiner" panels that "zoom in" on the contents of another panel. Similar to anaphora in language, refiners indexically connect inexplicit referential information in one unit (refiner, pronoun) to a more informative "antecedent" elsewhere in the discourse. Also like in language, refiners can follow their antecedents (anaphoric) or precede them (cataphoric), along with having either proximal or distant connections. We here explore the constraints on visual narrative refiners created by modulating these features of order and distance. Experiment 1 examined participants' preferences for where refiners are placed in a sequence using a force-choice test, which revealed that refiners are preferred to follow their antecedents and have proximal distances from them. Experiment 2 then showed that distance dependencies lead to slower self-paced viewing times. Finally, measurements of event-related brain potentials (ERPs) in Experiment 3 revealed that these patterns evoke similar brain responses as referential dependencies in language (i.e., N400, LAN, Nref). Across all three studies, the constraints and (neuro)cognitive responses to refiners parallel those shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

The N400 is Elicited by Meaning Changes but not Synonym Substitutions: Evidence From Persian Phrasal Verbs

We tested two accounts of the cognitive process underlying the N400 event-related potential component: one that it reflects meaning-based processing and one that it reflects the processing of specific words. The experimental design utilized separable Persian phrasal verbs, which form a strongly probabilistic, long-distance dependency, ideal for the study of probabilistic processing. In sentences strongly constraining for a particular continuation, we show evidence that between two low-probability words, only the word that changed the expected meaning of the sentence increased N400 amplitude, while a synonym of the most probable target word did not. The findings support an account of the N400 in which its underlying process is driven by the processing of meaning rather than of specific words.

An eye on semantics: a study on the influence of concreteness and predictability on early fixation durations

We used eye-tracking during natural reading to study how semantic control and representation mechanisms interact for the successful comprehension of sentences, by manipulating sentence context and single-word meaning. Specifically, we examined whether a word's semantic characteristic (concreteness) affects first fixation and gaze durations (FFDs and GDs) and whether it interacts with the predictability of a word. We used a linear mixed effects model including several possible psycholinguistic covariates. We found a small but reliable main effect of concreteness and replicated a predictability effect on FFDs, but we found no interaction between the two. The results parallel previous findings of additive effects of predictability (context) and frequency (lexical level) in fixation times. Our findings suggest that the semantics of a word and the context created by the preceding words additively influence early stages of word processing in natural sentence reading.

Ignoring the alternatives: The N400 is sensitive to stimulus preactivation alone

The N400 component of the event-related brain potential is a neural signal of processing difficulty. In the language domain, it is widely believed to be sensitive to the degree to which a given word or its semantic features have been preactivated in the brain based on the preceding context. However, it has also been shown that the brain often preactivates many words in parallel. It is currently unknown whether the N400 is also affected by the preactivations of alternative words other than the stimulus that is actually presented. This leaves a weak link in the derivation chain-how can we use the N400 to understand the mechanisms of preactivation if we do not know what it indexes? This study directly addresses this gap. We estimate the extent to which all words in a lexicon are preactivated in a given context using the predictions of contemporary large language models. We then directly compare two competing possibilities: that the amplitude of the N400 is sensitive only to the extent to which the stimulus is preactivated, and that it is also sensitive to the preactivation states of the alternatives. We find evidence of the former. This result allows for better grounded inferences about the mechanisms underlying the N400, lexical preactivation in the brain, and language processing more generally.

Read carefully, because this is important! How value-driven strategies impact sentence memory

Little is understood about how people strategically process and remember important but complex information, such as sentences. In the current study, we asked whether people can effectively prioritize memory for sentences as a function of their relative importance (operationalized as a reward point value) and whether they do so, in part, by changing their sentence processing strategies when value information is available in advance. We adapted the value-directed remembering paradigm (Castel, Psychol Learn Motiv 48:225-270, 2007) for sentences that varied in constraint and predictability. Each sentence was associated with a high or low value for subsequent free recall (whole sentence) and recognition (sentence-final words) tests. Value information appeared after or before each sentence as a between-subject manipulation. Regardless of condition, we observed that high-value sentences were recalled more often than low-value sentences, showing that people can strategically prioritize their encoding of sentences. However, memory patterns differed depending on when value information was available. Recall for high-value sentences that ended unexpectedly (and therefore violated one's predictions) was reduced in the Before compared to the After condition. Before condition participants also showed a greater tendency to false alarm to lures (words that were the predicted - but not obtained - ending) from strongly constraining sentences. These observations suggest that when people try to prioritize sentence-level information that they know is valuable, the reading strategies they employ may paradoxically lead to worse memory.

Unveiling the need of interactions for social N400s and supporting the N400 inhibition hypothesis

When participants (Pps) are presented with stimuli in the presence of another person, they may consider that person's perspective. Indeed, five recent ERP studies show that the amplitudes of their N400s are increased. The two most recent ones reveal that these social-N400 increases occur even when instructions do not require a focus on the other's perspective. These increases also happen when Pps know that this other person has the same stimulus information as they have. However, in all these works, Pps could see the other person. Here, we tested whether the interaction occurring with this sight is important or whether these social N400 increases also occur when the other person is seated a bit behind Pps, who are aware of it. All had to decide whether the word ending short stories was coherent, incoherent, or equivocal. No social N400 increase was observed: N400s elicited by those words in Pps who were with a confederate (n = 50) were similar to those of Pps who were alone (n = 51). On the other hand, equivocal endings did not elicit larger N400s than coherent ones but triggered larger late posterior positivities (LPPs), like in previous studies. The discussion focuses on the circumstances in which perspective-taking occurs and on the functional significance of the N400 and the LPP.

Predictive coding across the left fronto-temporal hierarchy during language comprehension

We used magnetoencephalography (MEG) and event-related potentials (ERPs) to track the time-course and localization of evoked activity produced by expected, unexpected plausible, and implausible words during incremental language comprehension. We suggest that the full pattern of results can be explained within a hierarchical predictive coding framework in which increased evoked activity reflects the activation of residual information that was not already represented at a given level of the fronto-temporal hierarchy ("error" activity). Between 300 and 500 ms, the three conditions produced progressively larger responses within left temporal cortex (lexico-semantic prediction error), whereas implausible inputs produced a selectively enhanced response within inferior frontal cortex (prediction error at the level of the event model). Between 600 and 1,000 ms, unexpected plausible words activated left inferior frontal and middle temporal cortices (feedback activity that produced top-down error), whereas highly implausible inputs activated left inferior frontal cortex, posterior fusiform (unsuppressed orthographic prediction error/reprocessing), and medial temporal cortex (possibly supporting new learning). Therefore, predictive coding may provide a unifying theory that links language comprehension to other domains of cognition.

Taxonomically-related Word Pairs Evoke both N400 and LPC at Long SOA in Turkish

Semantic priming in Turkish was examined in 36 right-handed healthy participants in a delayed lexical decision task via taxonomic relations using EEG. Prime-target relations included related- unrelated- and pseudo-words. Taxonomically related words at long stimulus onset asynchrony (SOA) were shown to modulate N400 and late positive component (LPC) amplitudes. N400 semantic priming effect in the time window of 300-500 ms was the largest for pseudo-words, intermediate for semantically-unrelated targets, and smallest for semantically-related targets as a reflection of lexical-semantic retrieval. This finding contributes to the ERP literature showing how remarkably universal the N400 brain potential is, with similar effects across languages and orthography. The ERP data also revealed different influences of related, unrelated, and pseudo-word conditions on the amplitude of the LPC. Attention scores and mean LPC amplitudes of related words in parietal region showed a moderate correlation, indicating LPC may be related to "relationship-detection process".

ERPs Reveal How Semantic and Syntactic Processing Unfold across Parafoveal and Foveal Vision during Sentence Comprehension

We examined how readers process content and function words in sentence comprehension with ERPs. Participants read simple declarative sentences using a rapid serial visual presentation (RSVP) with flankers paradigm. Sentences contained either an unexpected semantically anomalous content word, an unexpected syntactically anomalous function word or were well formed with no anomalies. ERPs were examined when target words were in the parafoveal or foveal vision. Unexpected content words elicited a typically distributed N400 when displayed in the parafovea, followed by a longer-lasting, widely distributed positivity starting around 300 ms once foveated. Unexpected function words elicited a left lateralized LAN-like component when presented in the parafovea, followed by a left lateralized, posteriorly distributed P600 when foveated. These results suggested that both semantic and syntactic processing involve two stages-the initial, fast process that can be completed in parafovea, followed by a more in depth attentionally mediated assessment that occurs with direct attention.

The last course of coarse coding: Hemispheric similarities in associative and categorical semantic processing

To test theories that posit differences in how semantic information is represented in the cerebral hemispheres, we assessed semantic priming for associatively and categorically related prime-target pairs that were graded in relatedness strength. Visual half-field presentation was used to bias processing to the right or left hemisphere, and event-related potential (ERP) and behavioral responses were measured while participants completed a semantic relatedness judgement task. Contrary to theories positing representational differences across the cerebral hemispheres, in two experiments using (1) centralized prime presentation and lateralized targets and (2) lateralized primes and targets, we found similar priming patterns across the two hemispheres at the level of semantic access (N400), on later measures of explicit processing (late positive complex; LPC), and in behavioral response speeds and accuracy. We argue that hemispheric differences, when they arise, are more likely due to differences in task demands than in how the hemispheres fundamentally represent semantic information.

Age-related Changes in the Structure and Dynamics of the Semantic Network

Normal aging has variable effects on language comprehension, perhaps because comprehension mechanisms vary in their dependence on network structure versus network dynamics. To test claims that processing dynamics are more affected by age than structure, we used EEG to measure and compare the impact of neighborhood size, a core measure of the structure of the lexico-semantic network, and repetition, a simple measure of processing dynamics, on single word processing. Older adults showed robust effects of neighborhood size on the N400, comparable to those elicited by young adults, but reduced effects of repetition. Furthermore, older adults with greater verbal fluency, print exposure, and reading comprehension showed greater repetition effects, suggesting some older adults can maintain processing dynamics that are similar to those of young adults. Thus, the organizational structure of the semantic network seems stable across normal aging, but (some) older adults may struggle to adjust activation states within that network.

Cursed Concepts: New insights on combinatorial processing from ERP correlates of swearing in context

Expressives (damn) convey speaker attitude and when used in context (Tom lost the damn dog) can be flexibly applied locally to the noun (dog) or globally to the whole sentence (the situation). We used ERPs to explore brain responses to expressives in sentences. Participants read expressive, descriptive, and pseudoword adjectives followed by nouns in sentences (The damn/black/flerg dog peed on the couch). At the adjective late-positivity-component (LPC), expressives and descriptives showed no difference, suggesting reduced social threat and that readers employ a 'wait-and-see' strategy to interpret expressives. Nouns preceded by expressives elicited a larger frontal P200, as well as reduced N400 and LPC than nouns preceded by descriptives. We associated the frontal P200 with emotional salience, the frontal N400 with mental imagery, and the LPC with cognitive load for combinatorics. We suggest that expressive adjectives are not bound to conceptual integration and conclude that parsers wait-and-see what is being damned.

Connecting and considering: Electrophysiology provides insights into comprehension

The ability to rapidly and systematically access knowledge stored in long-term memory in response to incoming sensory information-that is, to derive meaning from the world-lies at the core of human cognition. Research using methods that can precisely track brain activity over time has begun to reveal the multiple cognitive and neural mechanisms that make this possible. In this article, I delineate how a process of connecting affords an effortless, continuous infusion of meaning into human perception. In a relatively invariant time window, uncovered through studies using the N400 component of the event-related potential, incoming sensory information naturally induces a graded landscape of activation across long-term semantic memory, creating what might be called "proto-concepts". Connecting can be (but is not always) followed by a process of further considering those activations, wherein a set of more attentionally demanding "active comprehension" mechanisms mediate the selection, augmentation, and transformation of the initial semantic representations. The result is a limited set of more stable bindings that can be arranged in time or space, revised as needed, and brought to awareness. With this research, we are coming closer to understanding how the human brain is able to fluidly link sensation to experience, to appreciate language sequences and event structures, and, sometimes, to even predict what might be coming up next.

The N400 ERP component reflects an error-based implicit learning signal during language comprehension

The functional significance of the N400 evoked-response component is still actively debated. An increasing amount of theoretical and computational modelling work is built on the interpretation of the N400 as a prediction error. In neural network modelling work, it was proposed that the N400 component can be interpreted as the change in a probabilistic representation of meaning that drives the continuous adaptation of an internal model of the statistics of the environment. These results imply that increased N400 amplitudes should correspond to greater adaptation, which can be measured via implicit memory. To investigate this model derived hypothesis, the current study manipulated expectancy in a sentence reading task to influence N400 amplitudes and subsequently presented the previously expected vs. unexpected words in a perceptual identification task to measure implicit memory. As predicted, reaction times in the perceptual identification task were significantly faster for previously unexpected words that induced larger N400 amplitudes in the previous sentence reading task. Additionally, it could be demonstrated that this adaptation seems to specifically depend on the process underlying N400 amplitudes, as participants with larger N400 differences during sentence reading also exhibited a larger implicit memory benefit in the perceptual identification task. These findings support the interpretation of the N400 as an implicit learning signal driving adaptation in language processing.

Unexpected words or unexpected languages? Two ERP effects of code-switching in naturalistic discourse

Bilingual speakers often switch between languages in conversation without any advance notice. Psycholinguistic research has found that these language shifts (or code-switches) can be costly for comprehenders in certain situations. The present study explores the nature of these costs by comparing code-switches to other types of unexpected linguistic material. To do this, we used a novel EEG paradigm, the Storytime task, in which we record readings of natural texts, and then experimentally manipulate their properties by splicing in words. In this study, we manipulated the language of our target words (English, Spanish) and their fit with the preceding context (strong-fit, weak-fit). If code-switching incurs a unique cost beyond that incurred by an unexpected word, then we should see an additive pattern in our ERP indices. If an effect is driven by lexical expectation alone, then there should be a non-additive interaction such that all unexpected forms incur a similar cost. We found three effects: a general prediction effect (a non-additive N400), a post-lexical recognition of the switch in languages (an LPC for code-switched words), and a prolonged integration difficulty associated with weak-fitting words regardless of language (a sustained negativity). We interpret these findings as suggesting that the processing difficulties experienced by bilinguals can largely be understood within more general frameworks for understanding language comprehension. Our findings are consistent with the broader literature demonstrating that bilinguals do not have two wholly separate language systems but rather a single language system capable of using two coding systems.

What's new? A comprehension bias in favor of informativity

Language is used as a channel by which speakers convey, among other things, newsworthy and informative messages, i.e., content that is otherwise unpredictable to the comprehender. We therefore might expect comprehenders to show a preference for such messages. However, comprehension studies tend to emphasize the opposite: i.e., processing ease for situation-predictable content (e.g., chopping carrots with a knife). Comprehenders are known to deploy knowledge about situation plausibility during processing in fine-grained context-sensitive ways. Using self-paced reading, we test whether comprehenders can also deploy this knowledge in favor of newsworthy content to yield informativity-driven effects alongside, or instead of, plausibility-driven effects. We manipulate semantic context (unusual protagonists), syntactic construction (wh- clefts), and the communicative environment (text messages). Reading times (primarily sentence-finally) show facilitation for sentences containing newsworthy content (e.g., chopping carrots with a shovel), where the content is both unpredictable at the situation level because of its atypicality and also unpredictable at the word level because of the large number of atypical elements a speaker could potentially mention. Our studies are the first to show that informativity-driven effects are observable at all, and the results highlight the need for models that distinguish between comprehenders' estimate of content plausibility and their estimate of a speaker's decision to talk about that content.

Cloze enough? Hemodynamic effects of predictive processing during natural reading

Evidence accrues that readers form multiple hypotheses about upcoming words. The present study investigated the hemodynamic effects of predictive processing during natural reading by means of combining fMRI and eye movement recordings. In particular, we investigated the neural and behavioral correlates of precision-weighted prediction errors, which are thought to be indicative of subsequent belief updating. Participants silently read sentences in which we manipulated the cloze probability and the semantic congruency of the final word that served as an index for precision and prediction error respectively. With respect to the neural correlates, our findings indicate an enhanced activation within the left inferior frontal and middle temporal gyrus suggesting an effect of precision on prediction update in higher (lexico-)semantic levels. Despite being evident at the neural level, we did not observe any evidence that this mechanism resulted in disproportionate reading times on participants' eye movements. The results speak against discrete predictions, but favor the notion that multiple words are activated in parallel during reading.

Visual and Verbal Narrative Comprehension in Children and Adolescents with Autism Spectrum Disorders: An ERP Study

We examined semantic processing in ASD children by presenting sentences with congruent or incongruent final words and visual narratives with congruent or incongruent final panels. An N400 effect to incongruent words appeared as compared to congruent ones, which was attenuated for the ASD children. We observed a negativity sustained to incongruous than congruous words, but only for the TD children. Incongruent panels evoked a greater fronto-central N400 amplitude than congruent panels in both groups. In addition, incongruent panels evoked a centro-parietal late positivity, only in controls. In conclusion, ASD children face processing deficits in both verbal and visual materials when integrating meaning across information, though such impairments may arise in different parts of the interpretive process, depending on the modality.

Code-Switching Does Not Equal Code-Switching. An Event-Related Potentials Study on Switching From L2 German to L1 Russian at Prepositions and Nouns

Studies on event-related potentials (ERP) in code-switching (CS) have concentrated on single-word insertions, usually nouns. However, CS ranges from inserting single words into the main language of discourse to alternating languages for larger segments of a discourse, and can occur at various syntactic positions and with various word classes. This ERP study examined native speakers of Russian who had learned German as a second language; they were asked to listen to sentences with CS from their second language, German, to their first language, Russian. CS included either a whole prepositional phrase or only the lexical head noun of a prepositional phrase. CS at nouns resulted in a late positive complex (LPC), whereas CS at prepositions resulted in a broad early negativity, which was followed by an anterior negativity with a posterior positivity. Only in the last time window (800–1000 ms) did CS at prepositions result in a broad positivity similar to CS at nouns. The differences between both types of CS indicate that they relate to different psycholinguistic processes.

Literacy skill and intra-individual variability in eye-fixation durations during reading: Evidence from a diverse community-based adult sample

To understand the effects of literacy on fundamental processes involved in reading, we report a secondary data analysis examining individual differences in global eye-movement measures and first-pass eye-movement distributions in a diverse sample of community-dwelling adults aged 16 to 64. Participants (n = 80) completed an assessment battery probing verbal and non-verbal cognitive abilities and read simple two-sentence passages while their eye movements were recorded. Analyses were focused on characterising the effects of literacy skill on both global indices of eye-fixation distributions and distributional differences in the sensitivity to lexical features. Global reading measures showed that lower literate adults read more slowly on average. However, distributional analyses of fixation durations revealed that the first-pass fixation durations of adults with lower literacy skill were not slower in general (i.e., there was no shift in the fixation duration distribution among lower literate adults). Instead, lower literacy was associated with greater intra-individual variability in first-pass fixation durations, including an increased proportion of extremely long fixations, differentially skewing the distribution of both first-fixation and gaze durations. Exploratory repeated-measures quantile regression analyses of gaze duration revealed differentially greater influences of word length among lower literate readers and greater activation of phonological and orthographic neighbours among higher literate readers, particularly in the tail of the distribution. Collectively, these findings suggest that literacy skill in adulthood is associated with systematic differences in both global and lexically driven eye-movement control during reading.

Individual Differences in Reading Speed are Linked to Variability in the Processing of Lexical and Contextual Information: Evidence from Single-trial Event-related Brain Potentials

In the current paper, we examined the effects of lexical (e.g. word frequency, orthographic neighborhood density) and contextual (e.g. word predictability in the form of cloze probability) features on single-trial event-related brain potentials in a self-paced reading paradigm. Critically, we examined whether individual differences in reading speed modulated single-trial effects on the N400, an ERP component linked to semantic memory access. Consistent with past work, we found that word frequency effects on the N400 were attenuated with increasing predictability. However, effects of orthographic neighborhood density were robust across all levels of predictability. Importantly, individual differences in reading speed moderated the influence of both frequency and predictability (but not orthographic neighborhood density) on the N400, such that slower readers showed reduced effects compared to faster readers. These data show that different lexical factors influence word processing through dissociable mechanisms. Our findings support a dynamic semantic-memory access model of the N400, in which information at multiple levels (lexical, sentential, individual) simultaneously contributes to the unfolding neural dynamics of comprehension.

Event-related brain potentials reveal how multiple aspects of semantic processing unfold across parafoveal and foveal vision during sentence reading

Recent event-related brain potential (ERP) experiments have demonstrated parafoveal N400 expectancy and congruity effects, showing that semantic information can be accessed from words in parafoveal vision (a conclusion also supported by some eye-tracking work). At the same time, it is unclear how higher-order integrative aspects of language comprehension unfold across the visual field during reading. In the current study, we recorded ERPs in a parafoveal flanker paradigm, while readers were instructed to read passively for comprehension or to judge the plausibility of sentences in which target words varied in their semantic expectancy and congruity. We directly replicated prior work showing graded N400 effects for parafoveal viewing, which are then not duplicated when the target words are processed foveally. Critically, although N400 effects were not modulated by task goals, a posteriorly distributed late positive component thought to reflect semantic integration processes was observed to semantic incongruities only in the plausibility judgment task. However, this effect was observed at a considerable delay, appearing only after words had moved into foveal vision. Our findings thus suggest that semantic access can be initiated in parafoveal vision, whereas central foveal vision may be necessary to enact higher-order (and task-dependent) integrative processing.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension

The past decade has seen a rapid growth of cognitive and brain research focused on visual narratives like comics and picture stories. This paper will summarize and integrate this emerging literature into the Parallel Interfacing Narrative‐Semantics Model (PINS Model)—a theory of sequential image processing characterized by an interaction between two representational levels: semantics and narrative structure. Ongoing semantic processes build meaning into an evolving mental model of a visual discourse. Updating of spatial, referential, and event information then incurs costs when they are discontinuous with the growing context. In parallel, a narrative structure organizes semantic information into coherent sequences by assigning images to categorical roles, which are then embedded within a hierarchic constituent structure. Narrative constructional schemas allow for specific predictions of structural sequencing, independent of semantics. Together, these interacting levels of representation engage in an iterative process of retrieval of semantic and narrative information, prediction of upcoming information based on those assessments, and subsequent updating based on discontinuity. These core mechanisms are argued to be domain‐general—spanning across expressive systems—as suggested by similar electrophysiological brain responses (N400, P600, anterior negativities) generated in response to manipulation of sequential images, music, and language. Such similarities between visual narratives and other domains thus pose fundamental questions for the linguistic and cognitive sciences.

Visual narratives like comics involve a range of complex cognitive operations in order to be understood. The Parallel Interfacing Narrative‐Semantics (PINS) Model integrates an emerging literature showing that comprehension of wordless image sequences balances two representational levels of semantic and narrative structure. The neurocognitive mechanisms that guide these processes are argued to overlap with other domains, such as language and music.

The temporal dynamics of structure and content in sentence comprehension: Evidence from fMRI-constrained MEG

Humans have a striking capacity to combine words into sentences that express new meanings. Previous research has identified key brain regions involved in this capacity, but little is known about the time course of activity in these regions, as hemodynamic methods such as fMRI provide little insight into temporal dynamics of neural activation. We performed an MEG experiment to elucidate the temporal dynamics of structure and content processing within four brain regions implicated by fMRI data from the same experiment: the temporo-parietal junction (TPJ), the posterior temporal lobe (PTL), the anterior temporal lobe (ATL), and the anterior inferior frontal gyrus (IFG). The TPJ showed increased activity for both structure and content near the end of the sentence, consistent with a role in incremental interpretation of event semantics. The PTL, a region not often associated with core aspects of syntax, showed a strong early effect of structure, consistent with predictive parsing models, and both structural and semantic context effects on function words. These results provide converging evidence that the PTL plays an important role in lexicalized syntactic processing. The ATL and IFG, regions traditionally associated with syntax, showed minimal effects of sentence structure. The ATL, PTL and IFG all showed effects of semantic content: increased activation for real words relative to nonwords. Our fMRI-guided MEG investigation therefore helps identify syntactic and semantic aspects of sentence comprehension in the brain in both spatial and temporal dimensions.

Behavioral and neural evidence on the processing of ambiguous adjective-noun dependencies in Korean sentence comprehension

In Korean, it is allowed for an adjective to modify a distant noun that appears after an intervening relative clause instead of an adjacent noun. The current study investigated the time course of syntactic and semantic integration between an adjective (A) and an adjacent noun (N1) and/or a distant noun (N2) during on-line reading comprehension of Korean sentences. Semantic congruence between adjectives and nouns were manipulated, such that A was congruent with both N1 and N2, either with N1 or N2, or with none of N1/N2. The reading times and ERPs to critical words revealed that under A-N1 semantic incongruence, not the processing load of N1, but those of the relative clause verb and N2 which is semantically incongruent with A increased. These results imply that the semantic incongruence suppressed the A-N1 integration until the relative clause verb occurred, and the processor immediately attempted the A-N2 integration for a way out from the ultimate processing breakdown even before the occurrence of the main verb.

Visual and linguistic narrative comprehension in autism spectrum disorders: Neural evidence for modality-independent impairments

Individuals with autism spectrum disorders (ASD) have notable language difficulties, including with understanding narratives. However, most narrative comprehension studies have used written or spoken narratives, making it unclear whether narrative difficulties stem from language impairments or more global impairments in the kinds of general cognitive processes (such as understanding meaning and structural sequencing) that are involved in narrative comprehension. Using event-related potentials (ERPs), we directly compared semantic comprehension of linguistic narratives (short sentences) and visual narratives (comic panels) in adults with ASD and typically-developing (TD) adults. Compared to the TD group, the ASD group showed reduced N400 effects for both linguistic and visual narratives, suggesting comprehension impairments for both types of narratives and thereby implicating a more domain-general impairment. Based on these results, we propose that individuals with ASD use a more bottom-up style of processing during narrative comprehension.

Listening beyond seeing: Event-related potentials to audiovisual processing in visual narrative

Every day we integrate meaningful information coming from different sensory modalities, and previous work has debated whether conceptual knowledge is represented in modality-specific neural stores specialized for specific types of information, and/or in an amodal, shared system. In the current study, we investigated semantic processing through a cross-modal paradigm which asked whether auditory semantic processing could be modulated by the constraints of context built up across a meaningful visual narrative sequence. We recorded event-related brain potentials (ERPs) to auditory words and sounds associated to events in visual narratives-i.e., seeing images of someone spitting while hearing either a word (Spitting!) or a sound (the sound of spitting)-which were either semantically congruent or incongruent with the climactic visual event. Our results showed that both incongruent sounds and words evoked an N400 effect, however, the distribution of the N400 effect to words (centro-parietal) differed from that of sounds (frontal). In addition, words had an earlier latency N400 than sounds. Despite these differences, a sustained late frontal negativity followed the N400s and did not differ between modalities. These results support the idea that semantic memory balances a distributed cortical network accessible from multiple modalities, yet also engages amodal processing insensitive to specific modalities.

Contextual constraints on lexico-semantic processing in aging: Evidence from single-word event-related brain potentials

The current study reports the effects of accumulating contextual constraints on neural indices of lexico-semantic processing (i.e., effects of word frequency and orthographic neighborhood) as a function of normal aging. Event-related brain potentials were measured from a sample of older adults as they read sentences that were semantically congruent, provided only syntactic constraints (syntactic prose), or were random word strings. A linear mixed-effects modeling approach was used to probe the effects of accumulating contextual constraints on N400 responses to individual words. Like young adults in prior work, older adults exhibited a classic word position context effect on the N400 in congruent sentences, although the magnitude of the effect was reduced in older relative to younger adults. Moreover, by modeling single-word variability in N400 responses, we observed robust effects of orthographic neighborhood density that were larger in older adults than the young, and preserved effects word frequency. Importantly, in older adults, frequency effects were not modulated by accumulating contextual constraints, unlike in the young. Collectively, these findings indicate that older adults are less likely (or able) to use accumulating top-down contextual constraints, and therefore rely more strongly on bottom-up lexical features to guide semantic access of individual words during sentence comprehension.