Chapter 1 Time for Meaning

The Costs (and Benefits?) of Effortful Listening for Older Adults: Insights from Simultaneous Electrophysiology, Pupillometry, and Memory

Although the impact of acoustic challenge on speech processing and memory increases as a person ages, older adults may engage in strategies that help them compensate for these demands. In the current preregistered study, older adults (n = 48) listened to sentences-presented in quiet or in noise-that were high constraint with either expected or unexpected endings or were low constraint with unexpected endings. Pupillometry and EEG were simultaneously recorded, and subsequent sentence recognition and word recall were measured. Like young adults in prior work, we found that noise led to increases in pupil size, delayed and reduced ERP responses, and decreased recall for unexpected words. However, in contrast to prior work in young adults where a larger pupillary response predicted a recovery of the N400 at the cost of poorer memory performance in noise, older adults did not show an associated recovery of the N400 despite decreased memory performance. Instead, we found that in quiet, increases in pupil size were associated with delays in N400 onset latencies and increased recognition memory performance. In conclusion, we found that transient variation in pupil-linked arousal predicted trade-offs between real-time lexical processing and memory that emerged at lower levels of task demand in aging. Moreover, with increased acoustic challenge, older adults still exhibited costs associated with transient increases in arousal without the corresponding benefits.

Flexible Conceptual Representations

A view that has been gaining prevalence over the past decade is that the human conceptual system is malleable, dynamic, context-dependent, and task-dependent, that is, flexible. Within the flexible conceptual representation framework, conceptual representations are constructed ad hoc, forming a different, idiosyncratic instantiation upon each occurrence. In this review, we scrutinize the neurocognitive literature to better understand the nature of this flexibility. First, we identify some key characteristics of these representations. Next, we consider how these flexible representations are constructed by addressing some of the open questions in this framework: We review the age-old question of how to reconcile flexibility with the apparent need for shareable stable definitions to anchor meaning and come to mutual understanding, as well as some newer questions we find critical, namely, the nature of relations among flexible representations, the role of feature saliency in activation, and the viability of all-or-none feature activations. We suggest replacing the debate about the existence of a definitional stable core that is obligatorily activated with a question of the degree and probability of activation of the information constituting a conceptual representation. We rely on published works to suggest that (1) prior featural salience matters, (2) feature activation may be graded, and (3) Bayesian updating of prior information according to current demands offers a viable account of how flexible representations are constructed. This proposal provides a theoretical mechanism for incorporating a changing momentary context into a constructed representation, while still preserving some of the concept's constituent meaning.

A predictive coding model of the N400

The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.

Insights from simultaneous EEG-fMRI and patient data illuminate the role of the anterior medial temporal lobe in N400 generation

The N400, a negative event-related potential (ERP) peaking approximately 400 ms after stimulus onset, is known to reflect the processing of semantic information. While scalp recordings have contributed to understanding the psychological processes underlying the N400, they have been limited in identifying its neural basis. However, recent intracranial ERP recordings and fMRI studies have shed light on the crucial role of the anterior medial temporal lobe (AMTL) in semantic information processing. These findings suggest that the N400 partially represents activity in the AMTL structures. To investigate the neural underpinnings of the N400 effect, we simultaneously recorded ERPs and event-related fMRI during a semantic priming paradigm in a sample of 12 young, healthy subjects. Additionally, we collected ERPs and structural brain data from older healthy adults and patients with amnestic mild cognitive impairment (aMCI), a population characterized by neurodegenerative changes in the AMTL. In our fMRI results, we identified bilateral loci in the AMTL as the global maxima. Employing an EEG-informed fMRI analysis, we explored trial-to-trial fluctuations in semantic processing by linking single-trial N400 amplitudes to the Blood Oxygen Level Dependent (BOLD) signal. This approach provided the first direct evidence linking the N400 recorded at the scalp level to the corresponding BOLD signal in the AMTL. Consistent with these findings, patients with aMCI exhibited a diminished N400 effect compared to healthy older adults. Furthermore, voxel-based morphometry analysis revealed a correlation between the magnitude of the N400 effect and the integrity of the AMTL. By integrating data from simultaneous EEG-fMRI, and patient studies, our research advances our understanding of the neural substrate of the N400 and highlights the critical involvement of the AMTL in semantic processing.

Graded and ungraded expectation patterns: Prediction dynamics during active comprehension

Language comprehension can be facilitated by the accurate prediction of upcoming words, but prediction effects are not ubiquitous, and comprehenders likely use predictive processing to varying degrees depending on task demands. To ascertain the processing consequences of prioritizing prediction, we here compared ERPs elicited when young adult participants simply read for comprehension with those collected in a subsequent block that required active prediction. We were particularly interested in frontally-distributed post-N400 effects for expected and unexpected words in strongly constraining contexts, which have previously been documented as two distinct patterns: an enhanced positivity ("anterior positivity") observed for prediction violations compared to words that are merely unpredictable (because they occur in weakly constraining sentences) and a distinction between expected endings in more constraining contexts and those same weakly constrained words ("frontal negativity" to the strongly predicted words). We found that the size of the anterior positivity effect was unchanged between passive comprehension and active prediction, suggesting that some processes related to prediction may engage state-like networks. On the other hand, the frontal negativity showed graded patterns from the interaction of task and sentence type. These differing patterns support the hypothesis that there are two separate effects with frontal scalp distributions that occur after the N400 and further suggest that the impact of violating predictions (as long as prediction is engaged at all) is largely stable across varying levels of effort/attention directed toward prediction, whereas other comprehension processes can be modulated by task demands.

The P600 as a continuous index of integration effort

The integration of word meaning into an unfolding utterance representation is a core operation of incremental language comprehension. There is considerable debate, however, as to which component of the ERP signal-the N400 or the P600-directly reflects integrative processes, with far reaching consequences for the temporal organization and architecture of the comprehension system. Multi-stream models maintaining the N400 as integration crucially rely on the presence of a semantically attractive plausible alternative interpretation to account for the absence of an N400 effect in response to certain semantic anomalies, as reported in previous studies. The single-stream Retrieval-Integration account posits the P600 as an index of integration, further predicting that its amplitude varies continuously with integrative effort. Here, we directly test these competing hypotheses using a context manipulation design in which a semantically attractive alternative is either available or not, and target word plausibility is varied across three levels. An initial self-paced reading study revealed graded reading times for plausibility, suggesting differential integration effort. A subsequent ERP study showed no N400 differences across conditions, and that P600 amplitude is graded for plausibility. These findings are inconsistent with the interpretation of the N400 as an index of integration, as no N400 effect emerged even in the absence of a semantically attractive alternative. By contrast, the link between plausibility, reading times, and P600 amplitude supports the view that the P600 is a continuous index of integration effort. More generally, our results support a single-stream architecture and eschew the need for multi-stream accounts.

Mapping access to meaning in adolescents with autism: Atypical lateralization and spatiotemporal patterns as a function of language ability

Individuals with autism spectrum disorders (ASD) vary in their language abilities, associated with atypical patterns of brain activity. However, few studies have examined the spatiotemporal profiles of lexico-semantic processing in ASD, particularly as a function of language heterogeneity. Thirty-nine high-functioning adolescents with ASD and 21 typically developing (TD) peers took part in a lexical decision task that combined semantic access with demands on cognitive control. Spatiotemporal characteristics of the processing stages were examined with a multimodal anatomically-constrained magnetoencephalography (aMEG) approach, which integrates MEG with structural MRI. Additional EEG data were acquired from a limited montage simultaneously with MEG. TD adolescents showed the canonical left-dominant activity in frontotemporal regions during both early (N250m) and late (N400m) stages of lexical access and semantic integration. In contrast, the ASD participants showed bilateral engagement of the frontotemporal language network, indicative of compensatory recruitment of the right hemisphere. The left temporal N400m was prominent in both groups, confirming preserved attempts to access meaning. In contrast, the left prefrontal N400m was reduced in ASD participants, consistent with impaired semantic/contextual integration and inhibitory control. To further investigate the impact of language proficiency, the ASD sample was stratified into high- and low-performing (H-ASD and L-ASD) subgroups based on their task accuracy. The H-ASD subgroup performed on par with the TD group and showed greater activity in the right prefrontal and bilateral temporal cortices relative to the L-ASD subgroup, suggesting compensatory engagement. The L-ASD subgroup additionally showed reduced and delayed left prefrontal N400m, consistent with more profound semantic and executive impairments in this subgroup. These distinct spatiotemporal activity profiles reveal the neural underpinnings of the ASD-specific access to meaning and provide insight into the phenotypic heterogeneity of language in ASD, which may be a result of different neurodevelopmental trajectories and adoption of compensatory strategies.

Taxonomically-related Word Pairs Evoke both N400 and LPC at Long SOA in Turkish

Semantic priming in Turkish was examined in 36 right-handed healthy participants in a delayed lexical decision task via taxonomic relations using EEG. Prime-target relations included related- unrelated- and pseudo-words. Taxonomically related words at long stimulus onset asynchrony (SOA) were shown to modulate N400 and late positive component (LPC) amplitudes. N400 semantic priming effect in the time window of 300-500 ms was the largest for pseudo-words, intermediate for semantically-unrelated targets, and smallest for semantically-related targets as a reflection of lexical-semantic retrieval. This finding contributes to the ERP literature showing how remarkably universal the N400 brain potential is, with similar effects across languages and orthography. The ERP data also revealed different influences of related, unrelated, and pseudo-word conditions on the amplitude of the LPC. Attention scores and mean LPC amplitudes of related words in parietal region showed a moderate correlation, indicating LPC may be related to "relationship-detection process".

Mu-desynchronization, N400 and corticospinal excitability during observation of natural and anatomically unnatural finger movements

The action observation networks (AON) (or the mirror neuron system) are the neural underpinnings of visuomotor integration and play an important role in motor control. Besides, one of the main functions of the human mirror neuron system is recognition of observed actions and the prediction of its outcome through the comparison with the internal mental motor representation. Previous studies focused on the human mirror neurons (MNs) activation during object-oriented movements observation, therefore intransitive movements observation effects on MNs activity remains relatively little-studied. Moreover, the dependence of MNs activation on the biomechanical characteristics of observed movement and their biological plausibility remained highly underexplored. In this study we proposed that naturalness of observed intransitive movement can modulate the MNs activity. Event-related desynchronization (ERD) of sensorimotor electroencephalography (EEG) rhythms, N400 event-related potentials (ERPs) component and corticospinal excitability were investigated in twenty healthy volunteers during observation of simple non-transitive finger flexion that might be either biomechanically natural or unnatural when finger wriggled out toward the dorsal side of palm. We showed that both natural and unnatural movements caused mu/beta-desynchronization, which gradually increased during the flexion phase and returned to baseline while observation of extension. Desynchronization of the mu-rhythm was significantly higher during observation of the natural movements. At the same time, beta-rhythm was not found to be sensitive to the action naturalness. Also, observation of unnatural movements caused an increased amplitude of the N400 component registered in the centro-parietal regions. We suggest that the sensitivity of N400 to intransitive action observation with no explicit semantic context might imply the broader role of N400 sources within AON. Surprisingly, no changes in corticospinal excitability were found. This lack of excitability modulation by action observation could be related with dependence of the M1 activity on the observed movement phase.

Context-based facilitation of semantic access follows both logarithmic and linear functions of stimulus probability

Stimuli are easier to process when context makes them predictable, but does context-based facilitation arise from preactivation of a limited set of relatively probable upcoming stimuli (with facilitation then linearly related to probability) or, instead, because the system maintains and updates a probability distribution across all items (with facilitation logarithmically related to probability)? We measured the N400, an index of semantic access, to words of varying probability, including unpredictable words. Word predictability was measured using both cloze probabilities and a state-of-the-art machine learning language model (GPT-2). We reanalyzed five datasets (n = 138) to demonstrate and then replicate that context-based facilitation on the N400 is graded, even among unpredictable words. Furthermore, we established that the relationship between word predictability and context-based facilitation combines linear and logarithmic functions. We argue that this composite function reveals properties of the mapping between words and semantic features and how feature- and word-related information is activated on-line.

Semantic systems are mentalistically activated for and by social partners

A recently discovered electrophysiological response, the social N400, suggests that we use our language system to track how social partners comprehend language. Listeners show an increased N400 response, when themselves not, only a communicative partner experiences a semantic incongruity. Does the N400 reflect purely semantic or mentalistic computations as well? Do we attribute language comprehension to communicative partners using our semantic systems? In five electrophysiological experiments we identified two subcomponents of the social N400. First, we manipulated the presence-absence of an Observer during object naming: the semantic memory system was activated by the presence of a social partner in addition to semantic predictions for the self. Next, we induced a false belief—and a consequent miscomprehension—in the Observer. Participants showed the social N400, over and above the social presence effect, to labels that were incongruent for the Observer, even though they were congruent for them. This effect appeared only if participants received explicit instructions to track the comprehension of the Observer. These findings suggest that the semantic systems of the brain are not merely sensitive to social information and contribute to the attribution of comprehension, but they appear to be mentalistic in nature.

Age-related Changes in the Structure and Dynamics of the Semantic Network

Normal aging has variable effects on language comprehension, perhaps because comprehension mechanisms vary in their dependence on network structure versus network dynamics. To test claims that processing dynamics are more affected by age than structure, we used EEG to measure and compare the impact of neighborhood size, a core measure of the structure of the lexico-semantic network, and repetition, a simple measure of processing dynamics, on single word processing. Older adults showed robust effects of neighborhood size on the N400, comparable to those elicited by young adults, but reduced effects of repetition. Furthermore, older adults with greater verbal fluency, print exposure, and reading comprehension showed greater repetition effects, suggesting some older adults can maintain processing dynamics that are similar to those of young adults. Thus, the organizational structure of the semantic network seems stable across normal aging, but (some) older adults may struggle to adjust activation states within that network.

Does the character-based dimension of stories impact narrative processing? An event-related potentials (ERPs) study

This event-related potentials (ERPs) study investigated online processes of integration of information relating to characters in narrative comprehension. The final sample included twenty-nine participants who read short third-person stories in which the plausibility of the characters' actions was manipulated. Stories were administered in three conditions: a character-based congruent condition including a target word that was consistent with the character's job; a character-based incongruent condition with a target word inconsistent with the character's job; a character-based neutral condition, narrating the action of a character presented by his/her proper name without information about his/her job. Results comparing the ERPs elicited by the experimental conditions revealed a greater negative amplitude of the N400 in the right temporal regions in response to the character-based incongruent compared to the character-based congruent narratives. This finding shows that implicit background character-based information affects the N400, with readers rapidly using this information to comprehend narratives.

Neuroplasticity in the phonological system: The PMN and the N400 as markers for the perception of non-native phonemic contrasts by late second language learners

Second language (L2) learners frequently encounter persistent difficulty in perceiving certain non-native sound contrasts, i.e., a phenomenon called "phonological deafness". However, if extensive L2 experience leads to neuroplastic changes in the phonological system, then the capacity to discriminate non-native phonemic contrasts should progressively improve. Such perceptual changes should be attested by modifications at the neurophysiological level. We designed an EEG experiment in which the listeners' perceptual capacities to discriminate second language phonemic contrasts influence the processing of lexical-semantic violations. Semantic congruency of critical words in a sentence context was driven by a phonemic contrast that was unique to the L2, English (e.g.,/ɪ/-/i:/, ship - sheep). Twenty-eight young adult native speakers of French with intermediate proficiency in English listened to sentences that contained either a semantically congruent or incongruent critical word (e.g., The anchor of theship/*sheepwas let down) while EEG was recorded. Three ERP effects were found to relate to increasing L2 proficiency: (1) a left frontal auditory N100 effect, (2) a smaller fronto-central phonological mismatch negativity (PMN) effect and (3) a semantic N400 effect. No effect of proficiency was found on oscillatory markers. The current findings suggest that neuronal plasticity in the human brain allows for the late acquisition of even hard-wired linguistic features such as the discrimination of phonemic contrasts in a second language. This is the first time that behavioral and neurophysiological evidence for the critical role of neural plasticity underlying L2 phonological processing and its interdependence with semantic processing has been provided. Our data strongly support the idea that pieces of information from different levels of linguistic processing (e.g., phonological, semantic) strongly interact and influence each other during online language processing.

Evidence for self-positivity bias in a subliminal self-cue: An event-related potential study

Previous studies studies indicate that individuals tend to integrate positive information into their self-concept. However, whether such self-positivity bias would still be observed without an explicit self-related cue is unknown. In the present study, 29 participants were asked to evaluate a series of positive and negative trait adjectives, after the participants were presented with their own name or another name subliminally. During the task, their electroencephalograms were recorded. The results showed participants responded faster to positive traits than to negative traits in the self-name cue conditions. In addition, both the latencies and the amplitudes of the N400 showed significant interaction between name-cue and valence in N400 (240-440 ms) amplitudes. The earlier N400 latencies and smaller N400 amplitudes were associated with positive traits in the self-name cue. These results suggested that the self-positivity bias can also be observed in a subliminally presenting self-cue, indicating the robustness of self-positivity bias.

A review on the electroencephalography markers of Stroop executive control processes

The present article on executive control addresses the issue of the locus of the Stroop effect by examining neurophysiological components marking conflict monitoring, interference suppression, and conflict resolution. Our goal was to provide an overview of a series of determining neurophysiological findings including neural source reconstruction data on distinct executive control processes and sub-processes involved in the Stroop task. Consistently, a fronto-central N2 component is found to reflect conflict monitoring processes, with its main neural generator being the anterior cingulate cortex (ACC). Then, for cognitive control tasks that involve a linguistic component like the Stroop task, the N2 is followed by a centro-posterior N400 and subsequently a late sustained potential (LSP). The N400 is mainly generated by the ACC and the prefrontal cortex (PFC) and is thought to reflect interference suppression, whereas the LSP plausibly reflects conflict resolution processes. The present overview shows that ERP constitute a reliable methodological tool for tracing with precision the time course of different executive processes and sub-processes involved in experimental tasks involving a cognitive conflict. Future research should shed light on the fine-grained mechanisms of control respectively involved in linguistic and non-linguistic tasks.

Brain indices associated with semantic cues prior to and after a word in noise

It is well established that identification of words in noise improves when it is preceded by a semantically related word, but comparatively little is known about the effect of subsequent context in guiding word in noise identification. We build on the findings of a previous behavioural study (Chan & Alain, 2019) by measuring neuro-electric brain activity while manipulating the semantic content of a cue that either preceded or followed a word in noise. Participants were more accurate in identifying the word in noise when it was preceded or followed by a cue that was semantically related. This gain in accuracy coincided with a late positive component, which was time-locked to the word in noise when preceded by a cue and time-locked to the cue when it followed the word in noise. Distributed source analyses of this positive component revealed different patterns in source activity between the two temporal conditions. The effects of relatedness also generated an event-related potential modulation around 400 ms (N400) that was present at cue presentation when it followed the word in noise, but not for the word in noise when preceded by the cue, consistent with findings regarding its sensitivity to signal degradation. Exploratory analyses examined a subset of data based on participants' subjective perceived clarity, which revealed a posterior deflection over the left hemisphere that showed a relatedness effect. We discuss these findings in light of research on prediction as well as a reflective attention framework.

Neural oscillations and event-related potentials reveal how semantic congruence drives long-term memory in both young and older humans

Long-term memory can improve when incoming information is congruent with known semantic information. This so-called congruence effect has widely been shown in younger adults, but age-related changes and neural mechanisms remain unclear. Here, congruence improved recognition memory in younger and older adults (i.e. congruence effect), with only weak evidence for age-related decline in one behavioral study. In an EEG study, however, no significant behavioral differences in the congruence effect could be observed between age-groups. In line with this observation, electroencephalography data show that, in both groups, congruence led to widespread differences in Event-Related Potentials (ERPs), starting at around 400 ms after stimulus onset, and theta, alpha and beta oscillations (4-20 Hz). Importantly, these congruence-related ERPs were associated to increases in memory performance for congruent items, in both age groups. Finally, the described ERPs and neural oscillations in the theta-alpha range (5-13 Hz) were less pronounced in the elderly despite a preserved congruence effect. Together, semantic congruence increases long-term memory across the lifespan, and, at the neural level, this could be linked to neural oscillations in the theta, alpha and beta range, as well as ERPs that were previously associated with semantic processing.

Exposure to linguistic labels during childhood modulates the neural architecture of race categorical perception

Perceptually categorizing a face to its racial belonging may have important consequences on interacting with people. However, race categorical perception (CP) has been scarcely investigated nor its developmental pathway. In this study, we tested the neurolinguistics rewiring hypothesis, stating that language acquisition modulates the brain processing of social perceptual categories. Accordingly, we investigated the electrophysiological correlates of race CP in a group of adults and children between 3 and 5 years of age. For both groups we found a greater modulation of the N400 connected with the processing of between category boundaries (i.e., faces belonging to different race groups) than within-category boundaries (i.e., different faces belonging to the same race group). This effect was the same in both adults and children, as shown by the comparable between-group amplitude of the differential wave (DW) elicited by the between-category faces. Remarkably, this effect was positively correlated with racial-labels acquisition, but not with age, in children. Finally, brain source analysis revealed the activation of a more modularized cortical network in adults than in children, with unique activation of the left superior temporal gyrus (STG) and the inferior frontal gyrus (IFG), which are areas connected to language processing. These are the first results accounting for an effect of language in rewiring brain connectedness when processing racial categories.

Individual Differences in Reading Speed are Linked to Variability in the Processing of Lexical and Contextual Information: Evidence from Single-trial Event-related Brain Potentials

In the current paper, we examined the effects of lexical (e.g. word frequency, orthographic neighborhood density) and contextual (e.g. word predictability in the form of cloze probability) features on single-trial event-related brain potentials in a self-paced reading paradigm. Critically, we examined whether individual differences in reading speed modulated single-trial effects on the N400, an ERP component linked to semantic memory access. Consistent with past work, we found that word frequency effects on the N400 were attenuated with increasing predictability. However, effects of orthographic neighborhood density were robust across all levels of predictability. Importantly, individual differences in reading speed moderated the influence of both frequency and predictability (but not orthographic neighborhood density) on the N400, such that slower readers showed reduced effects compared to faster readers. These data show that different lexical factors influence word processing through dissociable mechanisms. Our findings support a dynamic semantic-memory access model of the N400, in which information at multiple levels (lexical, sentential, individual) simultaneously contributes to the unfolding neural dynamics of comprehension.

Adult Age Differences in the Use of Conceptual Combination as an Associative Encoding Strategy

It is well-established that aging impairs memory for associations more than it does memory for single items. Aging also impacts processes involved in online language comprehension, including the ability to form integrated, message-level representations. These changes in comprehension processes could impact older adults’ associative memory performance, perhaps by reducing or altering the effectiveness of encoding strategies that encourage semantic integration. The present study examined age differences in the use of a strategy termed conceptual combination, which involves integrating two words (e.g., “winter” and “salad”) into a single concept (“a salad for winter”). We recorded ERPs while participants studied unrelated noun pairs using a strategy that either did or did not encourage conceptual combination. We also varied the concreteness of the first noun in each pair in order to measure compositional concreteness effects, or ERP differences at the second noun due to the concreteness of the first noun. At the first nouns, older adults showed word-level concreteness effects that were similar to those of younger adults. However, compositional concreteness effects were diminished in older adults, consistent with reduced semantic integration. Older adults’ associative memory performance was better for word pairs studied during the conceptual combination task versus the non-combinatory encoding task; however, the magnitude of the age-related associative memory deficit did not differ between tasks. Finally, analyses of both memory accuracy and trial-by-trial ratings of perceived combination success suggested that older adults had disproportionate difficulty applying the conceptual combination strategy to word pairs that began with abstract nouns. Overall, these results indicate that changes to integrative language processing that occur with age are not independent of – and may sometimes exacerbate – age-related memory decline.

Having your cake and eating it too: Flexibility and power with mass univariate statistics for ERP data

ERP studies produce large spatiotemporal data sets. These rich data sets are key to enabling us to understand cognitive and neural processes. However, they also present a massive multiple comparisons problem, potentially leading to a large number of studies with false positive effects (a high Type I error rate). Standard approaches to ERP statistical analysis, which average over time windows and regions of interest, do not always control for Type I error, and their inflexibility can lead to low power to detect true effects. Mass univariate approaches offer an alternative analytic method. However, they have thus far been viewed as appropriate primarily for exploratory statistical analysis and only applicable to simple designs. Here, we present new simulation studies showing that permutation-based mass univariate tests can be employed with complex factorial designs. Most importantly, we show that mass univariate approaches provide slightly greater power than traditional spatiotemporal averaging approaches when strong a priori time windows and spatial regions are used. Moreover, their power decreases only modestly when more exploratory spatiotemporal parameters are used. We argue that mass univariate approaches are preferable to traditional spatiotemporal averaging analysis approaches for many ERP studies.

Inferior Longitudinal Fasciculus' Role in Visual Processing and Language Comprehension: A Combined MEG-DTI Study

The inferior longitudinal fasciculus (ILF) is a white matter tract that connects the occipital and the temporal lobes. ILF abnormalities have been associated with deficits in visual processing and language comprehension in dementia patients, thus suggesting that its integrity is important for semantic processing. However, it remains elusive whether ILF microstructural organization per se impacts the visual semantic processing efficiency in the healthy brain. The present study aims to investigate whether there is an association between ILF’s microstructural organization and visual semantic processing at the individual level. We hypothesized that the efficiency of visual semantic processing positively correlates with the degree of anisotropy of the ILF. We studied 10 healthy right-handed subjects. We determined fractional anisotropy (FA) of the ILF using diffusion tensor imaging (DTI). We extracted N400m latency and amplitude from magnetoencephalography (MEG) signals during a visual semantic decision task. N400m and mean FA of the ILF were left lateralized with the higher FA value in the left hemisphere. Inter-individual analysis showed that FA of the ILF negatively correlated with the N400m latency and amplitude, which suggests that high ILF anisotropy is associated with more efficient semantic processing. In summary, our findings provide supporting evidence for a role of the ILF in language comprehension.

Double-checking emotions: Valence and emotion category in contextual integration of facial expressions of emotion

Faces showing happy, angry or fearful expressions were presented in emotionally congruent or incongruent situational contexts (short sentences describing events that would usually provoke happiness, anger or fear). The participants were assigned the task of judging whether the expression was appropriate or not to the context (congruency judgment task). Effects of emotional congruency were observed at both the behavioral and electrophysiological levels. Behavioral results showed evidence of congruency effects based on specific emotion content (e.g., less accurate and slower responses to fear faces in angry contexts). Event-related-potentials (ERP) results also showed emotional congruency effects at different post-stimulus onset latencies, beginning with the face-sensitive N170 component. An effect of emotional congruency was also shown on the N400 component that is typically sensitive to semantic congruency. Finally, a late positive potential (LPP), appearing at 450-650 ms post-stimulus onset showed a complex pattern of effects with modulations driven by the different combinations of contexts and target expressions. These results are interpreted in terms of a double process of valence and emotion checking that is supposed to underlie affective processing and contextual integration of facial expressions of emotions.

Talker-specific predictions during language processing

Language comprehension is shaped by world knowledge. After hearing about "a farm animal," meanings of typical ("cow") versus atypical exemplars ("ox") are more accessible, as evidenced by N400 responses. Moreover, atypical exemplars elicit a larger post-N400 frontal positivity than typical and incongruous ("ivy") exemplars, indexing the integration of unexpected information. Do listeners adapt this category knowledge to specific talkers? We first replicated typicality effects in the auditory modality. Then, we extended the design to a two-talker context: talkers alternated cueing (Bob: "Susan, name a farm animal") and answering (Susan: "cow"). Critically, participants first heard interviews in which one talker revealed strong associations with atypical exemplars (Susan works on an ox farm). We observed increased frontal positivity to a typical exemplar ("cow") said by Susan compared to Bob, indicating participants appreciated that the typical exemplar was atypical for Susan. These results suggest that comprehenders can tailor their expectations to the talker.

What does "it" mean, anyway? Examining the time course of semantic activation in reference resolution

Pronouns serve a critical referential function, yet the cognitive processes engaged during pronoun comprehension remain incompletely understood. One view is that encountering a pronoun leads the comprehender to reactivate the semantic features of its antecedent. We examined this by manipulating the concreteness of a noun antecedent and assessing whether an Event Related Potential (ERP) concreteness effect was elicited at a downstream pronoun. We observed a robust concreteness effect at the noun, but no similar effect at the pronoun. We also examined whether N400 semantic priming from the antecedent would increase on content words shortly following the pronoun, relative to those preceding it. Again, although we observed semantic priming following the noun, it did not increase following the pronoun. Our data suggest that pronouns do not induce the activation of (much) new semantic information in long-term memory, perhaps instead triggering an attentional shift towards their antecedents' extant representations within working memory.

Contextual constraints on lexico-semantic processing in aging: Evidence from single-word event-related brain potentials

The current study reports the effects of accumulating contextual constraints on neural indices of lexico-semantic processing (i.e., effects of word frequency and orthographic neighborhood) as a function of normal aging. Event-related brain potentials were measured from a sample of older adults as they read sentences that were semantically congruent, provided only syntactic constraints (syntactic prose), or were random word strings. A linear mixed-effects modeling approach was used to probe the effects of accumulating contextual constraints on N400 responses to individual words. Like young adults in prior work, older adults exhibited a classic word position context effect on the N400 in congruent sentences, although the magnitude of the effect was reduced in older relative to younger adults. Moreover, by modeling single-word variability in N400 responses, we observed robust effects of orthographic neighborhood density that were larger in older adults than the young, and preserved effects word frequency. Importantly, in older adults, frequency effects were not modulated by accumulating contextual constraints, unlike in the young. Collectively, these findings indicate that older adults are less likely (or able) to use accumulating top-down contextual constraints, and therefore rely more strongly on bottom-up lexical features to guide semantic access of individual words during sentence comprehension.

Priming production: Neural evidence for enhanced automatic semantic activity preceding language production in schizophrenia

Introduction

Lexico-semantic disturbances are considered central to schizophrenia. Clinically, their clearest manifestation is in language production. However, most studies probing their underlying mechanisms have used comprehension or categorization tasks. Here, we probed automatic semantic activity prior to language production in schizophrenia using event-related potentials (ERPs).

Methods

19 people with schizophrenia and 16 demographically-matched healthy controls named target pictures that were very quickly preceded by masked prime words. To probe automatic semantic activity prior to production, we measured the N400 ERP component evoked by these targets. To determine the origin of any automatic semantic abnormalities, we manipulated the type of relationship between prime and target such that they overlapped in (a) their semantic features (semantically related, e.g. "cake" preceding a < picture of a pie >, (b) their initial phonemes (phonemically related, e.g. "stomach" preceding a < picture of a starfish >), or (c) both their semantic features and their orthographic/phonological word form (identity related, e.g. "socks" preceding a < picture of socks >). For each of these three types of relationship, the same targets were paired with unrelated prime words (counterbalanced across lists). We contrasted ERPs and naming times to each type of related target with its corresponding unrelated target.

Results

People with schizophrenia showed abnormal N400 modulation prior to naming identity related (versus unrelated) targets: whereas healthy control participants produced a smaller amplitude N400 to identity related than unrelated targets, patients showed the opposite pattern, producing a larger N400 to identity related than unrelated targets. This abnormality was specific to the identity related targets. Just like healthy control participants, people with schizophrenia produced a smaller N400 to semantically related than to unrelated targets, and showed no difference in the N400 evoked by phonemically related and unrelated targets. There were no differences between the two groups in the pattern of naming times across conditions.

Conclusion

People with schizophrenia can show abnormal neural activity associated with automatic semantic processing prior to language production. The specificity of this abnormality to the identity related targets suggests that that, rather than arising from abnormalities of either semantic features or lexical form alone, it may stem from disruptions of mappings (connections) between the meaning of words and their form.

Using Event-related Potentials to Inform the Neurocognitive Processes Underlying Knowledge Extension through Memory Integration

To build a general knowledge base, it is imperative that individuals acquire, integrate, and further extend knowledge across experiences. For instance, in one episode an individual may learn that George Washington was the first president. In a separate episode they may then learn that Washington was the commander of the Continental Army. Integration of the information in memory may then support self-derivation of the new knowledge that the leader of the Continental Army was also the first president. Despite a considerable amount of fMRI research aimed at further elucidating the neuroanatomical regions supporting this ability, a consensus has yet to be reached with regards to the precise neurocognitive processes involved. In the present research, we capitalized on the high temporal resolution of event-related potentials (ERPs) to inform the time course of processes elicited during successful integration and further extension of new factual knowledge. Adults read novel, related stem facts and were tested for self-derivation of novel integration facts while ERPs were recorded. Consistent with current theoretical models, memory integration was first triggered by novelty detection within 400 msec of experience of a second, related stem fact. Two additional temporally staged encoding processes were then observed interpreted to reflect (1) explicit meaning comprehension and (2) representation of the integrated relation in memory. During the test for self-derivation, a single ERP was elicited, which presumably reflected retrieval and/or recombination of previously integrated knowledge. Together, the present research provides important insight into the time course of neurocognitive processing associated with the formation of a knowledge base.

An Association between Auditory-Visual Synchrony Processing and Reading Comprehension: Behavioral and Electrophysiological Evidence

The perceptual system integrates synchronized auditory-visual signals in part to promote individuation of objects in cluttered environments. The processing of auditory-visual synchrony may more generally contribute to cognition by synchronizing internally generated multimodal signals. Reading is a prime example because the ability to synchronize internal phonological and/or lexical processing with visual orthographic processing may facilitate encoding of words and meanings. Consistent with this possibility, developmental and clinical research has suggested a link between reading performance and the ability to compare visual spatial/temporal patterns with auditory temporal patterns. Here, we provide converging behavioral and electrophysiological evidence suggesting that greater behavioral ability to judge auditory-visual synchrony (Experiment 1) and greater sensitivity of an electrophysiological marker of auditory-visual synchrony processing (Experiment 2) both predict superior reading comprehension performance, accounting for 16% and 25% of the variance, respectively. These results support the idea that the mechanisms that detect auditory-visual synchrony contribute to reading comprehension.

Pace Yourself: Intraindividual Variability in Context Use Revealed by Self-paced Event-related Brain Potentials

Event-related brain potentials (ERPs) have revealed multiple mechanisms by which contextual constraints impact language processing. At the same time, little work has examined the trial-to-trial dynamics of context use in the brain. In the current study, we probed intraindividual variability in behavioral and neural indices of context processing during reading. In a concurrent self-paced reading and ERP paradigm, participants read sentences that were either strongly or weakly constraining completed with an expected or unexpected target word. Our findings revealed substantial within-subject variability in behavioral and neural responses to contextual constraints. First, context-based amplitude reductions of the N400, a component linked to semantic memory access, were largest among trials eliciting the slowest RTs. Second, the RT distribution of unexpected words in strongly constraining contexts was positively skewed, reflecting an increased proportion of very slow RTs to trials that violated semantic predictions. Among those prediction-violating trials eliciting faster RTs, a late sustained anterior positivity was observed. However, among trials producing the differentially slowed RTs to prediction violations, we observed a markedly earlier effect of constraint in the form of an anterior N2, a component linked to conflict resolution and the cognitive control of behavior. The current study provides the first neurophysiological evidence for the direct role of cognitive control functions in the volitional control of reading. Collectively, our findings suggest that context use varies substantially within individual participants and that coregistering behavioral and neural indices of online sentence processing offers a window into these single-item dynamics.

Functional significance of the semantic P600: evidence from the event-related brain potential source localization

Previous event-related potentials studies of sentence comprehension have usually associated syntactic repair/reanalysis processes with the P600 component. However, the functional significance of the P600 was recently questioned on the basis of the observation of P600 effects in response to the processing of semantically anomalous sentences. In the present study, we investigated the functional significance of the 'semantic P600' using a semantic violation paradigm with a focus-on-syntax instruction. Using a source localization analysis, we tested three alternative hypotheses on the function of the P600: (i) a syntactic function, (ii) an executive function, and (iii) a semantic/integrational function. We assumed that distinct neuronal generators should reflect each of these functions. Although the syntactic generators are expected to be mainly located in the left inferior frontal gyrus, the semantic generators should be observed in the left superior temporal gyrus as well as in the right anterior prefrontal cortex (semantic retrieval); moreover, the generator of executive function (conflict monitoring) should be found in the anterior cingulate cortex. Critically, we defined a dipole model using 17 dipoles, 14 of which were placed in the three regions of interest corresponding to our hypotheses, namely, syntactic, executive, and semantic regions. Our data showed that the P600 effect was significant in the anterior cingulate cortex and marginally significant in the right anterior prefrontal cortex. This finding suggests that the P600 might reflect more general mechanisms of conflict monitoring and semantic reinterpretation leading to a retrieval of world knowledge from long-term memory rather purely syntactic processes.

Switching of auditory attention in "cocktail-party" listening: ERP evidence of cueing effects in younger and older adults

Verbal communication in a "cocktail-party situation" is a major challenge for the auditory system. In particular, changes in target speaker usually result in declined speech perception. Here, we investigated whether speech cues indicating a subsequent change in target speaker reduce the costs of switching in younger and older adults. We employed event-related potential (ERP) measures and a speech perception task, in which sequences of short words were simultaneously presented by four speakers. Changes in target speaker were either unpredictable or semantically cued by a word within the target stream. Cued changes resulted in a less decreased performance than uncued changes in both age groups. The ERP analysis revealed shorter latencies in the change-related N400 and late positive complex (LPC) after cued changes, suggesting an acceleration in context updating and attention switching. Thus, both younger and older listeners used semantic cues to prepare changes in speaker setting.

Revisiting the incremental effects of context on word processing: Evidence from single-word event-related brain potentials

The amplitude of the N400-an event-related potential (ERP) component linked to meaning processing and initial access to semantic memory-is inversely related to the incremental buildup of semantic context over the course of a sentence. We revisited the nature and scope of this incremental context effect, adopting a word-level linear mixed-effects modeling approach, with the goal of probing the continuous and incremental effects of semantic and syntactic context on multiple aspects of lexical processing during sentence comprehension (i.e., effects of word frequency and orthographic neighborhood). First, we replicated the classic word-position effect at the single-word level: Open-class words showed reductions in N400 amplitude with increasing word position in semantically congruent sentences only. Importantly, we found that accruing sentence context had separable influences on the effects of frequency and neighborhood on the N400. Word frequency effects were reduced with accumulating semantic context. However, orthographic neighborhood was unaffected by accumulating context, showing robust effects on the N400 across all words, even within congruent sentences. Additionally, we found that N400 amplitudes to closed-class words were reduced with incrementally constraining syntactic context in sentences that provided only syntactic constraints. Taken together, our findings indicate that modeling word-level variability in ERPs reveals mechanisms by which different sources of information simultaneously contribute to the unfolding neural dynamics of comprehension.

Processing changes when listening to foreign-accented speech

This study investigates the mechanisms responsible for fast changes in processing foreign-accented speech. Event Related brain Potentials (ERPs) were obtained while native speakers of Spanish listened to native and foreign-accented speakers of Spanish. We observed a less positive P200 component for foreign-accented speech relative to native speech comprehension. This suggests that the extraction of spectral information and other important acoustic features was hampered during foreign-accented speech comprehension. However, the amplitude of the N400 component for foreign-accented speech comprehension decreased across the experiment, suggesting the use of a higher level, lexical mechanism. Furthermore, during native speech comprehension, semantic violations in the critical words elicited an N400 effect followed by a late positivity. During foreign-accented speech comprehension, semantic violations only elicited an N400 effect. Overall, our results suggest that, despite a lack of improvement in phonetic discrimination, native listeners experience changes at lexical-semantic levels of processing after brief exposure to foreign-accented speech. Moreover, these results suggest that lexical access, semantic integration and linguistic re-analysis processes are permeable to external factors, such as the accent of the speaker.

Auditory and motion metaphors have different scalp distributions: an ERP study

While many links have been established between sensory-motor words used literally (kick the ball) and sensory-motor regions of the brain, it is less clear whether metaphorically used words (kick the habit) also show such signs of “embodiment.” Additionally, not much is known about the timing or nature of the connection between language and sensory-motor neural processing. We used stimuli divided into three figurativeness conditions—literal, metaphor, and anomalous—and two modality conditions—auditory (Her limousine was a privileged snort) and motion (The editorial was a brass-knuckle punch). The conditions were matched on a large number of potentially confounding factors including cloze probability. The electroencephalographic response to the final word of each sentence was measured at 64 electrode sites on the scalp of 22 participants and event-related potentials (ERPs) calculated. Analysis revealed greater amplitudes for metaphorical than literal sentences in both 350–500 ms and 500–650 ms timeframes. Results supported the possibility of different neural substrates for motion and auditory sentences. Greater differences for motion sentences were seen in the left posterior and left central electrode sites than elsewhere on the scalp. These findings are consistent with a sensory-motor neural categorization of language and with the integration of modal and amodal information during the N400 and P600 timeframes.