Brain potentials during sentence verification: late negativity and long-term memory strength

Strong Prediction: Language Model Surprisal Explains Multiple N400 Effects

Theoretical accounts of the N400 are divided as to whether the amplitude of the N400 response to a stimulus reflects the extent to which the stimulus was predicted, the extent to which the stimulus is semantically similar to its preceding context, or both. We use state-of-the-art machine learning tools to investigate which of these three accounts is best supported by the evidence. GPT-3, a neural language model trained to compute the conditional probability of any word based on the words that precede it, was used to operationalize contextual predictability. In particular, we used an information-theoretic construct known as surprisal (the negative logarithm of the conditional probability). Contextual semantic similarity was operationalized by using two high-quality co-occurrence-derived vector-based meaning representations for words: GloVe and fastText. The cosine between the vector representation of the sentence frame and final word was used to derive contextual cosine similarity estimates. A series of regression models were constructed, where these variables, along with cloze probability and plausibility ratings, were used to predict single trial N400 amplitudes recorded from healthy adults as they read sentences whose final word varied in its predictability, plausibility, and semantic relationship to the likeliest sentence completion. Statistical model comparison indicated GPT-3 surprisal provided the best account of N400 amplitude and suggested that apparently disparate N400 effects of expectancy, plausibility, and contextual semantic similarity can be reduced to variation in the predictability of words. The results are argued to support predictive coding in the human language network.

Dissociations of subliminal and supraliminal self-face from other-face processing: behavioral and ERP evidence

Self-related information has been found to be processed more quickly and accurately in studies with supraliminal self-stimuli and traditional paradigms such as masked priming. We conducted two experiments to investigate whether subliminal self-face processing enjoys this advantage and the neural correlates of processing self-faces at both subliminal and supraliminal levels. We found that self-faces were quicker than famous-other faces to gain dominance against dynamic noise patterns during prolonged interocular suppression to enter awareness (Experiment 1). Meanwhile, subliminal contrast of self- and famous-other face processing was reflected in a reduced early vertex positive potential (VPP) component, whereas supraliminal self-other face differentiation was reflected in an enhanced N170, as well as a more positive late component (300-580ms, Experiment 2) to the self-face. The clear dissociations of self- and other-face processing found across our two experiments validate the self advantage. Our findings also contribute to understandings of the mechanisms underlying self-face processing at different levels of awareness.

Validating the truth of propositions: behavioral and ERP indicators of truth evaluation processes

We investigated processes of truth validation during reading. Participants responded to 'true' and 'false' probes after reading simple true or false sentences. Compatible sentence/probe combinations (true/'true', false/'false') facilitated responding compared with incompatible combinations (true/'false', false/'true'), indicating truth validation. Evidence for truth validation was obtained after inducing an evaluative mindset but not after inducing a non-evaluative mindset, using additional intermixed tasks requiring true/false decisions or sentence comparisons, respectively. Event-related potentials revealed an increased late negativity (500-1000 ms after onset of the last word of sentences) for false compared with true sentences. Paralleling behavioral results, this electroencephalographic marker only obtained in the evaluative mindset condition. Further, mere semantic mismatches between subject and object of sentences led to an elevated N400 for both mindset conditions. Taken together, our findings suggest that truth validation is a conditionally automatic process that is dependent on the current task demands and resulting mindset, whereas the processing of word meaning and semantic relations between words proceeds in an unconditionally automatic fashion.

Right or Wrong?

How does the brain respond to statements that clash with a person's value system? We recorded event-related brain potentials while respondents from contrasting political-ethical backgrounds completed an attitude survey on drugs, medical ethics, social conduct, and other issues. Our results show that value-based disagreement is unlocked by language extremely rapidly, within 200 to 250 ms after the first word that indicates a clash with the reader's value system (e.g., “I think euthanasia is an acceptable/unacceptable…”). Furthermore, strong disagreement rapidly influences the ongoing analysis of meaning, which indicates that even very early processes in language comprehension are sensitive to a person's value system. Our results testify to rapid reciprocal links between neural systems for language and for valuation.

Electrophysiology reveals semantic memory use in language comprehension

The physical energy that we refer to as a word, whether in isolation or embedded in sentences, takes its meaning from the knowledge stored in our brains through a lifetime of experience. Much empirical evidence indicates that, although this knowledge can be used fairly flexibly, it is functionally organized in 'semantic memory' along a number of dimensions, including similarity and association. Here, we review recent findings using an electrophysiological brain component, the N400, that reveal the nature and timing of semantic memory use during language comprehension. These findings show that the organization of semantic memory has an inherent impact on sentence processing. The left hemisphere, in particular, seems to capitalize on the organization of semantic memory to pre-activate the meaning of forthcoming words, even if this strategy fails at times. In addition, these electrophysiological results support a view of memory in which world knowledge is distributed across multiple, plastic-yet-structured, largely modality-specific processing areas, and in which meaning is an emergent, temporally extended process, influenced by experience, context, and the nature of the brain itself.

Self-recognition and the right prefrontal cortex

Although the anatomical and functional substrates subserving face recognition have been subject to extensive investigation, the underpinnings of self-face recognition are not well understood. Given the evidence that own-face recognition has been demonstrated by a select number of species, it is intriguing to speculate whether self-face recognition is accomplished via a 'self-network' or simply a 'face-network' within the brain. Furthermore, the relationship of self-recognition to other self-processes, such as self-evaluation and autobiographical retrieval, are not clearly defined. However, data from fMRI, ERPs and repetitive transcranial magnetic stimulation as well as from split-brain studies and patients with focal lesions, indicate that the prefrontal cortex, with possible right hemisphere lateralization, may be a preferential component in self-recognition. Studies using these methods, as well as PET, have indicated that the self-processes of self-evaluation and autobiographical memory preferentially engage networks within the right fronto-temporal region. Although it is highly improbable that there is a 'self-recognition' or 'self' center, it appears that there may be a bias for the processing of 'self' within the right prefrontal cortex.

Many-valued logic and event-related potentials

In previous experiments on event-related potentials (ERPs) during linguistic judgments, the subjects' decisions have been categorical (e.g., true vs false). In this experiment, more realistic variations in truth value and subject certainty were used. Thirty-eight naive undergraduates read a story about a fictional murder. ERPs were recorded as the subjects rated the strength of their beliefs about statements relating to suspects in the crime. Because no subject was sure which of the suspects was guilty of committing the crime, binary (true-false) category judgments were inappropriate. Three components of the ERP waveforms were affected by the experimental manipulations. An early positive component was largest to sentences concerning the suspect considered most likely to have committed the crime. A subsequent broad posterior positivity (LPC) also showed significant sentence-type differences, but it was larger to sentences considered probable--whether they were true or false--than to more ambiguous sentences. A third ERP component (N400) was negative at midline electrode sites and peaked at approximately 420 ms. Subjects' truth-value judgments had no effect on the N400. N400 was, however, affected by the subject's task. It was more negative when subjects made graded judgments about truth value than when they made binary true-false judgments. Overall, naturalistic judgments of sentence validity produced a variety of brain responses that reflected different aspects of linguistic decision making.

Event-related electric microstates of the brain differ between words with visual and abstract meaning

The present study shows that different neural activity during mental imagery and abstract mentation can be assigned to well-defined steps of the brain's information-processing. During randomized visual presentation of single, imagery-type and abstract-type words, 27 channel event-related potential (ERP) field maps were obtained from 25 subjects (sequence-divided into a first and second group for statistics). The brain field map series showed a sequence of typical map configurations that were quasi-stable for brief time periods (microstates). The microstates were concatenated by rapid map changes. As different map configurations must result from different spatial patterns of neural activity, each microstate represents different active neural networks. Accordingly, microstates are assumed to correspond to discrete steps of information-processing. Comparing microstate topographies (using centroids) between imagery- and abstract-type words, significantly different microstates were found in both subject groups at 286-354 ms where imagery-type words were more right-lateralized than abstract-type words, and at 550-606 ms and 606-666 ms where anterior-posterior differences occurred. We conclude that language-processing consists of several, well-defined steps and that the brain-states incorporating those steps are altered by the stimuli's capacities to generate mental imagery or abstract mentation in a state-dependent manner.

Brain Activity Associated with Syntactic Incongruencies in Words and Pseudo-Words

Event-related brain potentials (EMS) were recorded while normal German subjects read either simple declarative sen- tences made up from real German words, or sentences that contained German pseudo-words instead of nouns and verbs. The verb (pseudo-verb) of the sentences disagreed in number with the subject noun (pseudo-noun) in 50% of the sentences. The subjects had the task either to read the sentences for an interspersed memory test (memory condition, pseudeword sentences only) or to make a syntactic judgment after each real-word/pseudo-word sentence.

While in the real-word condition a late and widespread positivity resembling the previously described syntactic positive shift was found for the disagreeing verbs, a negativity with an onset latency of about 300 msec was seen for the disagreeing pseudo-verbs. In the pseudo-word conditions no positivity followed the initial negativity. This dissociation of negative and positive waves occurring in response to morphosyntactic mismatches by the pseudo/real-word manipulation suggests that the positive shift is a concomitant of a recomputation routine initiated to account for the number incongruency. This routine is based upon the semantics of the sentence and therefore is not observed in the pseudo-word conditions. The earlier negativity, on the other hand, appears to be a more direct index of morphosyntactic incongruency.

Modulation of Event-Related Potentials by Semantic Priming: Effects of Color-Cued Selective Attention

The processing nature of N400, an event-related brain potential (ERP) component associated with semantic processing, was investigated in a paradigm combining a semantic priming lexical decision task and color-cued selective attention. Semantic priming effects on ERPs and reaction time (RT) were examined when targets and preceding semantically related primes were either both attended or both unattended, and when only either the prime or target was the focus of attention. Priming effects were determined by comparing semantically primed target ERPs (and RTs when appropriate) to their prime (in those conditions where the prime and target had the same attentional status) and/or to an unprimed control target matched to the attentional status of the primed target. Control stimuli were examined for ERP effects of color-cued selective attention unconfounded by priming factors. Experiment 1 required overt responses to words and nonwords in a binary choice task, while Experiment 2 required response only to nonwords. RTs in Experiment 1 indicated facilitatory priming effects to all semantically primed attended targets. In Experiments 1 and 2 ERPs to primes and controls were consistently more negative than the primed target trace in the N400 latency range in conditions with attended primes, suggesting priming effects on N400 are contingent on attentional processing of the prime. Removal of the attention manipulation (Experiment 3) resulted in an N400 component with a well-defined peak not evident in the first two experiments, indicating modulation of N400 by overlapping effects of attention.

Language-Related ERPs: Scalp Distributions and Modulation by Word Type and Semantic Priming

Event-related potentials (ERPs) were recorded from the scalp to investigate the processing of word stimuli. Three tasks were used: (1) a task comparing words that provided an anomalous or normal sentence ending, (2) a word-list task in which different word types were examined, and (3) a word-list task in which semantic priming was examined. ERPs were recorded from a 50-channel montage in an attempt to dissociate overlapping ERP features by their scalp distributions. The focus of these studies was the N400, an ERP previously associated with language processing (Kutas & Hillyard, 1980). The temporal interval typically associated with N400 (250–500 msec) was found to contain overlapping ERP features. Two of these features were common to both sentence and word-list tasks—but one appeared different. Anomalous sentence endings and words with semantic content in lists both showed coincident negative left frontotemporal and midline-anterior ERP foci, peaking at 332 msec for sentences and 316 msec for word lists. The most negative voltage obtained in the sentence task peaked at 386 msec and had a midline-posterior focus. A right frontotemporal focus developed after the midline-posterior focus and outlasted its duration. The most negative voltage for content words in lists was reached at 364 msec. The distribution of this ERP was extensive over the midline and appeared to differ from that observed in the sentence task. Modulation of language-related ERPs by word type and semantic priming was investigated using the word-list tasks, which required category-detection responses. Two novel findings were obtained: (1) The ERP distributions for words serving grammatical function and content words differed substantially in word lists. Even when devoid of any sentence context, function words presented significantly attenuated measures of N400 compared to content words. These findings support hypotheses that suggest a differential processing of content and function words. (2) Semantic priming functionally dissociated two ERP features in the 250–500 msec range. The later and most negative midline ERP feature (peaking at 364 msec) was attenuated by semantic priming. However, the earlier left frontotemporal feature (peaking at 316 msec) was enhanced by semantic priming. The isolation of this novel language-related ERF' that is sensitive to semantic manipulations has important consequences for temporal and mechanistic aspects of theories of language processing.

Auditory event-related potentials among dyslexic and normal-reading children: 3CLT and midline comparisons

Event-related potentials (ERP's) to verbal and non-verbal auditory stimuli were recorded from normal-reading and from dyslexic children while performing a target-detection task ("oddball" paradigm). Two methods of analysis were used: (1) Peak latency and amplitude measures of P3 recorded from 3 midline electrodes; (2) P3 apex latency, amplitude and orientation in the three-channel Lissajous' trajectory (3CLT) derived from 3 orthogonal pairs of electrodes. P3 peak amplitude was significantly attenuated in dyslexic children compared to normal-reading children and in response to verbal stimuli compared to non-verbal stimuli. P3 apex latencies were longer and apex amplitudes larger in response to non-verbal compared to verbal stimuli. The most striking finding involved P3 apex orientation, which pointed in an upward-posterior direction with a slight tilt to the left among normal readers, but with a tilt to the right in dyslexics.

Context effects on saccade-related brain potentials to words during reading

In two experiments saccade-related brain potentials (SRPs) to sentences were investigated under conditions approximating natural reading. Our aim was to look for electrophysiological (SRP) signs of sentence context on the processing of final words that were either congruent or incongruent with the meaning of the sentence. In Experiment 1 subjects indicated by a button-press whether or not the final word was congruent with the context, while in Experiment 2 they read silently without an overt decision. In Experiment 1, SRPs to incongruent words were more negative than SRPs to congruent words between 80-310 msec (from saccade offset). In Experiment 2, however, the inconcruent SRPs became more negative than the congruent SRPs only between 280-460 msec. These results suggest that in Experiment 1, during the processing of incongruent words the early sign of registering mismatch appears simultaneously with the analysis of the visual features of the word.

Sentence verification and event-related brain potentials

The purpose of this study was to examine whether the decision process affected scalp recorded event-related brain potentials (ERPs). ERPs were recorded while 10 subjects evaluated the veracity of four types of Japanese sentences; true-affirmative (TA), true-negative (TN), false-affirmative (FA), and false-negative (FN). Each sentence was presented visually, one word at a time in the following order: Subject (S); object (O); and verb (V). This order of words, which is normal in Japanese grammar, allowed the ERP waveforms associated with semantic mismatch between the S and O occurring in the middle of the sentence to be separated from those elicited by the decision concerning the sentence's truth or falsity occurring at the end of the sentence. Semantic mismatch was associated with a negative component (N310) following the O-word. In addition, P3s were elicited by sentence final words. Affirmative sentences elicited larger P3s than did negative sentences, and true sentences elicited earlier P3s than did false sentences. P3s were smaller in amplitude when they followed a mismatch between the S and O words. The result suggested that by using Japanese sentences, it may be possible to examine sentence-level rather than word-level processes.