Neural Correlates of Semantic Prediction and Resolution in Sentence Processing

Speech Prosody Serves Temporal Prediction of Language via Contextual Entrainment

Temporal prediction assists language comprehension. In a series of recent behavioral studies, we have shown that listeners specifically employ rhythmic modulations of prosody to estimate the duration of upcoming sentences, thereby speeding up comprehension. In the current human magnetoencephalography (MEG) study on participants of either sex, we show that the human brain achieves this function through a mechanism termed entrainment. Through entrainment, electrophysiological brain activity maintains and continues contextual rhythms beyond their offset. Our experiment combined exposure to repetitive prosodic contours with the subsequent presentation of visual sentences that either matched or mismatched the duration of the preceding contour. During exposure to prosodic contours, we observed MEG coherence with the contours, which was source-localized to right-hemispheric auditory areas. During the processing of the visual targets, activity at the frequency of the preceding contour was still detectable in the MEG; yet sources shifted to the (left) frontal cortex, in line with a functional inheritance of the rhythmic acoustic context for prediction. Strikingly, when the target sentence was shorter than expected from the preceding contour, an omission response appeared in the evoked potential record. We conclude that prosodic entrainment is a functional mechanism of temporal prediction in language comprehension. In general, acoustic rhythms appear to endow language for employing the brain's electrophysiological mechanisms of temporal prediction.

Reward-related regions play a role in natural story comprehension

The reward system was shown to be involved in a wide array of processes. Nevertheless, the exploration of the involvement of the reward system during language processing has not yet been directly tested. We investigated the role of reward-processing regions while listening to a natural story. We utilized a published dataset in which half of the participants listened to a natural story and the others listened to a scrambled version of it to compare the functional MRI signals in the reward system between these conditions and discovered a distinct pattern between conditions. This suggests that the reward system is activated during the comprehension of natural stories. We also show evidence that the fMRI signals in reward-related areas might potentially correlate with the predictability level of processed sentences. Further research is needed to determine the nature of the involvement and the way the activity interacts with various aspects of the sentences.

The prediction potential indexes the meaning and communicative function of upcoming utterances

Prediction has a fundamental role in language processing. However, predictions can be made at different levels, and it is not always clear whether speech sounds, morphemes, words, meanings, or communicative functions are anticipated during dialogues. Previous studies reported specific brain signatures of communicative pragmatic function, in particular enhanced brain responses immediately after encountering an utterance used to request an object from a partner, but relatively smaller ones when the same utterance was used for naming the object. The present experiment now investigates whether similar neuropragmatic signatures emerge in recipients before the onset of upcoming utterances carrying different predictable communicative functions. Trials started with a context question and object pictures displayed on the screen, raising the participant's expectation that words from a specific semantic category (food or tool) would subsequently be used to either name or request one of the objects. Already 600 msec before utterance onset, a larger prediction potential was observed when a request was anticipated relative to naming expectation. As this result is congruent with the neurophysiological difference previously observed right after the critical utterance, the anticipatory brain activity may index predictions about the social-communicative function of upcoming utterances. In addition, we also found that the predictable semantic category of the upcoming word was likewise reflected in the anticipatory brain potential. Thus, the neurophysiological characteristics of the prediction potential can capture different types of upcoming linguistic information, including semantic and pragmatic aspects of an upcoming utterance and communicative action.

Neurophysiological signatures of prediction in language: A critical review of anticipatory negativities

Recent event-related potential (ERP) studies in language comprehension converge in finding anticipatory negativities preceding words or word segments that can be pre-activated based on either sentence contexts or phonological cues. We review these findings from different paradigms in the light of evidence from other cognitive domains in which slow negative potentials have long been associated with anticipatory processes and discuss their potential underlying mechanisms. We propose that this family of anticipatory negativities captures common mechanisms associated with the pre-activation of linguistic information both within words and within sentences. Future studies could utilize these anticipatory negativities in combination with other, well-established ERPs, to simultaneously track prediction-related processes emerging at different time intervals (before and after the perception of pre-activated input) and with distinct time courses (shorter-lived and longer-lived cognitive operations).

Predictive Brain Activity Shows Congruent Semantic Specificity in Language Comprehension and Production

Sentence fragments strongly predicting a specific subsequent meaningful word elicit larger preword slow waves, prediction potentials (PPs), than unpredictive contexts. To test the current predictive processing models, 128-channel EEG data were collected from both sexes to examine whether (1) different semantic PPs are elicited in language comprehension and production and (2) whether these PPs originate from the same specific "prediction area(s)" or rather from widely distributed category-specific neuronal circuits reflecting the meaning of the predicted item. Slow waves larger after predictable than unpredictable contexts were present both before subjects heard the sentence-final word in the comprehension experiment and before they pronounced the sentence-final word in the production experiment. Crucially, cortical sources underlying the semantic PP were distributed across several cortical areas and differed between the semantic categories of the expected words. In both production and comprehension, the anticipation of animal words was reflected by sources in posterior visual areas, whereas predictable tool words were preceded by sources in the frontocentral sensorimotor cortex. For both modalities, PP size increased with higher cloze probability, thus further confirming that it reflects semantic prediction, and with shorter latencies with which participants completed sentence fragments. These results sit well with theories viewing distributed semantic category-specific circuits as the mechanistic basis of semantic prediction in the two modalities.

Predicting and coding sound into action translation in spinal cord injured people

Motor activation in response to perception of action-related stimuli may depend on a resonance mechanism subserving action understanding. The extent to which this mechanism is innate or learned from sensorimotor experience is still unclear. Here, we recorded EEG while people with paraplegia or tetraplegia consequent to spinal cord injury (SCI) and healthy control participants were presented with action sounds produced by body parts (mouth, hands or feet) that were or were not affected by SCI. Non-action sounds were used as further control. We observed reduced brain activation in subjects affected by SCI at both pre- and post-stimulus latencies specifically for those actions whose effector was disconnected by the spinal lesion (i.e., hand sound for tetraplegia and leg sound for both paraplegia and tetraplegia). Correlation analyses showed that these modulations were functionally linked with the chronicity of the lesion, indicating that the longer the time the lesion- EEG data acquisition interval and/or the more the lesion occurred at a young age, the weaker was the cortical activity in response to these action sounds. Tellingly, source estimations confirmed that these modulations originated from a deficit in the motor resonance mechanism, by showing diminished activity in premotor (during prediction and perception) and near the primary motor (during perception) areas. Such dissociation along the cortical hierarchy is consistent with both previous reports in healthy subjects and with hierarchical predictive coding accounts. Overall, these data expand on the notion that sensorimotor experience maintains the cortical representations relevant to anticipate and perceive action-related stimuli.

An ERP study on the certainty of epistemic modality in predictive inference processing

Previous psychological experiments have shown that predictive inference processing under different textual constraints is modulated by the directionality function of epistemic modality (EM) certainty within the context. Nevertheless, recent neuroscientific studies have not presented positive evidence for such a function during text reading. Consequently, the current study deposited Chinese EMs "" (possibly) and "" (surely) into a predictive inference context to examine whether a directionality of EM certainty influences the processing of predictive inference via the ERP technique. Two independent variables, namely textual constraint and EM certainty, were manipulated, and 36 participants were recruited. The results revealed that, in the anticipatory stage of predictive inference processing while under a weak textual constraint, low certainty evoked a larger N400 (300-500 ms) in the fronto-central and centro-parietal regions, indicating the augmentation of cognitive loads in calculating the possibility of representations of the forthcoming information. Meanwhile, high certainty elicited a right fronto-central late positive component (LPC) (500-700 ms) associated with semantically congruent but lexically unpredicted words. In the integration stage, low certainty resulted in larger right fronto-central and centro-frontal N400 (300-500 ms) effects in the weak textual constraint condition, associated with the facilitation of lexical-semantic retrieval or pre-activation, and high certainty successively elicited right fronto-central and centro-parietal LPC (500-700 ms) effects, associated respectively with lexical unpredictability and reanalysis of the sentence meaning. The results support the directionality function of EM certainty and reveal the complete neural processing of predictive inferences with high and low certainties under different textual constraint conditions.

Swift Prosodic Modulation of Lexical Access: Brain Potentials From Three North Germanic Language Varieties

PURPOSE

According to most models of spoken word recognition, listeners probabilistically activate a set of lexical candidates, which is incrementally updated as the speech signal unfolds. Speech carries segmental (speech sound) as well as suprasegmental (prosodic) information. The role of the latter in spoken word recognition is less clear. We investigated how suprasegments (tone and voice quality) in three North Germanic language varieties affected lexical access by scrutinizing temporally fine-grained neurophysiological effects of lexical uncertainty and information gain.

METHOD

Three event-related potential (ERP) studies were reanalyzed. In all varieties investigated, suprasegments are associated with specific word endings. Swedish has two lexical "word accents" realized as pitch falls with different timings across dialects. In Danish, the distinction is in voice quality. We combined pronunciation lexica and frequency lists to calculate estimates of lexical uncertainty about an unfolding word and information gain upon hearing a suprasegmental cue and the segment upon which it manifests. We used single-trial mixed-effects regression models run every 4 ms.

RESULTS

Only lexical uncertainty showed solid results: a frontal effect at 150-400 ms after suprasegmental cue onset and a later posterior effect after 200 ms. While a model including only segmental information mostly performed better, it was outperformed by the suprasegmental model at 200-330 ms at frontal sites.

CONCLUSIONS

The study points to suprasegmental cues contributing to lexical access over and beyond segments after around 200 ms in the North Germanic varieties investigated. Furthermore, the findings indicate that a previously reported "pre-activation negativity" predominantly reflects forward-looking processing.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.25016486.

Conditionals in context: Brain signatures of prediction in discourse processing

Comprehenders are known to generate expectations about upcoming linguistic input at the sentence and discourse level. However, most previous studies on prediction focused mainly on word-induced brain activity rather than examining neural activity preceding a critical stimulus in discourse processing, where prediction actually takes place. In this EEG study, participants were presented with multiple sentences resembling a discourse including conditional sentences with either only if or if, which are characterized by different semantics, triggering stronger or weaker predictions about the possible continuation of the presented discourses, respectively. Results revealed that discourses including only if, as compared to discourses with bare if, triggered an increased predictive neural activity before the expected critical word, resembling the readiness potential. Moreover, word-induced P300 brain responses were found to be enhanced by unpredictable discourse continuations and reduced in predictable discourse continuations. Intriguingly, brain responses preceding and following the critical word were found to be correlated, which yields evidence for predictive activity modulating word-induced processing on the discourse level. These findings shed light on the predictive nature of neural processes at the discourse level, critically advancing our understanding of the functional interconnection between discourse understanding and prediction processes in brain and mind.

Leading and following: Noise differently affects semantic and acoustic processing during naturalistic speech comprehension

Despite the distortion of speech signals caused by unavoidable noise in daily life, our ability to comprehend speech in noisy environments is relatively stable. However, the neural mechanisms underlying reliable speech-in-noise comprehension remain to be elucidated. The present study investigated the neural tracking of acoustic and semantic speech information during noisy naturalistic speech comprehension. Participants listened to narrative audio recordings mixed with spectrally matched stationary noise at three signal-to-ratio (SNR) levels (no noise, 3 dB, -3 dB), and 60-channel electroencephalography (EEG) signals were recorded. A temporal response function (TRF) method was employed to derive event-related-like responses to the continuous speech stream at both the acoustic and the semantic levels. Whereas the amplitude envelope of the naturalistic speech was taken as the acoustic feature, word entropy and word surprisal were extracted via the natural language processing method as two semantic features. Theta-band frontocentral TRF responses to the acoustic feature were observed at around 400 ms following speech fluctuation onset over all three SNR levels, and the response latencies were more delayed with increasing noise. Delta-band frontal TRF responses to the semantic feature of word entropy were observed at around 200 to 600 ms leading to speech fluctuation onset over all three SNR levels. The response latencies became more leading with increasing noise and decreasing speech comprehension and intelligibility. While the following responses to speech acoustics were consistent with previous studies, our study revealed the robustness of leading responses to speech semantics, which suggests a possible predictive mechanism at the semantic level for maintaining reliable speech comprehension in noisy environments.

Prediction during language comprehension: what is next?

Prediction is often regarded as an integral aspect of incremental language comprehension, but little is known about the cognitive architectures and mechanisms that support it. We review studies showing that listeners and readers use all manner of contextual information to generate multifaceted predictions about upcoming input. The nature of these predictions may vary between individuals owing to differences in language experience, among other factors. We then turn to unresolved questions which may guide the search for the underlying mechanisms. (i) Is prediction essential to language processing or an optional strategy? (ii) Are predictions generated from within the language system or by domain-general processes? (iii) What is the relationship between prediction and memory? (iv) Does prediction in comprehension require simulation via the production system? We discuss promising directions for making progress in answering these questions and for developing a mechanistic understanding of prediction in language.

Observing memory encoding while it unfolds: Functional interpretation and current debates regarding ERP subsequent memory effects

Our ability to remember the past depends on neural processes set in train in the moment an event is experienced. These processes can be studied by segregating brain activity according to whether an event is later remembered or forgotten. The present review integrates a large number of studies examining this differential brain activity, labeled subsequent memory effect (SME), with the ERP technique, into a functional organization and discusses routes for further research. Based on the reviewed literature, we suggest that memory encoding is implemented by multiple processes, typically reflected in three functionally different subcomponents of the ERP SME elicited by study stimuli, which presumably interact with preparatory SME activity preceding the to be encoded event. We argue that ERPs are a valuable method in the SME paradigm because they have a sufficiently high temporal resolution to disclose the subcomponents of encoding-related brain activity. Implications of the proposed functional organization for future studies using the SME procedure in basic and applied settings will be discussed.

Predicting phonological information in language comprehension: evidence from ERP representational similarity analysis and Chinese idioms

Do comprehenders predict the meaning and even the phonological form of upcoming words during language comprehension? With a growing body of evidence suggesting that semantic representations may be predicted, the evidence for phonological prediction is less clear and largely derived from studies conducted in languages utilizing an alphabetic script. In this research, we aim to examine the prediction of phonological information in the processing of Chinese idioms through the use of ERP representational similarity analysis (RSA). The study utilizes four-character Chinese idioms, and phonological overlap was manipulated by varying the syllable at the idiom-final part between idiom pairs so that pairs of idioms share a syllable (i.e. within-pairs) or not (between-pairs). We quantified the similarity between patterns of neural activity of idioms for within- and between-pairs. RSA results revealed greater similarity in neural activity patterns for idioms within-pairs, compared with between-pairs, and critically this similarity effect was observed prior to the presentation of the phonological similarity, providing evidence for the pre-activation of upcoming phonological information, under circumstances that encourage predictive processing.

A Theory of Mental Frameworks

Problem-solving skills are highly valued in modern society and are often touted as core elements of school mission statements, desirable traits for job applicants, and as some of the most complex thinking that the brain is capable of executing. While learning to problem-solve is a goal of education, and many strategies, methodologies, and activities exist to help teachers guide the development of these skills, there are few formal curriculum structures or broader frameworks that guide teachers toward the achievement of this educational objective. Problem-solving skills have been called "higher order cognitive functions" in cognitive neuroscience as they involve multiple complex networks in the brain, rely on constant rehearsal, and often take years to form. Children of all ages employ problem solving, from a newborn seeking out food to children learning in school settings, or adults tackling real-world conflicts. These skills are usually considered the end product of a good education when in fact, in order to be developed they comprise an ongoing process of learning. "Ways of thinking" have been studied by philosophers and neuroscientists alike, to pinpoint cognitive preferences for problem solving approaches that develop from exposure to distinct models, derived from and resulting in certain heuristics used by learners. This new theory paper suggests a novel understanding of the brain's approach to problem solving that structures existing problem-solving frameworks into an organized design. The authors surveyed problem-solving frameworks from business administration, design, engineering, philosophy, psychology, education, neuroscience and other learning sciences to assess their differences and similarities. This review lead to an appreciation that different problem-solving frameworks from different fields respond more or less accurately and efficiently depending on the kinds of problems being tackled, leading to our conclusion that a wider range of frameworks may help individuals approach more varied problems across fields, and that such frameworks can be organized in school curriculum. This paper proposes that explicit instruction of "mental frameworks" may help organize and formalize the instruction of thinking skills that underpin problem-solving-and by extension-that the more such models a person learns, the more tools they will have for future complex problem-solving. To begin, this paper explains the theoretical underpinnings of the mental frameworks concept, then explores some existing mental frameworks which are applicable to all age groups and subject areas. The paper concludes with a list of five limitations to this proposal and pairs them with counter-balancing benefits.

Predicting coarse-grained semantic features in language comprehension: evidence from ERP representational similarity analysis and Chinese classifier

Existing studies demonstrate that comprehenders can predict semantic information during language comprehension. Most evidence comes from a highly constraining context, in which a specific word is likely to be predicted. One question that has been investigated less is whether prediction can occur when prior context is less constraining for predicting specific words. Here, we aim to address this issue by examining the prediction of animacy features in low-constraining context, using electroencephalography (EEG), in combination with representational similarity analysis (RSA). In Chinese, a classifier follows a numeral and precedes a noun, and classifiers constrain animacy features of upcoming nouns. In the task, native Chinese Mandarin speakers were presented with either animate-constraining or inanimate-constraining classifiers followed by congruent or incongruent nouns. EEG amplitude analysis revealed an N400 effect for incongruent conditions, reflecting the difficulty of semantic integration when an incompatible noun is encountered. Critically, we quantified the similarity between patterns of neural activity following the classifiers. RSA results revealed that the similarity between patterns of neural activity following animate-constraining classifiers was greater than following inanimate-constraining classifiers, before the presentation of the nouns, reflecting pre-activation of animacy features of nouns. These findings provide evidence for the prediction of coarse-grained semantic feature of upcoming words.

Predictive language comprehension in Parkinson's disease

Verb and action knowledge deficits are reported in persons with Parkinson's disease (PD), even in the absence of dementia or mild cognitive impairment. However, the impact of these deficits on combinatorial semantic processing is less well understood. Following on previous verb and action knowledge findings, we tested the hypothesis that PD impairs the ability to integrate event-based thematic fit information during online sentence processing. Specifically, we anticipated persons with PD with age-typical cognitive abilities would perform more poorly than healthy controls during a visual world paradigm task requiring participants to predict a target object constrained by the thematic fit of the agent-verb combination. Twenty-four PD and 24 healthy age-matched participants completed comprehensive neuropsychological assessments. We recorded participants' eye movements as they heard predictive sentences (The fisherman rocks the boat) alongside target, agent-related, verb-related, and unrelated images. We tested effects of group (PD/control) on gaze using growth curve models. There were no significant differences between PD and control participants, suggesting that PD participants successfully and rapidly use combinatory thematic fit information to predict upcoming language. Baseline sentences with no predictive information (e.g., Look at the drum) confirmed that groups showed equivalent sentence processing and eye movement patterns. Additionally, we conducted an exploratory analysis contrasting PD and controls' performance on low-motion-content versus high-motion-content verbs. This analysis revealed fewer predictive fixations in high-motion sentences only for healthy older adults. PD participants may adapt to their disease by relying on spared, non-action-simulation-based language processing mechanisms, although this conclusion is speculative, as the analyses of high- vs. low-motion items was highly limited by the study design. These findings provide novel evidence that individuals with PD match healthy adults in their ability to use verb meaning to predict upcoming nouns despite previous findings of verb semantic impairment in PD across a variety of tasks.

A cognitive modeling approach to learning and using reference biases in language

During real-time language processing, people rely on linguistic and non-linguistic biases to anticipate upcoming linguistic input. One of these linguistic biases is known as the implicit causality bias, wherein language users anticipate that certain entities will be rementioned in the discourse based on the entity's particular role in an expressed causal event. For example, when language users encounter a sentence like “Elizabeth congratulated Tina…” during real-time language processing, they seemingly anticipate that the discourse will continue about Tina, the object referent, rather than Elizabeth, the subject referent. However, it is often unclear how these reference biases are acquired and how exactly they get used during real-time language processing. In order to investigate these questions, we developed a reference learning model within the PRIMs cognitive architecture that simulated the process of predicting upcoming discourse referents and their linguistic forms. Crucially, across the linguistic input the model was presented with, there were asymmetries with respect to how the discourse continued. By utilizing the learning mechanisms of the PRIMs architecture, the model was able to optimize its predictions, ultimately leading to biased model behavior. More specifically, following subject-biased implicit causality verbs the model was more likely to predict that the discourse would continue about the subject referent, whereas following object-biased implicit causality verbs the model was more likely to predict that the discourse would continue about the object referent. In a similar fashion, the model was more likely to predict that subject referent continuations would be in the form of a pronoun, whereas object referent continuations would be in the form of a proper name. These learned biases were also shown to generalize to novel contexts in which either the verb or the subject and object referents were new. The results of the present study demonstrate that seemingly complex linguistic behavior can be explained by cognitively plausible domain-general learning mechanisms. This study has implications for psycholinguistic accounts of predictive language processing and language learning, as well as for theories of implicit causality and reference processing.

Benefits and costs of predictive processing: How sentential constraint and word expectedness affect memory formation

This study investigated how the strength of schema support provided by strongly (SC) and weakly constraining (WC) sentences affects the encoding of expected and unexpected words, and how this is reflected in event-related potentials (ERPs). In a surprise recognition memory test, words studied on the previous day were presented together with new words and lures that were expected but not presented in the study phase. ERPs recorded in the study phase were compared for subsequently remembered and forgotten words. Better memory performance for expected over unexpected words was electrophysiologically supported by a parietal subsequent memory effect (SME) reflecting enhanced item-specific encoding of contextually expected words. SC sentences not only facilitated the semantic integration of sentence-ending words, as reflected in reduced N400 amplitudes, but also enabled the rapid successful encoding of these words into memory, which is evidenced by an SC > WC pattern in memory performance and correlations between pre- and post-stimulus SMEs for SC sentences. In contrast, words processed in WC sentence contexts necessitated sustained elaborative encoding processes as reflected in a late frontal slow wave SME. Expected but not presented words were associated with high rates of false positive memory decisions, indicating that these words remained in a state of high accessibility in memory even one day after the study phase. These mnemonic costs of predictive processing were more pronounced for expected words from SC sentences than from WC sentences and could reflect the lingering of strong semantic predictions which were associated with the pre-updating of sentence representations.

Understanding Design Features of Music and Language: The Choric/Dialogic Distinction

Music and spoken language share certain characteristics: both consist of sequences of acoustic elements that are combinatorically combined, and these elements partition the same continuous acoustic dimensions (frequency, formant space and duration). However, the resulting categories differ sharply: scale tones and note durations of small integer ratios appear in music, while speech uses phonemes, lexical tone, and non-isochronous durations. Why did music and language diverge into the two systems we have today, differing in these specific features? We propose a framework based on information theory and a reverse-engineering perspective, suggesting that design features of music and language are a response to their differential deployment along three different continuous dimensions. These include the familiar propositional-aesthetic ('goal') and repetitive-novel ('novelty') dimensions, and a dialogic-choric ('interactivity') dimension that is our focus here. Specifically, we hypothesize that music exhibits specializations enhancing coherent production by several individuals concurrently-the 'choric' context. In contrast, language is specialized for exchange in tightly coordinated turn-taking-'dialogic' contexts. We examine the evidence for our framework, both from humans and non-human animals, and conclude that many proposed design features of music and language follow naturally from their use in distinct dialogic and choric communicative contexts. Furthermore, the hybrid nature of intermediate systems like poetry, chant, or solo lament follows from their deployment in the less typical interactive context.

OUP accepted manuscript

During conversations, speech prosody provides important clues about the speaker’s communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. Here, the neurophysiological basis of intonation and speech act understanding were investigated with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level. Already approximately 100 ms after the sentence-final word differing in prosody, questions, and statements expressed with the same sentences led to different neurophysiological activity recorded in the event-related potential. Interestingly, low-pass filtered sentences and acoustically matched nonvocal musical signals failed to show any neurophysiological dissociations, thus suggesting that the physical intonation alone cannot explain this modulation. Our results show rapid neurophysiological indexes of prosodic communicative information processing that emerge only when pragmatic and lexico-semantic information are fully expressed. The early enhancement of question-related activity compared with statements was due to sources in the articulatory-motor region, which may reflect the richer action knowledge immanent to questions, namely the expectation of the partner action of answering the question. The present findings demonstrate a neurophysiological correlate of prosodic communicative information processing, which enables humans to rapidly detect and understand speaker intentions in linguistic interactions.

Implicit auditory perception of local and global irregularities in passive listening condition

The auditory system detects differences in sounds at an implicit level, but data on this difference might not be sufficient to make explicit discrimination. The biomarkers of implicit auditory memory of ambiguous stimuli could shed light on unconscious auditory processing and implicit auditory learning. Mismatch negativity (MMN) and P3a, components of event-related potentials (ERPs) reflecting stimuli discrimination without direct attention, were previously detected in response to the local (short-term) irregularity in the auditory sequence even in an unconscious state. At the same time, P3b was elicited only in case of direct attention in response to the global (long-term) irregularity. In this study, we applied the local-global auditory paradigm to obtain possible electrophysiological signatures of implicit detection of hardly distinguishable auditory stimuli. ERPs were recorded from 20 healthy volunteers during active discrimination of deviant sounds in the old-ball sequence and passive listening of the same sounds in the sequence with local-global irregularity. The discrimination task consisted of two blocks with different deviant sounds targeted to respond. The sound discrimination accuracy was at an average of 40%, implying the difficulty of explicit sound recognition. Comparing ERPs to standard and deviant sounds, we found posterior negativity in ERP around 450-600 ms in response to targeted deviant sounds. MMN was significant only in response to non-target deviants. In the passive local-global paradigm, we observed an anterior positivity (284-412 ms), compatible with P3a, in response to a violation of local regularity. Violation of global regularity elicited an anterior negative response (228-586 ms), resembling the N400 component of ERPs. Importantly, the other indexes of auditory discrimination, such as MMN and P3b, were insignificant in ERPs to both regularity violations. The observed P3a and N400 components of ERPs may reflect prediction error signals in the implicit perception of sound patterns even if behavioral recognition was poor.

Prediction during simultaneous interpreting: Evidence from the visual-world paradigm

We report the results of an eye-tracking study which used the Visual World Paradigm (VWP) to investigate the time-course of prediction during a simultaneous interpreting task. Twenty-four L1 French professional conference interpreters and twenty-four L1 French professional translators untrained in simultaneous interpretation listened to sentences in English and interpreted them simultaneously into French while looking at a visual scene. Sentences contained a highly predictable word (e.g., The dentist asked the man to open his mouth a little wider). The visual scene comprised four objects, one of which depicted either the target object (mouth; bouche), an English phonological competitor (mouse; souris), a French phonological competitor (cork; bouchon), or an unrelated word (bone; os). We considered 1) whether interpreters and translators predict upcoming nouns during a simultaneous interpreting task, 2) whether interpreters and translators predict the form of these nouns in English and in French and 3) whether interpreters and translators manifest different predictive behaviour. Our results suggest that both interpreters and translators predict upcoming nouns, but neither group predicts the word-form of these nouns. In addition, we did not find significant differences between patterns of prediction in interpreters and translators. Thus, evidence from the visual-world paradigm shows that prediction takes place in simultaneous interpreting, regardless of training and experience. However, we were unable to establish whether word-form was predicted.

The Neural Basis of Semantic Prediction in Sentence Comprehension

Although prediction plays an important role in language comprehension, its precise neural basis remains unclear. This fMRI study investigated whether and how semantic-category-specific and common cerebral areas are recruited in predictive semantic processing during sentence comprehension. We manipulated the semantic constraint of sentence contexts, upon which a tool-related, a building-related, or no specific category of noun is highly predictable. This noun-predictability effect was measured not only over the target nouns but also over their preceding transitive verbs. Both before and after the appearance of target nouns, left anterior supramarginal gyrus was specifically activated for tool-related nouns and left parahippocampal place area was activated specifically for building-related nouns. The semantic-category common areas included a subset of left inferior frontal gyrus during the anticipation of incoming target nouns (activity enhancement for high predictability) and included a wide spread of areas (bilateral inferior frontal gyrus, left superior/middle temporal gyrus, left medial pFC, and left TPJ) during the integration of actually perceived nouns (activity reduction for high predictability). These results indicated that the human brain recruits fine divisions of cortical areas to distinguish different semantic categories of predicted words, and anticipatory semantic processing relies, at least partially, on top-down prediction conducted in higher-level cortical areas.

Semantic priming and neurobiology in schizophrenia: A theoretical review

In this theoretical review we bridge the cognitive and neurobiological sciences to shed light on the neurocognitive foundations of the semantic priming effect in schizophrenia. We review and theoretically evaluate the neurotransmitter systems (dopaminergic, GABAergic and glutamatergic) and neurobiological underpinnings of behavioural and electrophysiological (N400) semantic priming in the pathology, and the main hypotheses on their geneses: a disinhibition of the semantic spread of activation, a disorganised semantic storage or noisy lexical-semantic associations, a psychomotor artefact, an artefact of relatedness proportions, or an inability to mobilise contextual information. We further assess the literature on the endophenotype of Formal Thought Disorder from multiple standpoints, ranging from neurophysiology to cognition: considerations are weaved on neuronal (PV basket cell, SST, VIP) and receptor deficits (DRD1, NMDA), neurotransmitter imbalances (dopamine), cortical and dopaminergic lateralisation, inter alia. In conclusion, we put forth novel postulates on the underlying causes of controlled hypopriming, automatic hyperpriming, N400 reversals (larger amplitudes for close associations), indirect versus direct hyperpriming, and the endophenotype of lexical-semantic disturbances in schizophrenia.

Language Tasks and the Network Control Role of the Left Inferior Frontal Gyrus

Recent work has combined cognitive neuroscience and control theory to make predictions about cognitive control functions. Here, we test a link between whole-brain theories of semantics and the role of the left inferior frontal gyrus (LIFG) in controlled language performance using network control theory (NCT), a branch of systems engineering. Specifically, we examined whether two properties of node controllability, boundary and modal controllability, were linked to semantic selection and retrieval on sentence completion and verb generation tasks. We tested whether the controllability of the left IFG moderated language selection and retrieval costs and the effects of continuous θ burst stimulation (cTBS), an inhibitory form of transcranial magnetic stimulation (TMS) on behavior in 41 human subjects (25 active, 16 sham). We predicted that boundary controllability, a measure of the theoretical ability of a node to integrate and segregate brain networks, would be linked to word selection in the contextually-rich sentence completion task. In contrast, we expected that modal controllability, a measure of the theoretical ability of a node to drive the brain into specifically hard-to-reach states, would be linked to retrieval on the low-context verb generation task. Boundary controllability was linked to selection and to the ability of TMS to reduce response latencies on the sentence completion task. In contrast, modal controllability was not linked to performance on the tasks or TMS effects. Overall, our results suggest a link between the network integrating role of the LIFG and selection and the overall semantic demands of sentence completion.

Neurophysiological basis of the N400 deflection, from Mismatch Negativity to Semantic Prediction Potentials and late positive components

The first theoretical model on the neurophysiological basis of the N400: the deflection reflects layer I dendritic plateaus on a preparatory state of synaptic integration that precedes layer V somatic burst firing for conscious identification of the higher-order features of the stimulus (a late positive shift). Plateaus ensue from apical disinhibition by vasoactive intestinal polypeptide-positive interneurons (VIPs) through suppression of Martinotti cells, opening the gates for glutamatergic feedback to trigger dendritic regenerative potentials. Cholinergic transients contribute to these dynamics directly, holding a central role in the N400 deflection. The stereotypical timing of the (frontal) glutamatergic feedback and the accompanying cholinergic transients account for the enigmatic "invariability" of the peak latency in the face of a gamut of different stimuli and paradigms. The theoretical postulations presented here may bring about unprecedented level of detail for the N400 deflection to be used in the study of schizophrenia, Alzheimer's disease and other higher-order pathologies. The substrates of a late positive component, the Mismatch Negativity and the Semantic Prediction Potentials are also surveyed.

Brain signatures predict communicative function of speech production in interaction

People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.

Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity

With strong and valid predictions, grasping a message is easy, whereas more demanding processing is required in the absence of robust expectations. We here demonstrate that brain correlates of the interplay between prediction and perception mechanisms in the understanding of meaningful sentences. Sentence fragments that strongly predict subsequent words induced anticipatory brain activity preceding the expected words; this potential was absent if context did not strongly predict subsequent words. Subjective reports of certainty about upcoming words and objective corpus-based measures correlated with the size of the anticipatory signal, thus establishing its status as a semantic prediction potential (SPP). Crucially, there was an inverse correlation between the SPP and the N400 brain response. The main cortical generators of SPP and N400 were found in inferior prefrontal cortex and posterior temporal cortex, respectively. Interestingly, sentence meaning was reflected by both measures, with additional category-specific sources of SPPs and N400s falling into parieto-temporo-occipital (visual) and frontocentral (sensorimotor) areas for animal- and tool-related words, respectively. These results show that the well-known brain index of semantic comprehension, N400, has an antecedent with different brain localization but similar semantic discriminatory function. We discuss whether N400 dynamics may causally depend on mechanisms underlying SPP size and sources.

Direct Evidence for Prediction Signals in Frontal Cortex Independent of Prediction Error

Predictive coding (PC) has been suggested as one of the main mechanisms used by brains to interact with complex environments. PC theories posit top-down prediction signals, which are compared with actual outcomes, yielding in turn prediction error (PE) signals, which are used, bottom-up, to modify the ensuing predictions. However, disentangling prediction from PE signals has been challenging. Critically, while many studies found indirect evidence for PC in the form of PE signals, direct evidence for the prediction signal is mostly lacking. Here, we provide clear evidence, obtained from intracranial cortical recordings in human surgical patients, that the human lateral prefrontal cortex evinces prediction signals while anticipating an event. Patients listened to task-irrelevant sequences of repetitive tones including infrequent predictable or unpredictable pitch deviants. The broadband high-frequency amplitude (HFA) was decreased prior to the onset of expected relative to unexpected deviants in the frontal cortex only, and its amplitude was sensitive to the increasing likelihood of deviants following longer trains of standards in the unpredictable condition. Single-trial HFA predicted deviations and correlated with poststimulus response to deviations. These results provide direct evidence for frontal cortex prediction signals independent of PE signals.

Prediction and Mismatch Negativity Responses Reflect Impairments in Action Semantic Processing in Adults With Autism Spectrum Disorders

The neurophysiological mechanisms underlying motor and language difficulties in autism spectrum disorders (ASD) are still largely unclear. The present work investigates biological indicators of sound processing, (action-) semantic understanding and predictive coding and their correlation with clinical symptoms of ASD. Twenty-two adults with high-functioning ASD and 25 typically developed (TD) participants engaged in an auditory, passive listening, Mismatch Negativity (MMN) task while high-density electroencephalography (EEG) was recorded. Action and non-action words were presented in the context of sounds, which were either semantically congruent with regard to the body part they relate to or semantically incongruent or unrelated. The anticipatory activity before sound onset, the Prediction Potential (PP), was significantly reduced in the ASD group specifically for action, but not for non-action sounds. The early-MMN-like responses to words (latency: 120 ms) were differentially modulated across groups: controls showed larger amplitudes for words in action-sound compared to non-action contexts, whereas ASD participants demonstrated enlarged early-MMN-like responses only in a pure tone context, with no other modulation dependent on action sound context. Late-MMN-like responses around 560 ms post-stimulus onset revealed body-part-congruent action-semantic priming for words in control participants, but not in the ASD group. Importantly, neurophysiological indices of semantic priming in ASD participants correlated with the extent of autistic traits as revealed by the Autism Spectrum Quotient (AQ). The data suggest that high-functioning adults with ASD show a specific deficit in semantic processing and predictive coding of sounds and words related to action, which is absent for neutral, non-action, sounds.

Neurophysiological evidence for rapid processing of verbal and gestural information in understanding communicative actions

During everyday social interaction, gestures are a fundamental part of human communication. The communicative pragmatic role of hand gestures and their interaction with spoken language has been documented at the earliest stage of language development, in which two types of indexical gestures are most prominent: the pointing gesture for directing attention to objects and the give-me gesture for making requests. Here we study, in adult human participants, the neurophysiological signatures of gestural-linguistic acts of communicating the pragmatic intentions of naming and requesting by simultaneously presenting written words and gestures. Already at ~150 ms, brain responses diverged between naming and request actions expressed by word-gesture combination, whereas the same gestures presented in isolation elicited their earliest neurophysiological dissociations significantly later (at ~210 ms). There was an early enhancement of request-evoked brain activity as compared with naming, which was due to sources in the frontocentral cortex, consistent with access to action knowledge in request understanding. In addition, an enhanced N400-like response indicated late semantic integration of gesture-language interaction. The present study demonstrates that word-gesture combinations used to express communicative pragmatic intentions speed up the brain correlates of comprehension processes – compared with gesture-only understanding – thereby calling into question current serial linguistic models viewing pragmatic function decoding at the end of a language comprehension cascade. Instead, information about the social-interactive role of communicative acts is processed instantaneously.

Processing of Action and Sound Verbs in Context: An FMRI Study

Recent theories propose a flexible recruitment of sensory and motor brain regions during conceptual processing depending on context and task. The present functional magnetic resonance imaging study investigated the influence of context and task on conceptual processing of action and sound verbs. Participants first performed an explicit semantic context decision task, in which action and sound verbs were presented together with a context noun. The same verbs were repeatedly presented in a subsequent implicit lexical decision task together with new action and sound verbs. Thereafter, motor and acoustic localizer tasks were administered to identify brain regions involved in perception and action. During the explicit task, we found differential activations to action and sound verbs near corresponding sensorimotor brain regions. During the implicit lexical decision task, differences between action and sound verbs were absent. However, feature-specific repetition effects were observed near corresponding sensorimotor brain regions. The present results suggest flexible conceptual representations depending on context and task. Feature-specific effects were observed only near, but not within corresponding sensorimotor brain regions, as defined by the localizer tasks. Our results therefore only provide limited evidence in favor of grounded cognition theories assuming a close link between the conceptual and the sensorimotor systems.

Similar time courses for word form and meaning preactivation during sentence comprehension

Current psycholinguistic research generally acknowledges that aspects of sentence comprehension benefit from neural preactivation of different types of information. However, despite strong support from a number of studies, routine specific word form preactivation has been challenged by Ito, Corley, Pickering, Martin, and Nieuwland (2016). They suggest that word form prediction is contingent upon having enough processing time and resources (afforded by slower input rates) to progress through unidirectional, productionlike stages of comprehension to arrive at word forms via semantic feature preactivation. This conclusion is based on findings from their ERP study, which used a related anomaly paradigm and reported form preactivation at a slow (700 ms) word presentation rate but not a faster one (500 ms). The present experimental design is a conceptual replication of Ito et al. (2016), testing young adults by measuring ERP amplitudes to unpredictable words related either semantically/associatively or orthographically to predictable sentence continuations, relative to unrelated continuations. Results showed that, at a visual presentation rate of two words per second, both types of related words show similarly reduced N400s, as well as varying degrees of increased posterior post-N400 positivity. These findings indicate that word form preactivation during sentence comprehension is detectable along a similar time course as semantic feature preactivation, and such processing does not necessarily require additional time beyond that afforded by near-normal reading rates.

Specific lexico-semantic predictions are associated with unique spatial and temporal patterns of neural activity

We used Magnetoencephalography (MEG) in combination with Representational Similarity Analysis to probe neural activity associated with distinct, item-specific lexico-semantic predictions during language comprehension. MEG activity was measured as participants read highly constraining sentences in which the final words could be predicted. Before the onset of the predicted words, both the spatial and temporal patterns of brain activity were more similar when the same words were predicted than when different words were predicted. The temporal patterns localized to the left inferior and medial temporal lobe. These findings provide evidence that unique spatial and temporal patterns of neural activity are associated with item-specific lexico-semantic predictions. We suggest that the unique spatial patterns reflected the prediction of spatially distributed semantic features associated with the predicted word, and that the left inferior/medial temporal lobe played a role in temporally 'binding' these features, giving rise to unique lexico-semantic predictions.

Oscillatory dynamics of cortical functional connections in semantic prediction

An event related potential, known as the N400, has been particularly useful in investigating language processing as it serves as a neural index for semantic prediction. There are numerous studies on the functional segregation of N400 neural sources; however, the oscillatory dynamics of functional connections among the relevant sources has remained elusive. In this study we acquired magnetoencephalography data during a classic N400 paradigm, where the semantic predictability of a fixed target noun was manipulated in simple German sentences. We conducted inter-regional functional connectivity (FC) and time-frequency analysis on known regions of the semantic network, encompassing bilateral temporal, and prefrontal cortices. Increased FC was found in less predicted (LP) nouns compared with highly predicted (HP) nouns in three connections: (a) right inferior frontal gyrus (IFG) and right middle temporal gyrus (MTG) from 0 to 300 ms mainly within the alpha band, (b) left lateral orbitofrontal (LOF) and right IFG around 400 ms within the beta band, and (c) left superior temporal gyrus (STG) and left LOF from 300 to 700 ms in the beta and low gamma bands. Furthermore, gamma spectral power (31-70 Hz) was stronger in HP nouns than in LP nouns in left anterior temporal cortices in earlier time windows (0-200 ms). Our findings support recent theories in language comprehension, suggesting fronto-temporal top-down connections are mainly mediated through beta oscillations while gamma band frequencies are involved in matching between prediction and input.

Gamma Oscillatory Activity Related to Language Prediction

Using magnetoencephalography, the current study examined gamma activity associated with language prediction. Participants read high- and low-constraining sentences in which the final word of the sentence was either expected or unexpected. Although no consistent gamma power difference induced by the sentence-final words was found between the expected and unexpected conditions, the correlation of gamma power during the prediction and activation intervals of the sentence-final words was larger when the presented words matched with the prediction compared with when the prediction was violated or when no prediction was available. This suggests that gamma magnitude relates to the match between predicted and perceived words. Moreover, the expected words induced activity with a slower gamma frequency compared with that induced by unexpected words. Overall, the current study establishes that prediction is related to gamma power correlations and a slowing of the gamma frequency.

Flexibility in Language Action Interaction: The Influence of Movement Type

Recent neuropsychological studies in neurological patients and healthy subjects suggest a close functional relationship between the brain systems for language and action. Facilitation and inhibition effects of motor system activity on language processing have been demonstrated as well as causal effects in the reverse direction, from language processes on motor excitability or performance. However, as the documented effects between motor and language systems were sometimes facilitatory and sometimes inhibitory, the “sign” of these effects still remains to be explained. In a previous study, we reported a word-category-specific differential impairment of verbal working memory for concordant arm- and leg-related action words brought about by complex sequential movements of the hands and feet. In this article, we seek to determine whether the sign of the functional interaction between language and action systems of the human brain can be changed in a predictable manner by changing movement type. We here report that the sign of the effect of motor movement on action word memory can be reversed from interference to facilitation if, instead of complex movement sequences, simple repetitive movements are performed. Specifically, when engaged in finger tapping, subjects were able to remember relatively more arm-related action words (as compared to control conditions), thus documenting an enhancement of working memory brought about by simple hand movements. In contrast, when performing complex sequences of finger movements, an effector-specific degradation of action word memory was found. By manipulating the sign of the effect in accord with theory-driven predictions, these findings provide support for shared neural bases for motor movement and verbal working memory for action-related words and strengthen the argument that motor systems play a causal and functionally relevant role in language processing semantically related to action.

Neural reuse of action perception circuits for language, concepts and communication

Neurocognitive and neurolinguistics theories make explicit statements relating specialized cognitive and linguistic processes to specific brain loci. These linking hypotheses are in need of neurobiological justification and explanation. Recent mathematical models of human language mechanisms constrained by fundamental neuroscience principles and established knowledge about comparative neuroanatomy offer explanations for where, when and how language is processed in the human brain. In these models, network structure and connectivity along with action- and perception-induced correlation of neuronal activity co-determine neurocognitive mechanisms. Language learning leads to the formation of action perception circuits (APCs) with specific distributions across cortical areas. Cognitive and linguistic processes such as speech production, comprehension, verbal working memory and prediction are modelled by activity dynamics in these APCs, and combinatorial and communicative-interactive knowledge is organized in the dynamics within, and connections between APCs. The network models and, in particular, the concept of distributionally-specific circuits, can account for some previously not well understood facts about the cortical 'hubs' for semantic processing and the motor system's role in language understanding and speech sound recognition. A review of experimental data evaluates predictions of the APC model and alternative theories, also providing detailed discussion of some seemingly contradictory findings. Throughout, recent disputes about the role of mirror neurons and grounded cognition in language and communication are assessed critically.