Combined eye tracking and fMRI reveals neural basis of linguistic predictions during sentence comprehension

The hippocampus supports the representation of abstract concepts: Implications for the study of recognition memory

Words, unlike images, are symbolic representations. The associative details inherent within a word's meaning and the visual imagery it generates, are inextricably connected to the way words are processed and represented. It is well recognised that the hippocampus associatively binds components of a memory to form a lasting representation, and here we show that the hippocampus is especially sensitive to abstract word processing. Using fMRI during recognition, we found that the increased abstractness of words produced increased hippocampal activation regardless of memory outcome. Interestingly, word recollection produced hippocampal activation regardless of word content, while the parahippocampal cortex was sensitive to concreteness of word representations, regardless of memory outcome. We reason that the hippocampus has assumed a critical role in the representation of uncontextualized abstract word meaning, as its information-binding ability allows the retrieval of the semantic and visual associates that, when bound together, generate the abstract concept represented by word symbols. These insights have implications for research on word representation, memory, and hippocampal function, perhaps shedding light on how the human brain has adapted to encode and represent abstract concepts.

Sentence superiority in the reading brain

When a sequence of written words is briefly presented and participants are asked to identify just one word at a post-cued location, then word identification accuracy is higher when the word is presented in a grammatically correct sequence compared with an ungrammatical sequence. This sentence superiority effect has been reported in several behavioral studies and two EEG investigations. Taken together, the results of these studies support the hypothesis that the sentence superiority effect is primarily driven by rapid access to a sentence-level representation via partial word identification processes that operate in parallel over several words. Here we used MEG to examine the neural structures involved in this early stage of written sentence processing, and to further specify the timing of the different processes involved. Source activities over time showed grammatical vs. ungrammatical differences first in the left inferior frontal gyrus (IFG: 321-406 ms), then the left anterior temporal lobe (ATL: 466-531 ms), and finally in both left IFG (549-602 ms) and left posterior superior temporal gyrus (pSTG: 553-622 ms). We interpret the early IFG activity as reflecting the rapid bottom-up activation of sentence-level representations, including syntax, enabled by partly parallel word processing. Subsequent activity in ATL and pSTG is thought to reflect the constraints imposed by such sentence-level representations on on-going word-based semantic activation (ATL), and the subsequent development of a more detailed sentence-level representation (pSTG). These results provide further support for a cascaded interactive-activation account of sentence reading.

Combining EEG and 3D-eye-tracking to study the prediction of upcoming speech in naturalistic virtual environments: A proof of principle

EEG and eye-tracking provide complementary information when investigating language comprehension. Evidence that speech processing may be facilitated by speech prediction comes from the observation that a listener's eye gaze moves towards a referent before it is mentioned if the remainder of the spoken sentence is predictable. However, changes to the trajectory of anticipatory fixations could result from a change in prediction or an attention shift. Conversely, N400 amplitudes and concurrent spectral power provide information about the ease of word processing the moment the word is perceived. In a proof-of-principle investigation, we combined EEG and eye-tracking to study linguistic prediction in naturalistic, virtual environments. We observed increased processing, reflected in theta band power, either during verb processing - when the verb was predictive of the noun - or during noun processing - when the verb was not predictive of the noun. Alpha power was higher in response to the predictive verb and unpredictable nouns. We replicated typical effects of noun congruence but not predictability on the N400 in response to the noun. Thus, the rich visual context that accompanied speech in virtual reality influenced language processing compared to previous reports, where the visual context may have facilitated processing of unpredictable nouns. Finally, anticipatory fixations were predictive of spectral power during noun processing and the length of time fixating the target could be predicted by spectral power at verb onset, conditional on the object having been fixated. Overall, we show that combining EEG and eye-tracking provides a promising new method to answer novel research questions about the prediction of upcoming linguistic input, for example, regarding the role of extralinguistic cues in prediction during language comprehension.

The modulating effect of musical expertise on lexical-semantic prediction in speech-in-noise comprehension: Evidence from an EEG study

Musical expertise has been proposed to facilitate speech perception and comprehension in noisy environments. This study further examined the open question of whether musical expertise modulates high-level lexical-semantic prediction to aid online speech comprehension in noisy backgrounds. Musicians and nonmusicians listened to semantically strongly/weakly constraining sentences during EEG recording. At verbs prior to target nouns, both groups showed a positivity-ERP effect (Strong vs. Weak) associated with the predictability of incoming nouns; this correlation effect was stronger in musicians than in nonmusicians. After the target nouns appeared, both groups showed an N400 reduction effect (Strong vs. Weak) associated with noun predictability, but musicians exhibited an earlier onset latency and stronger effect size of this correlation effect than nonmusicians. To determine whether musical expertise enhances anticipatory semantic processing in general, the same group of participants participated in a control reading comprehension experiment. The results showed that, compared with nonmusicians, musicians demonstrated more delayed ERP correlation effects of noun predictability at words preceding the target nouns; musicians also exhibited more delayed and reduced N400 decrease effects correlated with noun predictability at the target nouns. Taken together, these results suggest that musical expertise enhances lexical-semantic predictive processing in speech-in-noise comprehension. This musical-expertise effect may be related to the strengthened hierarchical speech processing in particular.

Disruption to left inferior frontal cortex modulates semantic prediction effects in reading and subsequent memory: Evidence from simultaneous TMS-EEG

Readers use prior context to predict features of upcoming words. When predictions are accurate, this increases the efficiency of comprehension. However, little is known about the fate of predictable and unpredictable words in memory or the neural systems governing these processes. Several theories suggest that the speech production system, including the left inferior frontal cortex (LIFC), is recruited for prediction but evidence that LIFC plays a causal role is lacking. We first examined the effects of predictability on memory and then tested the role of posterior LIFC using transcranial magnetic stimulation (TMS). In Experiment 1, participants read category cues, followed by a predictable, unpredictable, or incongruent target word for later recall. We observed a predictability benefit to memory, with predictable words remembered better than unpredictable words. In Experiment 2, participants performed the same task with electroencephalography (EEG) while undergoing event-related TMS over posterior LIFC using a protocol known to disrupt speech production, or over the right hemisphere homologue as an active control site. Under control stimulation, predictable words were better recalled than unpredictable words, replicating Experiment 1. This predictability benefit to memory was eliminated under LIFC stimulation. Moreover, while an a priori ROI-based analysis did not yield evidence for a reduction in the N400 predictability effect, mass-univariate analyses did suggest that the N400 predictability effect was reduced in spatial and temporal extent under LIFC stimulation. Collectively, these results provide causal evidence that the LIFC is recruited for prediction during silent reading, consistent with prediction-through-production accounts.

Shared grey matter correlates of reading and attention

Disorders of reading (developmental dyslexia) and attention (ADHD) have a high rate of comorbidity (25-40%), yet little is known about the neural underpinnings of this phenomenon. The current study investigated the shared and unique neural correlates of reading and attention in 330 typically developing children ages 8-18 from the Philadelphia Neurodevelopmental Cohort. Multiple regression analyses were used to identify regions of the brain where grey matter (GM) volume was associated with reading or attention scores (p < 0.001, cluster FDR p < 0.05). Better attention scores correlated with increased GM in the precuneus and higher reading scores were associated with greater thalamic GM. An exploratory conjunction analysis (p < 0.05, k > 239) found that GM in the caudate and precuneus correlated with both reading and attention scores. These results are consistent with a recent meta-analysis which identified GM reductions in the caudate in both dyslexia and ADHD and reveal potential shared neural correlates of reading and attention.

The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.

Language reorganization in patients with left-hemispheric gliomas is associated with increased cortical volume in language-related areas and in the default mode network

BACKGROUND

Language function may reorganize to overcome focal impairment; however, the relation between functional and structural changes in patients with brain tumors remains unclear. We investigated the cortical volume of atypical language dominant (AD) patients with left frontal-insular high-grade (HGG) and low-grade glioma (LGG). We hypothesized atypical language being associated with areas of increased cortical volume in the right hemisphere, including language areas homologues.

METHODS

Patient were recruited following the criteria: left frontal-insular glioma; functional MRI and 3DT1-weighted images; no artifacts. We calculated an hemispheric language laterality index (LI), defined as: AD if LI < .2; left-dominant (LD) if LI ≥ .2. We measured cortical volume in three voxel-based morphometry (VBM) analyses: total AD vs. LD patients; AD vs. LD in HGG; AD vs. LD in LGG. We repeated the analysis in AD vs. LD healthy controls (HC). A minimum threshold of t > 2 and corrected p < .025 (Bonferroni) was employed.

RESULTS

We recruited 119 patients (44LGG, 75HGG). Hemispheric LI demonstrated 64/119AD and 55/119LD patients. The first VBM analysis demonstrated significantly increased cortical volume in AD patients in the right inferior frontal gyrus (IFG), right superior temporal gyrus (STG), right insula, right fusiform gyrus (FG), right precentral gyrus, right temporal-parietal junction, right posterior cingulate cortex (PCC), right hippocampus, right- and left cerebellum. AD patients with HGG showed the same areas of significantly increased cortical volume. AD patients with LGG displayed significantly increased cortical volume in right IFG, right STG, right insula, right FG, right anterior cingulate cortex, right PCC, right dorsal-lateral prefrontal cortex. HC showed no significant results.

CONCLUSION

Right-sided (atypical) language activations in patients with left-hemispheric gliomas are associated with areas of increased cortical volume. Additionally, default mode network nodes showed greater cortical volume in AD patients regardless of the tumor grade, supporting the idea of these cortices participating in the development of language plasticity.

Neural underpinning of Japanese particle processing in non-native speakers

Grammar acquisition by non-native learners (L2) is typically less successful and may produce fundamentally different grammatical systems than that by native speakers (L1). The neural representation of grammatical processing between L1 and L2 speakers remains controversial. We hypothesized that working memory is the primary source of L1/L2 differences, by considering working memory within the predictive coding account, which models grammatical processes as higher-level neuronal representations of cortical hierarchies, generating predictions (forward model) of lower-level representations. A functional MRI study was conducted with L1 Japanese speakers and highly proficient Japanese learners requiring oral production of grammatically correct Japanese particles. We assumed selecting proper particles requires forward model-dependent processes of working memory as their functions are highly context-dependent. As a control, participants read out a visually designated mora indicated by underlining. Particle selection by L1/L2 groups commonly activated the bilateral inferior frontal gyrus/insula, pre-supplementary motor area, left caudate, middle temporal gyrus, and right cerebellum, which constituted the core linguistic production system. In contrast, the left inferior frontal sulcus, known as the neural substrate of verbal working memory, showed more prominent activation in L2 than in L1. Thus, the working memory process causes L1/L2 differences even in highly proficient L2 learners.

Online neurostimulation of Broca's area does not interfere with syntactic predictions: A combined TMS-EEG approach to basic linguistic combination

Categorical predictions have been proposed as the key mechanism supporting the fast pace of syntactic composition in language. Accordingly, grammar-based expectations are formed-e.g., the determiner "a" triggers the prediction for a noun-and facilitate the analysis of incoming syntactic information, which is then checked against a single or few other word categories. Previous functional neuroimaging studies point towards Broca's area in the left inferior frontal gyrus (IFG) as one fundamental cortical region involved in categorical prediction during incremental language processing. Causal evidence for this hypothesis is however still missing. In this study, we combined Electroencephalography (EEG) and Transcranial Magnetic Stimulation (TMS) to test whether Broca's area is functionally relevant in predictive mechanisms for language. We transiently perturbed Broca's area during the first word in a two-word construction, while simultaneously measuring the Event-Related Potential (ERP) correlates of syntactic composition. We reasoned that if Broca's area is involved in predictive mechanisms for syntax, disruptive TMS during the first word would mitigate the difference in the ERP responses for predicted and unpredicted categories in basic two-word constructions. Contrary to this hypothesis, perturbation of Broca's area at the predictive stage did not affect the ERP correlates of basic composition. The correlation strength between the electrical field induced by TMS and the ERP responses further confirmed this pattern. We discuss the present results considering an alternative account of the role of Broca's area in syntactic composition, namely the bottom-up integration of words into constituents, and of compensatory mechanisms within the language predictive network.

The role of the hippocampus in statistical learning and language recovery in persons with post stroke aphasia

Although several studies have aimed for accurate predictions of language recovery in post stroke aphasia, individual language outcomes remain hard to predict. Large-scale prediction models are built using data from patients mainly in the chronic phase after stroke, although it is clinically more relevant to consider data from the acute phase. Previous research has mainly focused on deficits, i.e., behavioral deficits or specific brain damage, rather than compensatory mechanisms, i.e., intact cognitive skills or undamaged brain regions. One such unexplored brain region that might support language (re)learning in aphasia is the hippocampus, a region that has commonly been associated with an individual's learning potential, including statistical learning. This refers to a set of mechanisms upon which we rely heavily in daily life to learn a range of regularities across cognitive domains. Against this background, thirty-three patients with aphasia (22 males and 11 females, M = 69.76 years, SD = 10.57 years) were followed for 1 year in the acute (1-2 weeks), subacute (3-6 months) and chronic phase (9-12 months) post stroke. We evaluated the unique predictive value of early structural hippocampal measures for short-term and long-term language outcomes (measured by the ANELT). In addition, we investigated whether statistical learning abilities were intact in patients with aphasia using three different tasks: an auditory-linguistic and visual task based on the computation of transitional probabilities and a visuomotor serial reaction time task. Finally, we examined the association of individuals' statistical learning potential with acute measures of hippocampal gray and white matter. Using Bayesian statistics, we found moderate evidence for the contribution of left hippocampal gray matter in the acute phase to the prediction of long-term language outcomes, over and above information on the lesion and the initial language deficit (measured by the ScreeLing). Non-linguistic statistical learning in patients with aphasia, measured in the subacute phase, was intact at the group level compared to 23 healthy older controls (8 males and 15 females, M = 74.09 years, SD = 6.76 years). Visuomotor statistical learning correlated with acute hippocampal gray and white matter. These findings reveal that particularly left hippocampal gray matter in the acute phase is a potential marker of language recovery after stroke, possibly through its statistical learning ability.

Differential contributions of inferior frontal gyrus subregions to sentence processing guided by intonation

Auditory sentence comprehension involves processing content (semantics), grammar (syntax), and intonation (prosody). The left inferior frontal gyrus (IFG) is involved in sentence comprehension guided by these different cues, with neuroimaging studies preferentially locating syntactic and semantic processing in separate IFG subregions. However, this regional specialisation has not been confirmed with a neurostimulation method. Consequently, the causal role of such a specialisation remains unclear. This study probed the role of the posterior IFG (pIFG) for syntactic processing and the anterior IFG (aIFG) for semantic processing with repetitive transcranial magnetic stimulation (rTMS) in a task that required the interpretation of the sentence's prosodic realisation. Healthy participants performed a sentence completion task with syntactic and semantic decisions, while receiving 10 Hz rTMS over either left aIFG, pIFG, or vertex (control). Initial behavioural analyses showed an inhibitory effect on accuracy without task-specificity. However, electric field simulations revealed differential effects for both subregions. In the aIFG, stronger stimulation led to slower semantic processing, with no effect of pIFG stimulation. In contrast, we found a facilitatory effect on syntactic processing in both aIFG and pIFG, where higher stimulation strength was related to faster responses. Our results provide first evidence for the functional relevance of left aIFG in semantic processing guided by intonation. The stimulation effect on syntactic responses emphasises the importance of the IFG for syntax processing, without supporting the hypothesis of a pIFG-specific involvement. Together, the results support the notion of functionally specialised IFG subregions for diverse but fundamental cues for language processing.

A Scanner Deeply: Predicting Gaze Heatmaps On Visualizations Using Crowdsourced Eye Movement Data

Visual perception is a key component of data visualization. Much prior empirical work uses eye movement as a proxy to understand human visual perception. Diverse apparatus and techniques have been proposed to collect eye movements, but there is still no optimal approach. In this paper, we review 30 prior works for collecting eye movements based on three axes: (1) the tracker technology used to measure eye movements; (2) the image stimulus shown to participants; and (3) the collection methodology used to gather the data. Based on this taxonomy, we employ a webcam-based eyetracking approach using task-specific visualizations as the stimulus. The low technology requirement means that virtually anyone can participate, thus enabling us to collect data at large scale using crowdsourcing: approximately 12,000 samples in total. Choosing visualization images as stimulus means that the eye movements will be specific to perceptual tasks associated with visualization. We use these data to propose a SCANNER DEEPLY, a virtual eyetracker model that, given an image of a visualization, generates a gaze heatmap for that image. We employ a computationally efficient, yet powerful convolutional neural network for our model. We compare the results of our work with results from the DVS model and a neural network trained on the Salicon dataset. The analysis of our gaze patterns enables us to understand how users grasp the structure of visualized data. We also make our stimulus dataset of visualization images available as part of this paper's contribution.

Structural Correlates of Overt Sentence Reading in Mild Cognitive Impairment and Mild-to-Moderate Alzheimer's Disease

BACKGROUND

Overt sentence reading in mild cognitive impairment (MCI) and mild-tomoderate Alzheimer's disease (AD) has been associated with slowness of speech, characterized by a higher number of pauses, shorter speech units and slower speech rate and attributed to reduced working memory/ attention and language capacity.

OBJECTIVE

This preliminary case-control study investigates whether the temporal organization of speech is associated with the volume of brain regions involved in overt sentence reading and explores the discriminative ability of temporal speech parameters and standard volumetric MRI measures for the classification of MCI and AD.

METHODS

Individuals with MCI, mild-to-moderate AD, and healthy controls (HC) had a structural MRI scan and read aloud sentences varying in cognitive-linguistic demand (length). The association between speech features and regional brain volumes was examined by linear mixed-effect modeling. Genetic programming was used to explore the discriminative ability of temporal and MRI features.

RESULTS

Longer sentences, slower speech rate, and a higher number of pauses and shorter interpausal units were associated with reduced volumes of the reading network. Speech-based classifiers performed similarly to the MRI-based classifiers for MCI-HC (67% vs. 68%) and slightly better for AD-HC (80% vs. 64%) and AD-MCI (82% vs. 59%). Adding the speech features to the MRI features slightly improved the performance of MRI-based classification for AD-HC and MCI-HC but not HC-MCI.

CONCLUSION

The temporal organization of speech in overt sentence reading reflects underlying volume reductions. It may represent a sensitive marker for early assessment of structural changes and cognitive- linguistic deficits associated with healthy aging, MCI, and AD.

Predictive Processing in Sign Languages: A Systematic Review

The objective of this article was to review existing research to assess the evidence for predictive processing (PP) in sign language, the conditions under which it occurs, and the effects of language mastery (sign language as a first language, sign language as a second language, bimodal bilingualism) on the neural bases of PP. This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework. We searched peer-reviewed electronic databases (SCOPUS, Web of Science, PubMed, ScienceDirect, and EBSCO host) and gray literature (dissertations in ProQuest). We also searched the reference lists of records selected for the review and forward citations to identify all relevant publications. We searched for records based on five criteria (original work, peer-reviewed, published in English, research topic related to PP or neural entrainment, and human sign language processing). To reduce the risk of bias, the remaining two authors with expertise in sign language processing and a variety of research methods reviewed the results. Disagreements were resolved through extensive discussion. In the final review, 7 records were included, of which 5 were published articles and 2 were dissertations. The reviewed records provide evidence for PP in signing populations, although the underlying mechanism in the visual modality is not clear. The reviewed studies addressed the motor simulation proposals, neural basis of PP, as well as the development of PP. All studies used dynamic sign stimuli. Most of the studies focused on semantic prediction. The question of the mechanism for the interaction between one’s sign language competence (L1 vs. L2 vs. bimodal bilingual) and PP in the manual-visual modality remains unclear, primarily due to the scarcity of participants with varying degrees of language dominance. There is a paucity of evidence for PP in sign languages, especially for frequency-based, phonetic (articulatory), and syntactic prediction. However, studies published to date indicate that Deaf native/native-like L1 signers predict linguistic information during sign language processing, suggesting that PP is an amodal property of language processing.

Distinctive neural correlates of phonological and reading impairment in fetal alcohol-exposed adolescents with and without facial dysmorphology

Prenatal alcohol exposure (PAE) has been linked to atypical brain and cognitive development, including poor academic performance in reading. This study utilized functional magnetic resonance imaging and diffusion tensor imaging to characterize functional and structural mechanisms mediating reading deficits in 26 adolescents with PAE-related facial dysmorphology (fetal alcohol syndrome (FAS)/partial FAS (PFAS)), 29 heavily-exposed (HE) non-syndromal adolescents, in comparison with 19 typically developing controls. The FAS/PFAS and HE groups were balanced in terms of levels of PAE and reading (dis)ability. While neural alterations in the posterior association cortices were evident in both PAE groups, distinctive neural correlates of reading (dis)abilities were observed between adolescents with and without facial dysmorphology. Specifically, compared to the HE and control groups, the syndromal adolescents showed greater activation in the right precentral gyrus during phonological processing and rightward lateralization in an important reading-related tract (inferior longitudinal fasciculus, ILF), suggesting an atypical reliance on the right hemisphere. By contrast, in the HE, better reading skills were positively correlated with neural activation in the left angular gyrus and white matter organization of the left ILF, although the brain function-behavior relation was weaker than among the controls, suggesting less efficient function of the typical reading network. Our findings provide converging evidence at both the neural functional and structural levels for distinctive brain mechanisms underlying atypical reading and phonological processing in PAE adolescents with and without facial dysmorphology.

Missing links: The functional unification of language and memory (L∪M)

The field of neurocognition is currently undergoing a significant change of perspective. Traditional neurocognitive models evolved into an integrative and dynamic vision of cognitive functioning. Dynamic integration assumes an interaction between cognitive domains traditionally considered to be distinct. Language and declarative memory are regarded as separate functions supported by different neural systems. However, they also share anatomical structures (notably, the inferior frontal gyrus, the supplementary motor area, the superior and middle temporal gyrus, and the hippocampal complex) and cognitive processes (such as semantic and working memory) that merge to endorse our quintessential daily lives. We propose a new model, "L∪M" (i.e., Language/union/Memory), that considers these two functions interactively. We fractionated language and declarative memory into three fundamental dimensions or systems ("Receiver-Transmitter", "Controller-Manager" and "Transformer-Associative" Systems), that communicate reciprocally. We formalized their interactions at the brain level with a connectivity-based approach. This new taxonomy overcomes the modular view of cognitive functioning and reconciles functional specialization with plasticity in neurological disorders.

Predictive pre-activation of orthographic and lexical-semantic representations facilitates visual word recognition

To a crucial extent, the efficiency of reading results from the fact that visual word recognition is faster in predictive contexts. Predictive coding models suggest that this facilitation results from pre-activation of predictable stimulus features across multiple representational levels before stimulus onset. Still, it is not sufficiently understood which aspects of the rich set of linguistic representations that are activated during reading-visual, orthographic, phonological, and/or lexical-semantic-contribute to context-dependent facilitation. To investigate in detail which linguistic representations are pre-activated in a predictive context and how they affect subsequent stimulus processing, we combined a well-controlled repetition priming paradigm, including words and pseudowords (i.e., pronounceable nonwords), with behavioral and magnetoencephalography measurements. For statistical analysis, we used linear mixed modeling, which we found had a higher statistical power compared to conventional multivariate pattern decoding analysis. Behavioral data from 49 participants indicate that word predictability (i.e., context present vs. absent) facilitated orthographic and lexical-semantic, but not visual or phonological processes. Magnetoencephalography data from 38 participants show sustained activation of orthographic and lexical-semantic representations in the interval before processing the predicted stimulus, suggesting selective pre-activation at multiple levels of linguistic representation as proposed by predictive coding. However, we found more robust lexical-semantic representations when processing predictable in contrast to unpredictable letter strings, and pre-activation effects mainly resembled brain responses elicited when processing the expected letter string. This finding suggests that pre-activation did not result in "explaining away" predictable stimulus features, but rather in a "sharpening" of brain responses involved in word processing.

The Neural Basis of Semantic Prediction in Sentence Comprehension

Although prediction plays an important role in language comprehension, its precise neural basis remains unclear. This fMRI study investigated whether and how semantic-category-specific and common cerebral areas are recruited in predictive semantic processing during sentence comprehension. We manipulated the semantic constraint of sentence contexts, upon which a tool-related, a building-related, or no specific category of noun is highly predictable. This noun-predictability effect was measured not only over the target nouns but also over their preceding transitive verbs. Both before and after the appearance of target nouns, left anterior supramarginal gyrus was specifically activated for tool-related nouns and left parahippocampal place area was activated specifically for building-related nouns. The semantic-category common areas included a subset of left inferior frontal gyrus during the anticipation of incoming target nouns (activity enhancement for high predictability) and included a wide spread of areas (bilateral inferior frontal gyrus, left superior/middle temporal gyrus, left medial pFC, and left TPJ) during the integration of actually perceived nouns (activity reduction for high predictability). These results indicated that the human brain recruits fine divisions of cortical areas to distinguish different semantic categories of predicted words, and anticipatory semantic processing relies, at least partially, on top-down prediction conducted in higher-level cortical areas.

The impact of musical expertise on anticipatory semantic processing during online speech comprehension: An electroencephalography study

Musical experience has been found to aid speech perception. This electroencephalography study further examined whether and how musical expertise affects high-level predictive semantic processing in speech comprehension. Musicians and non-musicians listened to semantically strongly/weakly constraining sentences, with each sentence being primed by a congruent/incongruent sentence-prosody. At the target nouns, a N400 reduction effect (strongly vs. weakly constraining) was observed in both groups, with the onset-latency of this effect being delayed for incongruent (vs. congruent) priming. At the transitive verbs preceding these target nouns, musicians' event-related-potential amplitude (in incongruent-priming) and beta-band oscillatory power (in congruent- and incongruent-priming) showed a semantic-constraint effect, and were correlated with the predictability of incoming nouns; non-musicians only demonstrated an event-related-potential semantic-constraint effect, which was correlated with the predictability of current verbs. These results indicate musical expertise enhances semantic prediction tendency in speech comprehension, and this effect might be not just an aftereffect of facilitated acoustic/phonological processing.

Hierarchical control as a shared neurocognitive mechanism for language and music

Although comparative research has made substantial progress in clarifying the relationship between language and music as neurocognitive systems from both a theoretical and empirical perspective, there is still no consensus about which mechanisms, if any, are shared and how they bring about different neurocognitive systems. In this paper, we tackle these two questions by focusing on hierarchical control as a neurocognitive mechanism underlying syntax in language and music. We put forward the Coordinated Hierarchical Control (CHC) hypothesis: linguistic and musical syntax rely on hierarchical control, but engage this shared mechanism differently depending on the current control demand. While linguistic syntax preferably engages the abstract rule-based control circuit, musical syntax rather employs the coordination of the abstract rule-based and the more concrete motor-based control circuits. We provide evidence for our hypothesis by reviewing neuroimaging as well as neuropsychological studies on linguistic and musical syntax. The CHC hypothesis makes a set of novel testable predictions to guide future work on the relationship between language and music.

Tracking Cognitive Spare Capacity During Speech Perception With EEG/ERP: Effects of Cognitive Load and Sentence Predictability

OBJECTIVES

Listening to speech in adverse listening conditions is effortful. Objective assessment of cognitive spare capacity during listening can serve as an index of the effort needed to understand speech. Cognitive spare capacity is influenced both by signal-driven demands posed by listening conditions and top-down demands intrinsic to spoken language processing, such as memory use and semantic processing. Previous research indicates that electrophysiological responses, particularly alpha oscillatory power, may index listening effort. However, it is not known how these indices respond to memory and semantic processing demands during spoken language processing in adverse listening conditions. The aim of the present study was twofold: first, to assess the impact of memory demands on electrophysiological responses during recognition of degraded, spoken sentences, and second, to examine whether predictable sentence contexts increase or decrease cognitive spare capacity during listening.

DESIGN

Cognitive demand was varied in a memory load task in which young adult participants (n = 20) viewed either low-load (one digit) or high-load (seven digits) sequences of digits, then listened to noise-vocoded spoken sentences that were either predictable or unpredictable, and then reported the final word of the sentence and the digits. Alpha oscillations in the frequency domain and event-related potentials in the time domain of the electrophysiological data were analyzed, as was behavioral accuracy for both words and digits.

RESULTS

Measured during sentence processing, event-related desynchronization of alpha power was greater (more negative) under high load than low load and was also greater for unpredictable than predictable sentences. A complementary pattern was observed for the P300/late positive complex (LPC) to sentence-final words, such that P300/LPC amplitude was reduced under high load compared with low load and for unpredictable compared with predictable sentences. Both words and digits were identified more quickly and accurately on trials in which spoken sentences were predictable.

CONCLUSIONS

Results indicate that during a sentence-recognition task, both cognitive load and sentence predictability modulate electrophysiological indices of cognitive spare capacity, namely alpha oscillatory power and P300/LPC amplitude. Both electrophysiological and behavioral results indicate that a predictive sentence context reduces cognitive demands during listening. Findings contribute to a growing literature on objective measures of cognitive demand during listening and indicate predictable sentence context as a top-down factor that can support ease of listening.

Speech Perception under the Tent: A Domain-general Predictive Role for the Cerebellum

The role of the cerebellum in speech perception remains a mystery. Given its uniform architecture, we tested the hypothesis that it implements a domain-general predictive mechanism whose role in speech is determined by connectivity. We collated all neuroimaging studies reporting cerebellar activity in the Neurosynth database (n = 8206). From this set, we found all studies involving passive speech and sound perception (n = 72, 64% speech, 12.5% sounds, 12.5% music, and 11% tones) and speech production and articulation (n = 175). Standard and coactivation neuroimaging meta-analyses were used to compare cerebellar and associated cortical activations between passive perception and production. We found distinct regions of perception- and production-related activity in the cerebellum and regions of perception-production overlap. Each of these regions had distinct patterns of cortico-cerebellar connectivity. To test for domain-generality versus specificity, we identified all psychological and task-related terms in the Neurosynth database that predicted activity in cerebellar regions associated with passive perception and production. Regions in the cerebellum activated by speech perception were associated with domain-general terms related to prediction. One hallmark of predictive processing is metabolic savings (i.e., decreases in neural activity when events are predicted). To test the hypothesis that the cerebellum plays a predictive role in speech perception, we examined cortical activation between studies reporting cerebellar activation and those without cerebellar activation during speech perception. When the cerebellum was active during speech perception, there was far less cortical activation than when it was inactive. The results suggest that the cerebellum implements a domain-general mechanism related to prediction during speech perception.

Evidence of impaired naming in patients with hippocampal amnesia

Object naming involves accessing meaning and retrieving the associated word form from remote semantic memory. Historically, previously acquired semantic knowledge (i.e., remote semantic memory) was thought to be independent of the hippocampus via neocortical consolidation. This view is based on evidence demonstrating a dissociation in behavior in patients with hippocampal amnesia: amnesic patients are impaired in acquiring new vocabulary yet can name and define previously acquired words. More recently, the view that remote semantic memory is hippocampus-independent has been challenged by the documentation of disruptions in aspects of remote semantic memory in patients with hippocampal amnesia, particularly in language use and depth of semantic knowledge. Based on these findings, we hypothesized that the hippocampus plays a long-term role in remote semantic memory. We tested amnesic patients and demographically matched healthy comparison participants in an extensive naming task using photographic images of objects normalized for familiarity, object agreement, and visual complexity. Amnesic patients were less likely to correctly name objects than healthy comparison participants. Further, amnesic patients' performance worsened for words that were less familiar, more visually complex, and had less object agreement. These findings suggest that the hippocampus may play a long-term role in semantic memory processes, rather than a time-limited role in the initial acquisition of semantic information, and that hippocampal damage can disrupt object naming.

Neural correlates of visual attention during risky decision evidence integration

Value-based decision-making is presumed to involve a dynamic integration process that supports assessing the potential outcomes of different choice options. Decision frameworks assume the value of a decision rests on both the desirability and risk surrounding an outcome. Previous work has highlighted neural representations of risk in the human brain, and their relation to decision choice. Key neural regions including the insula and anterior cingulate cortex (ACC) have been implicated in encoding the effects of risk on decision outcomes, including approach and avoidance. Yet, it remains unknown whether these regions are involved in the dynamic integration processes that precede and drive choice, and their relationship with ongoing attention. Here, we used concurrent fMRI and eye-tracking to discern neural activation related to visual attention preceding choice between sure-thing (i.e. safe) and risky gamble options. We found activation in both dorsal ACC (dACC) and posterior insula (PI) scaled in opposite directions with the difference in attention to risky rewards relative to risky losses. PI activation also differentiated foveations on both risky options (rewards and losses) relative to a sure-thing option. These findings point to ACC involvement in ongoing evaluation of risky but higher value options. The role of PI in risky outcomes points to a more general evaluative role in the decision-making that compares both safe and risky outcomes, irrespective of potential for gains or losses.

Eye Tracking Measures for Studying Language Comprehension Deficits in Aphasia: A Systematic Search and Scoping Review

Aim The aim of this scoping review is to identify the eye tracking paradigms and eye movement measures used to investigate auditory and reading comprehension deficits in persons with aphasia (PWA). Method MEDLINE via PubMed, Cochrane, CINAHL, Embase, PsycINFO, OTseeker, Scopus, Google Scholar, Grey Literature Database, and ProQuest Search (Dissertations & Theses) were searched for relevant studies. The Covidence software was used to manage the initial and full-text screening process for the search. Results and Discussion From a total of 1,803 studies, 68 studies were included for full-text screening. In addition, 418 records from gray literature were also screened. After full-text screening, 16 studies were included for this review-12 studies for auditory comprehension in PWA and four studies for reading comprehension in PWA. The review highlights the use of common eye tracking paradigms used to study language comprehension in PWA. We also discusse eye movement measures and how they help in assessing auditory and reading comprehension. Methodological challenges of using eye tracking are discussed. Conclusion The studies summarized in this scoping review provide evidence that the eye tracking methods are beneficial for studying auditory and reading comprehension in PWA.

The limited role of hippocampal declarative memory in transient semantic activation during online language processing

Recent findings point to a role for hippocampus in the moment-by-moment processing of language, including the use and generation of semantic features in certain contexts. What role the hippocampus might play in the processing of semantic relations in spoken language comprehension, however, is unknown. Here we test patients with bilateral hippocampal damage and dense amnesia in order to examine the necessity of hippocampus for lexico-semantic mapping processes in spoken language understanding. In two visual-world eye-tracking experiments, we monitor eye movements to images that are semantically related to spoken words and sentences. We find no impairment in amnesia, relative to matched healthy comparison participants. These findings suggest, at least for close semantic links and simple language comprehension tasks, a lack of necessity for hippocampus in lexico-semantic mapping between spoken words and simple pictures.

Cloze enough? Hemodynamic effects of predictive processing during natural reading

Evidence accrues that readers form multiple hypotheses about upcoming words. The present study investigated the hemodynamic effects of predictive processing during natural reading by means of combining fMRI and eye movement recordings. In particular, we investigated the neural and behavioral correlates of precision-weighted prediction errors, which are thought to be indicative of subsequent belief updating. Participants silently read sentences in which we manipulated the cloze probability and the semantic congruency of the final word that served as an index for precision and prediction error respectively. With respect to the neural correlates, our findings indicate an enhanced activation within the left inferior frontal and middle temporal gyrus suggesting an effect of precision on prediction update in higher (lexico-)semantic levels. Despite being evident at the neural level, we did not observe any evidence that this mechanism resulted in disproportionate reading times on participants' eye movements. The results speak against discrete predictions, but favor the notion that multiple words are activated in parallel during reading.

The language of mental images: Characterizing hippocampal contributions to imageable word use during event construction

Accumulating evidence suggests that the hippocampus plays a critical role in the creative and flexible use of language at the sentence or discourse level. Yet it is currently unclear whether the hippocampus also supports language use at the level of single words. A recent study by Hilverman et al. (2017) found that amnesic patients with hippocampal damage use less imageable words when describing autobiographical episodes compared to healthy controls, but this deficit was attributed to patients' deficits in episodic memory rather than impairments in linguistic functions of the hippocampus per se. Yet, in addition to affecting word use by way of its role in memory, the hippocampus could also impact language use more directly. The current study aimed to test this hypothesis by investigating the status of imageable word use in amnesia during two different types of language production tasks. In Experiment 1, participants constructed narratives about events depicted in visually presented pictures (picture narratives). In Experiment 2, participants constructed verbal narratives about remembered events from the past or simulated events in the future (past/future narratives). Across all types of narratives, patients produced words that were rated as having similar levels of imageability compared to controls. Importantly, this was the case both in patients' picture narratives, which did not require generating details from episodic memory and were matched to those of controls with respect to narrative content, and in patients' narratives about past/future events, which required generating details from memory and which were reduced in narrative content compared to those of controls. These results distinguish between the quantity and quality of individual linguistic details produced in amnesia during narrative construction, and suggest that the use of imageable linguistic representations does not depend on intact episodic memory and can be supported by regions outside the hippocampus.

When a sentence loses semantics: Selective involvement of a left anterior temporal subregion in semantic processing

Although the left anterior temporal lobe (ATL) has been associated with semantic processing, the role of this region in syntactic structure building of sentences remains a subject of debate. Functional neuroimaging studies contrasting well-formed sentences with word lists lacking syntactic structure have produced mixed results. The current functional magnetic resonance imaging study examined whether the left ATL is selectively involved in semantic processing or also plays a role in syntactic structure building by manipulating syntactic complexity and meaningfulness in a novel way. To deprive semantic/pragmatic information from a sentence, we replaced all content words with pronounceable meaningless placeholders. We conducted an experiment with a 2 × 2 factorial design with factors of SEMANTICS (natural sentences [NAT]; sentences with placeholders [SPH]) and SYNTAX (the basic Japanese Subject-Object-Verb [SOV] word order; a changed Object-Subject-Verb [OSV] word order). A main effect of SEMANTICS (NAT > SPH) was found in the left ATL, as well as in the ventral occipitotemporal regions. The opposite contrast (SPH > NAT) revealed activation in the dorsal regions encompassing Brodmann area 44, the premotor area, and the parietal cortex in the left hemisphere. We found no main effect of SYNTAX (OSV > SOV) in a subregion of the left ATL that was more responsive to natural sentences than meaningless sentences. These results indicate selective involvement of a subregion of the left ATL in semantic/pragmatic processing.

Both activation and deactivation of functional networks support increased sentence processing costs

The research on the neural correlates underlying the language system has gradually moved away from the traditional Broca-Wernicke framework to a network perspective in the past 15 years. Language processing is found to be supported by the co-activation of both core and peripheral brain regions. However, the dynamic co-activation patterns of these brain regions serving different language functions remain to be fully revealed. The present functional magnetic resonance imaging (fMRI) study focused on sentence processing at different syntactic complexity levels to examine how the co-activation of different brain networks will be modulated by increased processing costs. Chinese relative clauses were used to probe the two dimensions of syntactic complexity: embeddedness (left-branching vs. center-embedded) and gap-filler dependency (subject-gap vs. object-gap) using the general linear model (GLM) approach, independent component analysis (ICA) and graph theoretical analysis. In contrast to localized activation revealed by the GLM approach, ICA identified more extensive networks both positively and negatively correlated with the task. We found that the posterior default mode network was anti-correlated to the gap-filler integration costs with increased deactivation for the left-branching object relative clauses compared to subject relative clauses, suggesting the involvement of this network in leveraging the cognitive resources based on the complexity level of the language task. Concurrent activation and deactivation of networks were found to be associated with the higher costs induced by center-embedding and its interaction with gap-filler integration. The graph theoretical analysis further unveiled that center-embeddedness imposed more attentional demand on the subject relative clause, as characterized by its higher degree and strength in the ventral attention network, and higher processing costs of syntactic reanalysis on the object relative clause, as characterized by increased intermodular connections of the language network with other networks. The results suggest that network activation and deactivation profiles are modulated by different dimensions of syntactic complexity to serve the higher demand of creating a coherent semantic representation.

Semantics-weighted lexical surprisal modeling of naturalistic functional MRI time-series during spoken narrative listening

Probabilistic language models are increasingly used to provide neural representations of linguistic features under naturalistic settings. Word surprisal models can be applied to continuous fMRI recordings during task-free listening of narratives, to detect regions linked to language prediction and comprehension. Here, to this purpose, a novel semantics-weighted lexical surprisal is applied to naturalistic fMRI data. FMRI was performed at 3 Tesla in 31 subjects during task-free listening to a 12-minute audiobook played in both original and word-reversed (control) version. Lexical-only and semantics-weighted lexical surprisal models were estimated for the original and control word series. The two series were alternatively chosen to build the predictor of interest in the first-level general linear model and were compared in the second-level (group) analysis. The addition of the surprisal predictor to the stimulus-related predictors significantly improved the fitting of the neural signal. In average, the semantics-weighted model yielded lower surprisal values and, in some areas, better fitting of the fMRI data compared to the lexical-only model. The two models produced both overlapping and distinct activations: while lexical-only surprisal activated secondary auditory areas in the superior temporal gyri and the cerebellum, semantics-weighted surprisal additionally activated the left inferior frontal gyrus. These results confirm the usefulness of surprisal models in the naturalistic fMRI analysis of linguistic processes and suggest that the use of semantic information may increase the sensitivity of a probabilistic language model in higher-order language-related areas, with possible implications for future naturalistic fMRI studies of language under normal and (clinically or pharmacologically) modified conditions.

Cerebellar Theta and Beta Noninvasive Stimulation Rhythms Differentially Influence Episodic Memory versus Semantic Prediction

The human cerebellum is thought to interact with distributed brain networks to support cognitive abilities such as episodic memory and semantic prediction. Hippocampal and fronto-temporo-parietal networks that respectively support episodic memory versus semantic prediction have been associated with distinct endogenous oscillatory activity frequency bands: theta (∼3-8 Hz) versus beta (∼13-30 Hz) respectively. We sought to test whether it is possible to toggle cerebellar participation in episodic memory versus semantic prediction by noninvasively stimulating with theta versus beta rhythmic transcranial magnetic stimulation. In human subjects of both sexes, cerebellar theta stimulation improved episodic memory encoding but did not influence neural signals of semantic prediction, whereas beta stimulation of the same cerebellar location increased neural signals of semantic prediction but did not influence episodic memory encoding. This constitutes evidence for double dissociation of cerebellar contributions to semantic prediction versus episodic memory based on stimulation rhythm, supporting the hypothesis that the cerebellum can be biased to support these distinct cognitive abilities at the command of network-specific rhythmic activity.SIGNIFICANCE STATEMENT The cerebellum interacts with several distinct large-scale brain networks for cognitive function, but the factors governing selectivity of such interactions for particular functions are not fully understood. We tested the hypothesis that cerebellar contributions to cognition are guided by neural oscillations with function-specific frequency bands. We demonstrated that matching noninvasive stimulation to network-specific frequencies selectively enhanced episodic memory versus semantic prediction. These findings suggest that cerebellar contributions to cognitive networks are selected based on corresponding activity rhythms and could be used to develop cerebellar stimulation interventions for specific neurocognitive impairments.

Syntax-Sensitive Regions of the Posterior Inferior Frontal Gyrus and the Posterior Temporal Lobe Are Differentially Recruited by Production and Perception

Matchin and Hickok (2020) proposed that the left posterior inferior frontal gyrus (PIFG) and the left posterior temporal lobe (PTL) both play a role in syntactic processing, broadly construed, attributing distinct functions to these regions with respect to production and perception. Consistent with this hypothesis, functional dissociations between these regions have been demonstrated with respect to lesion-symptom mapping in aphasia. However, neuroimaging studies of syntactic comprehension typically show similar activations in these regions. In order to identify whether these regions show distinct activation patterns with respect to syntactic perception and production, we performed an fMRI study contrasting the subvocal articulation and perception of structured jabberwocky phrases (syntactic), sequences of real words (lexical), and sequences of pseudowords (phonological). We defined two sets of language-selective regions of interest (ROIs) in individual subjects for the PIFG and the PTL using the contrasts [syntactic > lexical] and [syntactic > phonological]. We found robust significant interactions of comprehension and production between these 2 regions at the syntactic level, for both sets of language-selective ROIs. This suggests a core difference in the function of these regions with respect to production and perception, consistent with the lesion literature.

Natural Reading in Parkinson's Disease With and Without Mild Cognitive Impairment

Background: Patients with Parkinson’s disease (PD) show eye movement abnormalities and frequently complain about difficulties in reading. So far, it is unclear whether basal ganglia dysfunction or cognitive impairment has a greater impact on eye movements during reading.

Objective: To analyze eye movement behavior during a natural reading task with respect to cognitive state and dopaminergic therapy in PD and healthy controls.

Methods: Eye movements of 59 PD patients and 29 age- and education-matched healthy controls were recorded during mute, self-paced reading of a text. 25 cognitively normal PD patients performed the task additionally in off medication state. Clinical assessment included a comprehensive neuropsychological test battery and the motor section of MDS—Unified Parkinson’s Disease Rating Scale (MDS-UPDRS).

Results: PD-mild cognitive impairment (MCI) was diagnosed in 21 patients. Reading speed was significantly reduced in PD-MCI compared to healthy controls and PD patients without MCI due to higher numbers of progressive saccades. Cognitively intact PD patients showed no significant alterations of reading speed or eye movement pattern during reading. The fixation duration tended to be prolonged in PD compared to healthy controls and decreased significantly after levodopa intake. Scores for executive functions, attention, and language correlated with reading speed in the PD group.

Conclusion: The present study is the first to reveal (1) reduced reading speed with altered reading pattern in PD with MCI and (2) a relevant impact of levodopa on fixation duration during reading in PD. Further research is needed to determine whether therapeutic interventions, e.g., levodopa or neuropsychological training, improve the subjective reading experience for patients with PD.

Theta oscillations support the interface between language and memory

Recent evidence shows that hippocampal theta oscillations, usually linked to memory and navigation, are also observed during online language processing, suggesting a shared neurophysiological mechanism between language and memory. However, it remains to be established what specific roles hippocampal theta oscillations may play in language, and whether and how theta mediates the communication between the hippocampus and the perisylvian cortical areas, generally thought to support language processing. With whole-head magnetoencephalographic (MEG) recordings, the present study investigated these questions with two experiments. Using a violation paradigm, extensively used for studying neural underpinnings of different aspects of linguistic processing, we found increased theta power (4-8 ​Hz) in the hippocampal formation, when participants read a semantically incorrect vs. correct sentence ending. Such a pattern of results was replicated using different sentence stimuli in another cohort of participants. Importantly, no significant hippocampal theta power increase was found when participants read a semantically correct but syntactically incorrect sentence ending vs. a correct sentence ending. These findings may suggest that hippocampal theta oscillations are specifically linked to lexical-semantic related processing, and not general information processing in sentence reading. Furthermore, we found significantly transient theta phase coupling between the hippocampus and the left superior temporal gyrus, a hub area of the cortical network for language comprehension. This transient theta phase coupling may provide an important channel that links the memory and language systems for the generation of sentence meaning. Overall, these findings help specify the role of hippocampal theta in language, and provide a novel neurophysiological mechanism at the network level that may support the interface between memory and language.

The Cortical Organization of Syntax

Syntax, the structure of sentences, enables humans to express an infinite range of meanings through finite means. The neurobiology of syntax has been intensely studied but with little consensus. Two main candidate regions have been identified: the posterior inferior frontal gyrus (pIFG) and the posterior middle temporal gyrus (pMTG). Integrating research in linguistics, psycholinguistics, and neuroscience, we propose a neuroanatomical framework for syntax that attributes distinct syntactic computations to these regions in a unified model. The key theoretical advances are adopting a modern lexicalized view of syntax in which the lexicon and syntactic rules are intertwined, and recognizing a computational asymmetry in the role of syntax during comprehension and production. Our model postulates a hierarchical lexical-syntactic function to the pMTG, which interconnects previously identified speech perception and conceptual-semantic systems in the temporal and inferior parietal lobes, crucial for both sentence production and comprehension. These relational hierarchies are transformed via the pIFG into morpho-syntactic sequences, primarily tied to production. We show how this architecture provides a better account of the full range of data and is consistent with recent proposals regarding the organization of phonological processes in the brain.

fMRI reveals language-specific predictive coding during naturalistic sentence comprehension

Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.

Semantic Memory and the Hippocampus: Revisiting, Reaffirming, and Extending the Reach of Their Critical Relationship

Since Tulving proposed a distinction in memory between semantic and episodic memory, considerable effort has been directed towards understanding their similar and unique features. Of particular interest has been the extent to which semantic and episodic memory have a shared dependence on the hippocampus. In contrast to the definitive evidence for the link between hippocampus and episodic memory, the role of the hippocampus in semantic memory has been a topic of considerable debate. This debate stems, in part, from highly variable reports of new semantic memory learning in amnesia ranging from profound impairment to full preservation, and various degrees of deficit and ability in between. More recently, a number of significant advances in experimental methods have occurred, alongside new provocative data on the role of the hippocampus in semantic memory, making this an ideal moment to revisit this debate, to re-evaluate data, methods, and theories, and to synthesize new findings. In line with these advances, this review has two primary goals. First, we provide a historical lens with which to reevaluate and contextualize the literature on semantic memory and the hippocampus. The second goal of this review is to provide a synthesis of new findings on the role of the hippocampus and semantic memory. With the perspective of time and this critical review, we arrive at the interpretation that the hippocampus does indeed make necessary contributions to semantic memory. We argue that semantic memory, like episodic memory, is a highly flexible, (re)constructive, relational and multimodal system, and that there is value in developing methods and materials that fully capture this depth and richness to facilitate comparisons to episodic memory. Such efforts will be critical in addressing questions regarding the cognitive and neural (inter)dependencies among forms of memory, and the role that these forms of memory play in support of cognition more broadly. Such efforts also promise to advance our understanding of how words, concepts, and meaning, as well as episodes and events, are instantiated and maintained in memory and will yield new insights into our two most quintessentially human abilities: memory and language.

Is there a prediction network? Meta-analytic evidence for a cortical-subcortical network likely subserving prediction

Predictive coding is an increasingly influential and ambitious concept in neuroscience viewing the brain as a 'hypothesis testing machine' that constantly strives to minimize prediction error, the gap between its predictions and the actual sensory input. Despite the invaluable contribution of this framework to the formulation of brain function, its neuroanatomical foundations have not been fully defined. To address this gap, we conducted activation likelihood estimation (ALE) meta-analysis of 39 neuroimaging studies of three functional domains (action perception, language and music) inherently involving prediction. The ALE analysis revealed a widely distributed brain network encompassing regions within the inferior and middle frontal gyri, anterior insula, premotor cortex, pre-supplementary motor area, temporoparietal junction, striatum, thalamus/subthalamus and the cerebellum. This network is proposed to subserve domain-general prediction and its relevance to motor control, attention, implicit learning and social cognition is discussed in light of the predictive coding scheme. Better understanding of the presented network may help advance treatments of neuropsychiatric conditions related to aberrant prediction processing and promote cognitive enhancement in healthy individuals.

Sentence processing is modulated by the current linguistic environment and a priori information: An fMRI study

INTRODUCTION

Words are not processed in isolation but in rich contexts that are used to modulate and facilitate language comprehension. Here, we investigate distinct neural networks underlying two types of contexts, the current linguistic environment and verb-based syntactic preferences.

METHODS

We had two main manipulations. The first was the current linguistic environment, where the relative frequencies of two syntactic structures (prepositional object [PO] and double-object [DO]) would either follow everyday linguistic experience or not. The second concerned the preference toward one or the other structure depending on the verb; learned in everyday language use and stored in memory. German participants were reading PO and DO sentences in German while brain activity was measured with functional magnetic resonance imaging.

RESULTS

First, the anterior cingulate cortex (ACC) showed a pattern of activation that integrated the current linguistic environment with everyday linguistic experience. When the input did not match everyday experience, the unexpected frequent structure showed higher activation in the ACC than the other conditions and more connectivity from the ACC to posterior parts of the language network. Second, verb-based surprisal of seeing a structure given a verb (PO verb preference but DO structure presentation) resulted, within the language network (left inferior frontal and left middle/superior temporal gyrus) and the precuneus, in increased activation compared to a predictable verb-structure pairing.

CONCLUSION

In conclusion, (1) beyond the canonical language network, brain areas engaged in prediction and error signaling, such as the ACC, might use the statistics of syntactic structures to modulate language processing, (2) the language network is directly engaged in processing verb preferences. These two networks show distinct influences on sentence processing.

Prediction, Psychosis, and the Cerebellum

An increasingly influential hypothesis posits that many of the diverse symptoms of psychosis can be viewed as reflecting dysfunctional predictive mechanisms. Indeed, to perceive something is to take a sensory input and make a prediction of the external source of that signal; thus, prediction is perhaps the most fundamental neural computation. Given the ubiquity of prediction, a more challenging problem is to specify the unique predictive role or capability of a particular brain structure. This question is relevant when considering recent claims that one aspect of the predictive deficits observed in psychotic disorders might be related to cerebellar dysfunction, a subcortical structure known to play a critical role in predictive sensorimotor control and perhaps higher-level cognitive function. Here, we review evidence bearing on this question. We first focus on clinical, behavioral, and neuroimaging findings suggesting cerebellar involvement in psychosis and, specifically, schizophrenia. We then review a relatively novel line of research exploring whether computational models of cerebellar motor function can also account for cerebellar involvement in higher-order human cognition, and in particular, language function. We end the review by highlighting some key gaps in these literatures, limitations that currently preclude strong conclusions regarding cerebellar involvement in psychosis.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

Grammatical class modulates the (left) inferior frontal gyrus within 100 milliseconds when syntactic context is predictive

The current study set out to examine the spatiotemporal dynamics of predictive processing during syntactic processing. To do so, we conducted an MEG experiment in which we contrasted MRI-constrained sources elicited by nouns and verbs when they were preceded by a predictive syntactic context (i.e., possessive pronouns for nouns, and personal pronouns for verbs) versus a non-predictive syntactic context (visually matched symbols). The results showed rapid (from ~80 ms onwards) noun-verb differences in the left and (to a lesser extent) right inferior frontal gyri (IFG), but only when those nouns and verbs were preceded by the syntactically predictive context (i.e. their corresponding pronoun). Furthermore, the contrast between possessive and personal pronouns that preceded the rapid noun-verb modulations in the (L)IFG also produced differences in source activation in various regions of the prefrontal cortex (the superior frontal and orbitofrontal cortex). We suggest the data show that syntactic unification manifests very early on during processing in the LIFG. The speed of such syntactic unification operations is hypothesized to be driven by predictive top-down activations stemming from a domain-general network in the prefrontal cortex.

Specific lexico-semantic predictions are associated with unique spatial and temporal patterns of neural activity

We used Magnetoencephalography (MEG) in combination with Representational Similarity Analysis to probe neural activity associated with distinct, item-specific lexico-semantic predictions during language comprehension. MEG activity was measured as participants read highly constraining sentences in which the final words could be predicted. Before the onset of the predicted words, both the spatial and temporal patterns of brain activity were more similar when the same words were predicted than when different words were predicted. The temporal patterns localized to the left inferior and medial temporal lobe. These findings provide evidence that unique spatial and temporal patterns of neural activity are associated with item-specific lexico-semantic predictions. We suggest that the unique spatial patterns reflected the prediction of spatially distributed semantic features associated with the predicted word, and that the left inferior/medial temporal lobe played a role in temporally 'binding' these features, giving rise to unique lexico-semantic predictions.