Syntactic unification operations are reflected in oscillatory dynamics during on-line sentence comprehension

Reading Shakespearean tropes in a foreign tongue: Age of L2 acquisition modulates neural responses to functional shifts

Functional shifts (FSs) - morphosyntactically marked words evoking coherent but novel meanings - are ubiquitous in English and, specially, in Shakespearean literature. While their neural signatures have been explored in native speakers, no study has targeted foreign-language users, let alone comparing early and late bilinguals. Here, we administered a validated FS paradigm to subjects from both populations and evaluated time-frequency modulations evoked by FS and control sentences. Early bilinguals exhibited greater sensitivity towards FSs, indexed by reduced fronto-posterior theta-band oscillations across semantic- and structural-integration windows. Such oscillatory modulations may represent a key marker of age-of-acquisition effects during foreign-language wordplay processing.

Developmental differences in the neural oscillations underlying auditory sentence processing in children and adults

Although very young children seem to process ongoing language quickly and effortlessly, neuroimaging and behavioral studies reveal that children continue to mature in their language skills through adolescence. During this prolonged development, children likely engage the same basic cognitive processes and neural mechanisms to perform language tasks as adults, but in somewhat different ways. In this study we used time frequency analysis of EEG to identify developmental differences in the engagement of neural oscillations between children (ages 10-12) and adults while listening to naturally-paced sentences. Adults displayed consistent beta changes throughout the sentence compared to children, thought to be related to efficient syntactic integration, and children displayed more broadly distributed theta changes than adults, thought to be related to more effortful semantic integration. Few differences in alpha, related to verbal working memory, existed between groups. These findings shed new light on developmental changes in the neuronal processes underlying language comprehension.

Grounding the neurobiology of language in first principles: The necessity of non-language-centric explanations for language comprehension

Recent decades have ushered in tremendous progress in understanding the neural basis of language. Most of our current knowledge on language and the brain, however, is derived from lab-based experiments that are far removed from everyday language use, and that are inspired by questions originating in linguistic and psycholinguistic contexts. In this paper we argue that in order to make progress, the field needs to shift its focus to understanding the neurobiology of naturalistic language comprehension. We present here a new conceptual framework for understanding the neurobiological organization of language comprehension. This framework is non-language-centered in the computational/neurobiological constructs it identifies, and focuses strongly on context. Our core arguments address three general issues: (i) the difficulty in extending language-centric explanations to discourse; (ii) the necessity of taking context as a serious topic of study, modeling it formally and acknowledging the limitations on external validity when studying language comprehension outside context; and (iii) the tenuous status of the language network as an explanatory construct. We argue that adopting this framework means that neurobiological studies of language will be less focused on identifying correlations between brain activity patterns and mechanisms postulated by psycholinguistic theories. Instead, they will be less self-referential and increasingly more inclined towards integration of language with other cognitive systems, ultimately doing more justice to the neurobiological organization of language and how it supports language as it is used in everyday life.

Identifying the relationship between oscillatory dynamics and event-related responses

Event related potentials (ERPs) and time frequency analysis of the EEG can identify the temporally distinct coordination of groups of neurons across brain regions during sentence processing. Although there are strong arguments that ERP components and neural oscillations are driven by the same changes in the neural signal, others argue that the lack of clear associations between the two suggests oscillatory dynamics are more than just time frequency representations of ERP components, making it unclear how the two are related. The current study seeks to examine the neural activity underlying auditory sentence processing of both semantic and syntactic errors to clarify if ERP and time frequency analyses identify the same or unique neural responses. Thirty-nine adults completed an auditory semantic judgment task and a grammaticality judgment task. As expected, the semantic judgment task elicited a larger N400 and greater increase in theta power for semantic errors compared to correct sentences and the syntactic judgment task elicited a greater P600 and beta power decrease for both grammatical error types compared to syntactically correct sentences. Importantly, we identified a significant relationship between the N400 and P600 ERPs and theta and beta oscillatory dynamics during semantic and syntactic processing. These findings suggest that ERPs and neural oscillations measure similar neural processes; however, unaccounted for variance may indicate that neural oscillations provide additional information regarding fluctuations in power within a given frequency band. Future studies that vary semantic and syntactic complexity are necessary to understand the cognitive processes that are indexed by these oscillations.

Alignment of alpha-band desynchronization with syntactic structure predicts successful sentence comprehension

Sentence comprehension requires the encoding of phrases and their relationships into working memory. To date, despite the importance of neural oscillations in language comprehension, the neural-oscillatory dynamics of sentence encoding are only sparsely understood. Although oscillations in a wide range of frequency bands have been reported both for the encoding of unstructured word lists and for working-memory intensive sentences, it is unclear to what extent these frequency bands subserve processes specific to the working-memory component of sentence comprehension or to general verbal working memory. In our auditory electroencephalography study, we isolated the working-memory component of sentence comprehension by adapting a subsequent memory paradigm to sentence comprehension and assessing oscillatory power changes during successful sentence encoding. Time-frequency analyses and source reconstruction revealed alpha-power desynchronization in left-hemispheric language-relevant regions during successful sentence encoding. We further showed that sentence encoding was more successful when source-level alpha-band desynchronization aligned with computational measures of syntactic-compared to lexical-semantic-difficulty. Our results are a preliminary indication of a domain-general mechanism of cortical disinhibition via alpha-band desynchronization superimposed onto the language-relevant cortex, which is beneficial for encoding sentences into working memory.

Abnormal language-related oscillatory responses in primary progressive aphasia

Patients with Primary Progressive Aphasia (PPA) may react to linguistic stimuli differently than healthy controls, reflecting degeneration of language networks and engagement of compensatory mechanisms. We used magnetoencephalography (MEG) to evaluate oscillatory neural responses in sentence comprehension, in patients with PPA and age-matched controls. Participants viewed sentences containing semantically and syntactically anomalous words that evoke distinct oscillatory responses. For age-matched controls, semantic anomalies elicited left-lateralized 8–30 Hz power decreases distributed along ventral brain regions, whereas syntactic anomalies elicited bilateral power decreases in both ventral and dorsal regions. In comparison to controls, patients with PPA showed altered patterns of induced oscillations, characterized by delayed latencies and attenuated amplitude, which were correlated with linguistic impairment measured offline. The recruitment of right hemisphere temporo-parietal areas (also found in controls) was correlated with preserved semantic processing abilities, indicating that preserved neural activity in these regions was able to support successful semantic processing. In contrast, syntactic processing was more consistently impaired in PPA, regardless of neural activity patterns, suggesting that this domain of language is particularly vulnerable to the neuronal loss. In addition, we found that delayed peak latencies of oscillatory responses were associated with lower accuracy for detecting semantic anomalies, suggesting that language deficits observed in PPA may be linked to delayed or slowed information processing.

•

Evaluated induced oscillations in patients with PPA using MEG.

•

PPA patients showed delayed latencies and attenuated amplitude of responses.

•

Preserved right hemisphere regions support semantic processing.

•

Delayed latencies of oscillatory responses associated with impaired performance.

•

Language deficits in PPA linked to delayed or slowed information processing.

Oscillatory brain activity differentially reflects false belief understanding and complementation syntax processing

False belief understanding (FBU) enables people to consider conflicting beliefs about the same situation. While language has been demonstrated to be a correlate of FBU, there is still controversy about the extent to which a specific aspect of language, complementation syntax, is a necessary condition for FBU. The present study tested an important notion from the debate proposing that complementation syntax task is redundant to FBU measures. Specifically, we examined electrophysiological correlates of false belief, false complementation, and their respective true conditions in adults using electroencephalography (EEG), focusing on indices of oscillatory brain activity and large-scale connectivity. The results showed strong modulation of parieto-occipital alpha (8-12 Hz) and beta (13-20 Hz) power by the experimental manipulations, with heightened sustained alpha power reflective of effortful internal processing observed in the false compared to the true conditions and reliable beta power reductions sensitive to mentalizing and/or syntactic demands in the belief versus the complementation conditions. In addition, higher coupling between parieto-occipital regions and widespread frontal sites in the beta band was found for the false-belief condition selectively. The result of divergence in beta oscillatory activity and in connectivity between false belief and false complementation does not support the redundancy hypothesis.

Neural oscillatory mechanisms during novel grammar learning underlying language analytical abilities

The goal of the present study was to investigate the initial phases of novel grammar learning on a neural level, concentrating on mechanisms responsible for individual variability between learners. Two groups of participants, one with high and one with average language analytical abilities, performed an Artificial Grammar Learning (AGL) task consisting of learning and test phases. During the task, EEG signals from 32 cap-mounted electrodes were recorded and epochs corresponding to the learning phases were analysed. We investigated spectral power modulations over time, and functional connectivity patterns by means of a bivariate, frequency-specific index of phase synchronization termed Phase Locking Value (PLV). Behavioural data showed learning effects in both groups, with a steeper learning curve and higher ultimate attainment for the highly skilled learners. Moreover, we established that cortical connectivity patterns and profiles of spectral power modulations over time differentiated L2 learners with various levels of language analytical abilities. Over the course of the task, the learning process seemed to be driven by whole-brain functional connectivity between neuronal assemblies achieved by means of communication in the beta band frequency. On a shorter time-scale, increasing proficiency on the AGL task appeared to be supported by stronger local synchronisation within the right hemisphere regions. Finally, we observed that the highly skilled learners might have exerted less mental effort, or reduced attention for the task at hand once the learning was achieved, as evidenced by the higher alpha band power.

The neural oscillations of speech processing and language comprehension: state of the art and emerging mechanisms

Neural oscillations subserve a broad range of functions in speech processing and language comprehension. On the one hand, speech contains-somewhat-repetitive trains of air pressure bursts that occur at three dominant amplitude modulation frequencies, physically marking the linguistically meaningful progressions of phonemes, syllables and intonational phrase boundaries. To these acoustic events, neural oscillations of isomorphous operating frequencies are thought to synchronise, presumably resulting in an implicit temporal alignment of periods of neural excitability to linguistically meaningful spectral information on the three low-level linguistic description levels. On the other hand, speech is a carrier signal that codes for high-level linguistic meaning, such as syntactic structure and semantic information-which cannot be read from stimulus acoustics, but must be acquired during language acquisition and decoded for language comprehension. Neural oscillations subserve the processing of both syntactic structure and semantic information. Here, I synthesise a mapping from each linguistic processing domain to a unique set of subserving oscillatory mechanisms-the mapping is plausible given the role ascribed to different oscillatory mechanisms in different subfunctions of cortical information processing and faithful to the underlying electrophysiology. In sum, the present article provides an accessible and extensive review of the functional mechanisms that neural oscillations subserve in speech processing and language comprehension.

The involvement of alpha oscillations in voluntary attention directed towards encoding episodic memories

Forming episodic memories is often driven by top-down processes of allocating attention towards voluntarily remembering the details of an episode. This attention orientation is needed to make sure that information is encoded for later remembering. Here we designed an episodic long-term memory (LTM) EEG experiment where we examined brain oscillatory activity associated with attention allocation towards the temporal link between an item and its context. The remembering of this temporal conjunction is crucial for item-context binding and hence for the formation of episodic memories. Participants saw a background picture and a word in a central position on a computer screen and were instructed to memorise (a) the picture only, (b) the word, (c) both individually (i.e. ignoring their co-occurrence) and (d) both with them being presented together. Attention allocation towards item-context binding was associated with oscillatory alpha desynchronization in the upper alpha band (10-13 Hz) over dominantly left posterior brain areas. The results highlight the role of alpha desynchronization in voluntary attention allocation towards the temporal conjunction of item and its context in episodic binding and the involvement of posterior brain areas. The pattern of results suggest that they most likely reflect additional visual processes recruited by attentional mechanisms and do not tap into neural processes of item-context binding per se. Moreover, it indicates that the involvement of alpha oscillations in cognitive processes may be more complex.

Increased functional connectivity in the ventral and dorsal streams during retrieval of novel words in professional musicians

Current models of speech and language processing postulate the involvement of two parallel processing streams (the dual stream model): a ventral stream involved in mapping sensory and phonological representations onto lexical and conceptual representations and a dorsal stream contributing to sound-to-motor mapping, articulation, and to how verbal information is encoded and manipulated in memory. Based on previous evidence showing that music training has an influence on language processing, cognitive functions, and word learning, we examined EEG-based intracranial functional connectivity in the ventral and dorsal streams while musicians and nonmusicians learned the meaning of novel words through picture-word associations. In accordance with the dual stream model, word learning was generally associated with increased beta functional connectivity in the ventral stream compared to the dorsal stream. In addition, in the linguistically most demanding "semantic task," musicians outperformed nonmusicians, and this behavioral advantage was accompanied by increased left-hemispheric theta connectivity in both streams. Moreover, theta coherence in the left dorsal pathway was positively correlated with the number of years of music training. These results provide evidence for a complex interplay within a network of brain regions involved in semantic processing and verbal memory functions, and suggest that intensive music training can modify its functional architecture leading to advantages in novel word learning.

Characterizing Neural Entrainment to Hierarchical Linguistic Units using Electroencephalography (EEG)

To understand speech, listeners have to combine the words they hear into phrases and sentences. Recent magnetoencephalography (MEG) and electrocorticography (ECoG) studies show that cortical activity is concurrently entrained/synchronized to the rhythms of multiple levels of linguistic units including words, phrases, and sentences. Here we investigate whether this phenomenon can be observed using electroencephalography (EEG), a technique that is more widely available than MEG and ECoG. We show that the EEG responses concurrently track the rhythms of hierarchical linguistic units such as syllables/words, phrases, and sentences. The strength of the sentential-rate response correlates with how well each subject can detect random words embedded in a sequence of sentences. In contrast, only a syllabic-rate response is observed for an unintelligible control stimulus. In sum, EEG provides a useful tool to characterize neural encoding of hierarchical linguistic units, potentially even in individual participants.

θ-Band and β-Band Neural Activity Reflects Independent Syllable Tracking and Comprehension of Time-Compressed Speech

Recent psychophysics data suggest that speech perception is not limited by the capacity of the auditory system to encode fast acoustic variations through neural γ activity, but rather by the time given to the brain to decode them. Whether the decoding process is bounded by the capacity of θ rhythm to follow syllabic rhythms in speech, or constrained by a more endogenous top-down mechanism, e.g., involving β activity, is unknown. We addressed the dynamics of auditory decoding in speech comprehension by challenging syllable tracking and speech decoding using comprehensible and incomprehensible time-compressed auditory sentences. We recorded EEGs in human participants and found that neural activity in both θ and γ ranges was sensitive to syllabic rate. Phase patterns of slow neural activity consistently followed the syllabic rate (4-14 Hz), even when this rate went beyond the classical θ range (4-8 Hz). The power of θ activity increased linearly with syllabic rate but showed no sensitivity to comprehension. Conversely, the power of β (14-21 Hz) activity was insensitive to the syllabic rate, yet reflected comprehension on a single-trial basis. We found different long-range dynamics for θ and β activity, with β activity building up in time while more contextual information becomes available. This is consistent with the roles of θ and β activity in stimulus-driven versus endogenous mechanisms. These data show that speech comprehension is constrained by concurrent stimulus-driven θ and low-γ activity, and by endogenous β activity, but not primarily by the capacity of θ activity to track the syllabic rhythm.SIGNIFICANCE STATEMENT Speech comprehension partly depends on the ability of the auditory cortex to track syllable boundaries with θ-range neural oscillations. The reason comprehension drops when speech is accelerated could hence be because θ oscillations can no longer follow the syllabic rate. Here, we presented subjects with comprehensible and incomprehensible accelerated speech, and show that neural phase patterns in the θ band consistently reflect the syllabic rate, even when speech becomes too fast to be intelligible. The drop in comprehension, however, is signaled by a significant decrease in the power of low-β oscillations (14-21 Hz). These data suggest that speech comprehension is not limited by the capacity of θ oscillations to adapt to syllabic rate, but by an endogenous decoding process.

Domain-general neural correlates of dependency formation: Using complex tones to simulate language

There is an ongoing debate whether the P600 event-related potential component following syntactic anomalies reflects syntactic processes per se, or if it is an instance of the P300, a domain-general ERP component associated with attention and cognitive reorientation. A direct comparison of both components is challenging because of the huge discrepancy in experimental designs and stimulus choice between language and 'classic' P300 experiments. In the present study, we develop a new approach to mimic the interplay of sequential position as well as categorical and relational information in natural language syntax (word category and agreement) in a non-linguistic target detection paradigm using musical instruments. Participants were instructed to (covertly) detect target tones which were defined by instrument change and pitch rise between subsequent tones at the last two positions of four-tone sequences. We analysed the EEG using event-related averaging and time-frequency decomposition. Our results show striking similarities to results obtained from linguistic experiments. We found a P300 that showed sensitivity to sequential position and a late positivity sensitive to stimulus type and position. A time-frequency decomposition revealed significant effects of sequential position on the theta band and a significant influence of stimulus type on the delta band. Our results suggest that the detection of non-linguistic targets defined via complex feature conjunctions in the present study and the detection of syntactic anomalies share the same underlying processes: attentional shift and memory based matching processes that act upon multi-feature conjunctions. We discuss the results as supporting domain-general accounts of the P600 during natural language comprehension.

EEG beta-power changes reflect motor involvement in abstract action language processing

Brain oscillations in the α- and β-range become suppressed during motor processing and motor imagery. It has recently been discussed that such power changes also occur during action language processing. In our study, we compared β₂-oscillations (16-25Hz) during the observation of prototypical arm movements (revealed via motion tracking) as well as during semantic processing of concrete and abstract sentences containing arm-related action verbs. Whereas we did find a strong desynchronization in the β₂-range during action observation, the processing of action sentences evoked a rather weak desynchronization. However, this desynchronization occurred for action verbs in both concrete and abstract contexts. These results might indicate a tendency for abstract action language to be processed similar to concrete action language rather than abstract sentences. The oscillation patterns reflect the close relationship between language comprehension and motor functions - one of the core claims of current theories on embodied cognition.

Neurophysiological dynamics of phrase-structure building during sentence processing

Although sentences unfold sequentially, one word at a time, most linguistic theories propose that their underlying syntactic structure involves a tree of nested phrases rather than a linear sequence of words. Whether and how the brain builds such structures, however, remains largely unknown. Here, we used human intracranial recordings and visual word-by-word presentation of sentences and word lists to investigate how left-hemispheric brain activity varies during the formation of phrase structures. In a broad set of language-related areas, comprising multiple superior temporal and inferior frontal sites, high-gamma power increased with each successive word in a sentence but decreased suddenly whenever words could be merged into a phrase. Regression analyses showed that each additional word or multiword phrase contributed a similar amount of additional brain activity, providing evidence for a merge operation that applies equally to linguistic objects of arbitrary complexity. More superficial models of language, based solely on sequential transition probability over lexical and syntactic categories, only captured activity in the posterior middle temporal gyrus. Formal model comparison indicated that the model of multiword phrase construction provided a better fit than probability-based models at most sites in superior temporal and inferior frontal cortices. Activity in those regions was consistent with a neural implementation of a bottom-up or left-corner parser of the incoming language stream. Our results provide initial intracranial evidence for the neurophysiological reality of the merge operation postulated by linguists and suggest that the brain compresses syntactically well-formed sequences of words into a hierarchy of nested phrases.

Oscillatory EEG dynamics underlying automatic chunking during sentence processing

Sentences are easier to remember than random word sequences, likely because linguistic regularities facilitate chunking of words into meaningful groups. The present electroencephalography study investigated the neural oscillations modulated by this so-called sentence superiority effect during the encoding and maintenance of sentence fragments versus word lists. We hypothesized a chunking-related modulation of neural processing during the encoding and retention of sentences (i.e., sentence fragments) as compared to word lists. Time-frequency analysis revealed a two-fold oscillatory pattern for the memorization of sentences: Sentence encoding was accompanied by higher delta amplitude (4Hz), originating both from regions processing syntax as well as semantics (bilateral superior/middle temporal regions and fusiform gyrus). Subsequent sentence retention was reflected in decreased theta (6Hz) and beta/gamma (27-32Hz) amplitude instead. Notably, whether participants simply read or properly memorized the sentences did not impact chunking-related activity during encoding. Therefore, we argue that the sentence superiority effect is grounded in highly automatized language processing mechanisms, which generate meaningful memory chunks irrespective of task demands.

Alpha and theta band dynamics related to sentential constraint and word expectancy

Despite strong evidence for prediction during language comprehension, the underlying mechanisms, and the extent to which they are specific to language, remain unclear. Re-analyzing an ERP study, we examined responses in the time-frequency domain to expected and unexpected (but plausible) words in strongly and weakly constraining sentences, and found results similar to those reported in nonverbal domains. Relative to expected words, unexpected words elicited an increase in the theta band (4-7 Hz) in strongly constraining contexts, suggesting the involvement of control processes to deal with the consequences of having a prediction disconfirmed. Prior to critical word onset, strongly constraining sentences exhibited a decrease in the alpha band (8-12 Hz) relative to weakly constraining sentences, suggesting that comprehenders can take advantage of predictive sentence contexts to prepare for the input. The results suggest that the brain recruits domain-general preparation and control mechanisms when making and assessing predictions during sentence comprehension.

Oscillatory Brain Dynamics during Sentence Reading: A Fixation-Related Spectral Perturbation Analysis

The present study investigated oscillatory brain dynamics during self-paced sentence-level processing. Participants read fully correct sentences, sentences containing a semantic violation and "sentences" in which the order of the words was randomized. At the target word level, fixations on semantically unrelated words elicited a lower-beta band (13-18 Hz) desynchronization. At the sentence level, gamma power (31-55 Hz) increased linearly for syntactically correct sentences, but not when the order of the words was randomized. In the 300-900 ms time window after sentence onsets, theta power (4-7 Hz) was greater for syntactically correct sentences as compared to sentences where no syntactic structure was preserved (random words condition). We interpret our results as conforming with a recently formulated predictive-coding framework for oscillatory neural dynamics during sentence-level language comprehension. Additionally, we discuss how our results relate to previous findings with serial visual presentation vs. self-paced reading.

Functional reorganization of language networks for semantics and syntax in chronic stroke: Evidence from MEG

Using magnetoencephalography, we investigated the potential of perilesional and contralesional activity to support language recovery in patients with poststroke aphasia. In healthy young controls, left-lateralized ventral frontotemporal regions responded to semantic anomalies during sentence comprehension and bilateral dorsal frontoparietal regions responded to syntactic anomalies. Older adults showed more extensive bilateral responses to the syntactic anomalies and less lateralized responses to the semantic anomalies, with decreased activation in the left occipital and parietal regions for both semantic and syntactic anomalies. In aphasic participants, we observed compensatory recruitment in the right hemisphere (RH), which varied depending on the type of linguistic information that was processed. For semantic anomalies, aphasic patients activated some preserved left hemisphere regions adjacent to the lesion, as well as homologous parietal and temporal RH areas. Patients also recruited right inferior and dorsolateral frontal cortex that was not activated in the healthy participants. Responses for syntactic anomalies did not reach significance in patients. Correlation analyses indicated that recruitment of homologous temporoparietal RH areas is associated with better semantic performance, whereas higher accuracy on the syntactic task was related to bilateral superior temporoparietal and right frontal activity. The results suggest that better recovery of semantic processing is associated with a shift to ventral brain regions in the RH. In contrast, preservation of syntactic processing is mediated by dorsal areas, bilaterally, although recovery of syntactic processing tends to be poorer than semantic. Hum Brain Mapp 37:2869-2893, 2016. © 2016 Wiley Periodicals, Inc.

Neural activity during sentence processing as reflected in theta, alpha, beta, and gamma oscillations

We used magnetoencephalography (MEG) to explore the spatiotemporal dynamics of neural oscillations associated with sentence processing in 102 participants. We quantified changes in oscillatory power as the sentence unfolded, and in response to individual words in the sentence. For words early in a sentence compared to those late in the same sentence, we observed differences in left temporal and frontal areas, and bilateral frontal and right parietal regions for the theta, alpha, and beta frequency bands. The neural response to words in a sentence differed from the response to words in scrambled sentences in left-lateralized theta, alpha, beta, and gamma. The theta band effects suggest that a sentential context facilitates lexical retrieval, and that this facilitation is stronger for words late in the sentence. Effects in the alpha and beta bands may reflect the unification of semantic and syntactic information, and are suggestive of easier unification late in a sentence. The gamma oscillations are indicative of predicting the upcoming word during sentence processing. In conclusion, changes in oscillatory neuronal activity capture aspects of sentence processing. Our results support earlier claims that language (sentence) processing recruits areas distributed across both hemispheres, and extends beyond the classical language regions.

A Predictive Coding Perspective on Beta Oscillations during Sentence-Level Language Comprehension

Oscillatory neural dynamics have been steadily receiving more attention as a robust and temporally precise signature of network activity related to language processing. We have recently proposed that oscillatory dynamics in the beta and gamma frequency ranges measured during sentence-level comprehension might be best explained from a predictive coding perspective. Under our proposal we related beta oscillations to both the maintenance/change of the neural network configuration responsible for the construction and representation of sentence-level meaning, and to top–down predictions about upcoming linguistic input based on that sentence-level meaning. Here we zoom in on these particular aspects of our proposal, and discuss both old and new supporting evidence. Finally, we present some preliminary magnetoencephalography data from an experiment comparing Dutch subject- and object-relative clauses that was specifically designed to test our predictive coding framework. Initial results support the first of the two suggested roles for beta oscillations in sentence-level language comprehension.

Developmental differences in beta and theta power during sentence processing

•

Changes in ERPs and oscillatory dynamic occur during auditory sentence processing.

•

Adults are significantly better at identifying syntactic errors compared to children.

•

Adults display a significant P600 effect and theta/beta power decrease.

•

Children display a significant N400 effect and smaller decrease in theta/beta power.

•

These findings suggest syntactic processing skills are still developing by age 12.

Although very young children process ongoing language quickly and effortlessly, research indicates that they continue to improve and mature in their language skills through adolescence. This prolonged development may be related to differing engagement of semantic and syntactic processes. This study used event related potentials and time frequency analysis of EEG to identify developmental differences in neural engagement as children (ages 10–12) and adults performed an auditory verb agreement grammaticality judgment task. Adults and children revealed very few differences in comprehending grammatically correct sentences. When identifying grammatical errors, however, adults displayed widely distributed beta and theta power decreases that were significantly less pronounced in children. Adults also demonstrated a significant P600 effect, while children exhibited an apparent N400 effect. Thus, when identifying subtle grammatical errors in real time, adults display greater neural activation that is traditionally associated with syntactic processing whereas children exhibit greater activity more commonly associated with semantic processing. These findings support previous claims that the cognitive and neural underpinnings of syntactic processing are still developing in adolescence, and add to them by more clearly identifying developmental changes in the neural oscillations underlying grammatical processing.

Cortical tracking of hierarchical linguistic structures in connected speech

The most critical attribute of human language is its unbounded combinatorial nature: smaller elements can be combined into larger structures on the basis of a grammatical system, resulting in a hierarchy of linguistic units, such as words, phrases and sentences. Mentally parsing and representing such structures, however, poses challenges for speech comprehension. In speech, hierarchical linguistic structures do not have boundaries that are clearly defined by acoustic cues and must therefore be internally and incrementally constructed during comprehension. We found that, during listening to connected speech, cortical activity of different timescales concurrently tracked the time course of abstract linguistic structures at different hierarchical levels, such as words, phrases and sentences. Notably, the neural tracking of hierarchical linguistic structures was dissociated from the encoding of acoustic cues and from the predictability of incoming words. Our results indicate that a hierarchy of neural processing timescales underlies grammar-based internal construction of hierarchical linguistic structure.

Sensitivity to Referential Ambiguity in Discourse: The Role of Attention, Working Memory, and Verbal Ability

The establishment of reference is essential to language comprehension. The goal of this study was to examine listeners' sensitivity to referential ambiguity as a function of individual variation in attention, working memory capacity, and verbal ability. Participants listened to stories in which two entities were introduced that were either very similar (e.g., two oaks) or less similar (e.g., one oak and one elm). The manipulation rendered an anaphor in a subsequent sentence (e.g., oak) ambiguous or unambiguous. EEG was recorded as listeners comprehended the story, after which participants completed tasks to assess working memory, verbal ability, and the ability to use context in task performance. Power in the alpha and theta frequency bands when listeners received critical information about the discourse entities (e.g., oaks) was used to index attention and the involvement of the working memory system in processing the entities. These measures were then used to predict an ERP component that is sensitive to referential ambiguity, the Nref, which was recorded when listeners received the anaphor. Nref amplitude at the anaphor was predicted by alpha power during the earlier critical sentence: Individuals with increased alpha power in ambiguous compared with unambiguous stories were less sensitive to the anaphor's ambiguity. Verbal ability was also predictive of greater sensitivity to referential ambiguity. Finally, increased theta power in the ambiguous compared with unambiguous condition was associated with higher working-memory span. These results highlight the role of attention and working memory in referential processing during listening comprehension.

Fast oscillatory dynamics during language comprehension: Unification versus maintenance and prediction?

The role of neuronal oscillations during language comprehension is not yet well understood. In this paper we review and reinterpret the functional roles of beta- and gamma-band oscillatory activity during language comprehension at the sentence and discourse level. We discuss the evidence in favor of a role for beta and gamma in unification (the unification hypothesis), and in light of mounting evidence that cannot be accounted for under this hypothesis, we explore an alternative proposal linking beta and gamma oscillations to maintenance and prediction (respectively) during language comprehension. Our maintenance/prediction hypothesis is able to account for most of the findings that are currently available relating beta and gamma oscillations to language comprehension, and is in good agreement with other proposals about the roles of beta and gamma in domain-general cognitive processing. In conclusion we discuss proposals for further testing and comparing the prediction and unification hypotheses.

Frontal-posterior theta oscillations reflect memory retrieval during sentence comprehension

Successful working-memory retrieval requires that items be retained as distinct units. At the neural level, it has been shown that theta-band oscillatory power increases with the number of to-be-distinguished items during working-memory retrieval. Here we hypothesized that during sentence comprehension, verbal-working-memory retrieval demands lead to increased theta power over frontal cortex, supposedly supporting the distinction amongst stored items during verbal-working-memory retrieval. Also, synchronicity may increase between the frontal cortex and the posterior cortex, with the latter supposedly supporting item retention. We operationalized retrieval by using pronouns, which refer to and trigger the retrieval of antecedent nouns from a preceding sentence part. Retrieval demand was systematically varied by changing the pronoun antecedent: Either, it was non-embedded in the preceding main clause, and thus easy-to-retrieve across a single clause boundary, or embedded in the preceding subordinate clause, and thus hard-to-retrieve across a double clause boundary. We combined electroencephalography (EEG), scalp-level time-frequency analysis, source localization, and source-level coherence analysis, observing a frontal-midline and broad left-hemispheric theta-power increase for embedded-antecedent compared to non-embedded-antecedent retrieval. Sources were localized to left-frontal, left-parietal, and bilateral-inferior-temporal cortices. Coherence analyses suggested synchronicity between left-frontal and left-parietal and between left-frontal and right-inferior-temporal cortices. Activity of an array of left-frontal, left-parietal, and bilateral-inferior-temporal cortices may thus assist retrieval during sentence comprehension, potentially indexing the orchestration of item distinction, verbal working memory, and long-term memory. Our results extend prior findings by mapping prior knowledge on the functional role of theta oscillations onto processes genuine to human sentence comprehension.

Frequency-based Segregation of Syntactic and Semantic Unification during Online Sentence Level Language Comprehension

During sentence level language comprehension, semantic and syntactic unification are functionally distinct operations. Nevertheless, both recruit roughly the same brain areas (spatially overlapping networks in the left frontotemporal cortex) and happen at the same time (in the first few hundred milliseconds after word onset). We tested the hypothesis that semantic and syntactic unification are segregated by means of neuronal synchronization of the functionally relevant networks in different frequency ranges: gamma (40 Hz and up) for semantic unification and lower beta (10-20 Hz) for syntactic unification. EEG power changes were quantified as participants read either correct sentences, syntactically correct though meaningless sentences (syntactic prose), or sentences that did not contain any syntactic structure (random word lists). Other sentences contained either a semantic anomaly or a syntactic violation at a critical word in the sentence. Larger EEG gamma-band power was observed for semantically coherent than for semantically anomalous sentences. Similarly, beta-band power was larger for syntactically correct sentences than for incorrect ones. These results confirm the existence of a functional dissociation in EEG oscillatory dynamics during sentence level language comprehension that is compatible with the notion of a frequency-based segregation of syntactic and semantic unification.

Putative mechanisms mediating tolerance for audiovisual stimulus onset asynchrony

Audiovisual (AV) speech perception is robust to temporal asynchronies between visual and auditory stimuli. We investigated the neural mechanisms that facilitate tolerance for audiovisual stimulus onset asynchrony (AVOA) with EEG. Individuals were presented with AV words that were asynchronous in onsets of voice and mouth movement and judged whether they were synchronous or not. Behaviorally, individuals tolerated (perceived as synchronous) longer AVOAs when mouth movement preceded the speech (V-A) stimuli than when the speech preceded mouth movement (A-V). Neurophysiologically, the P1-N1-P2 auditory evoked potentials (AEPs), time-locked to sound onsets and known to arise in and surrounding the primary auditory cortex (PAC), were smaller for the in-sync than the out-of-sync percepts. Spectral power of oscillatory activity in the beta band (14–30 Hz) following the AEPs was larger during the in-sync than out-of-sync perception for both A-V and V-A conditions. However, alpha power (8–14 Hz), also following AEPs, was larger for the in-sync than out-of-sync percepts only in the V-A condition. These results demonstrate that AVOA tolerance is enhanced by inhibiting low-level auditory activity (e.g., AEPs representing generators in and surrounding PAC) that code for acoustic onsets. By reducing sensitivity to acoustic onsets, visual-to-auditory onset mapping is weakened, allowing for greater AVOA tolerance. In contrast, beta and alpha results suggest the involvement of higher-level neural processes that may code for language cues (phonetic, lexical), selective attention, and binding of AV percepts, allowing for wider neural windows of temporal integration, i.e., greater AVOA tolerance.

Complex brain network properties in late L2 learners and native speakers

Whether the neural mechanisms that underlie the processing of a second language in highly proficient late bilinguals (L2 late learners) are similar or not to those that underlie the processing of the first language (L1) is still an issue under debate. In this study, a group of late learners of Spanish whose native language is English and a group of Spanish monolinguals were compared while they read sentences, some of which contained syntactic violations. A brain complex network analysis approach was used to assess the time-varying topological properties of the functional networks extracted from the electroencephalography (EEG) recording. Late L2 learners showed a lower degree of parallel information transfer and a slower propagation between regions of the brain functional networks while processing sentences containing a gender mismatch condition as compared with a standard sentence configuration. In contrast, no such differences between these conditions were detected in the Spanish monolinguals. This indicates that when a morphosyntactic language incongruence that does not exist in the native language is presented in the second language, the neural activation pattern is configured differently in highly proficient late bilinguals than in monolinguals.

Early and late beta-band power reflect audiovisual perception in the McGurk illusion

The McGurk illusion is a prominent example of audiovisual speech perception and the influence that visual stimuli can have on auditory perception. In this illusion, a visual speech stimulus influences the perception of an incongruent auditory stimulus, resulting in a fused novel percept. In this high-density electroencephalography (EEG) study, we were interested in the neural signatures of the subjective percept of the McGurk illusion as a phenomenon of speech-specific multisensory integration. Therefore, we examined the role of cortical oscillations and event-related responses in the perception of congruent and incongruent audiovisual speech. We compared the cortical activity elicited by objectively congruent syllables with incongruent audiovisual stimuli. Importantly, the latter elicited a subjectively congruent percept: the McGurk illusion. We found that early event-related responses (N1) to audiovisual stimuli were reduced during the perception of the McGurk illusion compared with congruent stimuli. Most interestingly, our study showed a stronger poststimulus suppression of beta-band power (13-30 Hz) at short (0-500 ms) and long (500-800 ms) latencies during the perception of the McGurk illusion compared with congruent stimuli. Our study demonstrates that auditory perception is influenced by visual context and that the subsequent formation of a McGurk illusion requires stronger audiovisual integration even at early processing stages. Our results provide evidence that beta-band suppression at early stages reflects stronger stimulus processing in the McGurk illusion. Moreover, stronger late beta-band suppression in McGurk illusion indicates the resolution of incongruent physical audiovisual input and the formation of a coherent, illusory multisensory percept.

Localization of electrophysiological responses to semantic and syntactic anomalies in language comprehension with MEG

Syntactically and semantically anomalous words encountered during sentence comprehension are known to elicit dissociable electrophysiological responses, which are thought to reflect distinct aspects of language processing. However, the sources of these responses have not been well characterized. We used beamforming analysis of magnetoencephalography (MEG) data to map generators of electrophysiological responses to linguistic anomalies. Anomalous words occurred in the context of a sentence acceptability judgement task conducted in both visual and auditory modalities. Time-frequency analysis revealed that both kinds of violations elicited event-related synchronization (ERS) in the delta-theta frequency range (1-5 Hz), and desynchronization (ERD) in the alpha-beta range (8-30 Hz). In addition, these responses were differentially modulated by violation type and presentation modality. 1-5 Hz responses were consistently localized within medial prefrontal cortex and did not vary significantly across violation types, but were stronger for visual presentation. In contrast, 8-30 Hz ERD occurred in different regions for different violation types. For semantic violations the distribution was predominantly in the bilateral occipital cortex and left temporal and inferior frontal regions, and these effects did not differ for visual and auditory presentation. In contrast, syntactic responses were strongly affected by presentation modality. Under visual presentation, syntactic violations elicited bilateral 8-30 Hz ERD extending into dorsal parietal and frontal regions, whereas effects were much weaker and mostly statistically insignificant in the auditory modality. These results suggest that delta-theta ERS reflects generalized increases in working memory demands related to linguistic anomaly detection, while alpha-beta ERD reflects specific activation of cortical regions involved in distinct aspects of linguistic processing, such as semantic vs. phonological short-term memory. Beamforming analysis of time-domain average signals (ERFs) revealed an N400m effect for semantic anomalies in both modalities, localized to left superior temporal and posterior frontal regions, and a later P600-like effect for syntactic anomalies in both modalities, widespread over bilateral frontal, posterior temporal, and parietal regions. These results indicate that time-domain averaged responses and induced oscillatory responses have distinct properties, including localization and modality dependence, and likely reflect dissociable and complementary aspects of neural activity related to language comprehension and additional task-related processes.

Between Thought and Expression, a Magnetoencephalography Study of the “Tip-of-the-Tongue” Phenomenon

“Tip-of-the-tongue” (TOT) is the phenomenon associated with the inaccessibility of a known word from memory. It is universally experienced, increases in frequency with age, and is most common for proper nouns. It is a good model for the symptom of anomia experienced much more frequently by some aphasic patients following brain injury. Here, we induced the TOT state in older participants while they underwent brain scanning with magnetoencephalography to investigate the changes in oscillatory brain activity associated with failed retrieval of known words. Using confrontation naming of pictures of celebrities, we successfully induced the TOT state in 29% of trials and contrasted it with two other states: “Know” where the participants both correctly recognized the celebrity's face and retrieved their name and “Don't Know” when the participants did not recognize the celebrity. We wished to test Levelt's influential model of speech output by carrying out two analyses, one epoching the data to the point in time when the picture was displayed and the other looking back in time from when the participants first articulated their responses. Our main findings supported the components of Levelt's model, but not their serial activation over time as both semantic and motor areas were identified in both analyses. We also found enduring decreases in the alpha frequency band in the left ventral temporal region during the TOT state, suggesting ongoing semantic search. Finally, we identified reduced beta power in classical peri-sylvian language areas for the TOT condition, suggesting that brain regions that encode linguistic memories are also involved in their attempted retrieval.

Early anticipation lies behind the speed of response in conversation

RTs in conversation, with average gaps of 200 msec and often less, beat standard RTs, despite the complexity of response and the lag in speech production (600 msec or more). This can only be achieved by anticipation of timing and content of turns in conversation, about which little is known. Using EEG and an experimental task with conversational stimuli, we show that estimation of turn durations are based on anticipating the way the turn would be completed. We found a neuronal correlate of turn-end anticipation localized in ACC and inferior parietal lobule, namely a beta-frequency desynchronization as early as 1250 msec, before the end of the turn. We suggest that anticipation of the other's utterance leads to accurately timed transitions in everyday conversations.

Oscillatory responses to semantic and syntactic violations

EEG studies employing time-frequency analysis have revealed changes in theta and alpha power in a variety of language and memory tasks. Semantic and syntactic violations embedded in sentences evoke well-known ERPs, but little is known about the oscillatory responses to these violations. We investigated oscillatory responses to both kinds of violations, while monolingual and bilingual participants performed an acceptability judgment task. Both violations elicited power decreases (event-related desynchronization, ERD) in the 8-30 Hz frequency range, but with different scalp topographies. In addition, semantic anomalies elicited power increases (event-related synchronization, ERS) in the 1-5 Hz frequency band. The 1-5 Hz ERS was strongly phase-locked to stimulus onset and highly correlated with time domain averages, whereas the 8-30 Hz ERD response varied independently of these. In addition, the results showed that language expertise modulated 8-30 Hz ERD for syntactic violations as a function of the executive demands of the task. When the executive function demands were increased using a grammaticality judgment task, bilinguals but not monolinguals demonstrated reduced 8-30 Hz ERD for syntactic violations. These findings suggest a putative role of the 8-30 Hz ERD response as a marker of linguistic processing that likely represents a separate neural process from those underlying ERPs.

The relation between thematic role computing and semantic relatedness processing during on-line sentence comprehension

Sentence comprehension involves timely computing different types of relations between its verbs and noun arguments, such as morphosyntactic, semantic, and thematic relations. Here, we used EEG technique to investigate the potential differences in thematic role computing and lexical-semantic relatedness processing during on-line sentence comprehension, and the interaction between these two types of processes. Mandarin Chinese sentences were used as materials. The basic structure of those sentences is “Noun+Verb+‘le’+a two-character word”, with the Noun being the initial argument. The verb disambiguates the initial argument as an agent or a patient. Meanwhile, the initial argument and the verb are highly or lowly semantically related. The ERPs at the verbs revealed that: relative to the agent condition, the patient condition evoked a larger N400 only when the argument and verb were lowly semantically related; however, relative to the high-relatedness condition, the low-relatedness condition elicited a larger N400 regardless of the thematic relation; although both thematic role variation and semantic relatedness variation elicited N400 effects, the N400 effect elicited by the former was broadly distributed and reached maximum over the frontal electrodes, and the N400 effect elicited by the latter had a posterior distribution. In addition, the brain oscillations results showed that, although thematic role variation (patient vs. agent) induced power decreases around the beta frequency band (15–30 Hz), semantic relatedness variation (low-relatedness vs. high-relatedness) induced power increases in the theta frequency band (4–7 Hz). These results suggested that, in the sentence context, thematic role computing is modulated by the semantic relatedness between the verb and its argument; semantic relatedness processing, however, is in some degree independent from the thematic relations. Moreover, our results indicated that, during on-line sentence comprehension, thematic role computing and semantic relatedness processing are mediated by distinct neural systems.

Temporal sampling in vision and the implications for dyslexia

It has recently been suggested that dyslexia may manifest as a deficit in the neural synchrony underlying language-based codes (Goswami, 2011), such that the phonological deficits apparent in dyslexia occur as a consequence of poor synchronisation of oscillatory brain signals to the sounds of language. There is compelling evidence to support this suggestion, and it provides an intriguing new development in understanding the aetiology of dyslexia. It is undeniable that dyslexia is associated with poor phonological coding, however, reading is also a visual task, and dyslexia has also been associated with poor visual coding, particularly visuo-spatial sensitivity. It has been hypothesized for some time that specific frequency oscillations underlie visual perception. Although little research has been done looking specifically at dyslexia and cortical frequency oscillations, it is possible to draw on converging evidence from visual tasks to speculate that similar deficits could occur in temporal frequency oscillations in the visual domain in dyslexia. Thus, here the plausibility of a visual correlate of the Temporal Sampling Framework is considered, leading to specific hypotheses and predictions for future research. A common underlying neural mechanism in dyslexia, may subsume qualitatively different manifestations of reading difficulty, which is consistent with the heterogeneity of the disorder, and may open the door for a new generation of exciting research.

Attention to memory: orienting attention to sound object representations

Despite a growing acceptance that attention and memory interact, and that attention can be focused on an active internal mental representation (i.e., reflective attention), there has been a paucity of work focusing on reflective attention to 'sound objects' (i.e., mental representations of actual sound sources in the environment). Further research on the dynamic interactions between auditory attention and memory, as well as its degree of neuroplasticity, is important for understanding how sound objects are represented, maintained, and accessed in the brain. This knowledge can then guide the development of training programs to help individuals with attention and memory problems. This review article focuses on attention to memory with an emphasis on behavioral and neuroimaging studies that have begun to explore the mechanisms that mediate reflective attentional orienting in vision and more recently, in audition. Reflective attention refers to situations in which attention is oriented toward internal representations rather than focused on external stimuli. We propose four general principles underlying attention to short-term memory. Furthermore, we suggest that mechanisms involved in orienting attention to visual object representations may also apply for orienting attention to sound object representations.