Oscillatory neuronal dynamics during language comprehension

Hierarchical syntax model of music predicts theta power during music listening

Linguistic research showed that the depth of syntactic embedding is reflected in brain theta power. Here, we test whether this also extends to non-linguistic stimuli, specifically music. We used a hierarchical model of musical syntax to continuously quantify two types of expert-annotated harmonic dependencies throughout a piece of Western classical music: prolongation and preparation. Prolongations can roughly be understood as a musical analogue to linguistic coordination between constituents that share the same function (e.g., 'pizza' and 'pasta' in 'I ate pizza and pasta'). Preparation refers to the dependency between two harmonies whereby the first implies a resolution towards the second (e.g., dominant towards tonic; similar to how the adjective implies the presence of a noun in 'I like spicy … '). Source reconstructed MEG data of sixty-five participants listening to the musical piece was then analysed. We used Bayesian Mixed Effects models to predict theta envelope in the brain, using the number of open prolongation and preparation dependencies as predictors whilst controlling for audio envelope. We observed that prolongation and preparation both carry independent and distinguishable predictive value for theta band fluctuation in key linguistic areas such as the Angular, Superior Temporal, and Heschl's Gyri, or their right-lateralised homologues, with preparation showing additional predictive value for areas associated with the reward system and prediction. Musical expertise further mediated these effects in language-related brain areas. Results show that predictions of precisely formalised music-theoretical models are reflected in the brain activity of listeners which furthers our understanding of the perception and cognition of musical structure.

Neural activity related to productive vocabulary knowledge effects during second language comprehension

Second language learners and educators often believe that improving one's listening ability hinges on acquiring an extensive vocabulary and engaging in thorough listening practice. Our previous study suggested that listening comprehension is also impacted by the ability to produce vocabulary. Nevertheless, it remained uncertain whether quick comprehension could be attributed to a simple acceleration of processing or to changes in neural activity. To identify neural activity changes during sentence listening comprehension according to different levels of lexical knowledge (productive, only comprehensive, uncomprehensive), we measured participants' electrical activity in the brain via electroencephalography (EEG) and conducted a time-frequency-based EEG power analysis. Additionally, we employed a decoding model to verify the predictability of vocabulary knowledge levels based on neural activity. The decoding results showed that EEG activity could discriminate between listening to sentences containing phrases that include productive knowledge and ones without. The positive impact of productive vocabulary knowledge on sentence comprehension, driven by distinctive neural processing during sentence comprehension, was unequivocally evident. Our study emphasizes the importance of productive vocabulary knowledge acquisition to enhance the process of second language listening comprehension.

Detection of Language Lateralization Using Spectral Analysis of EEG

PURPOSE

Language lateralization relies on expensive equipment and can be difficult to tolerate. We assessed if lateralized brain responses to a language task can be detected with spectral analysis of electroencephalography (EEG).

METHODS

Twenty right-handed, neurotypical adults (28 ± 10 years; five males) performed a verb generation task and two control tasks (word listening and repetition). We measured changes in EEG activity elicited by tasks (the event-related spectral perturbation [ERSP]) in the theta, alpha, beta, and gamma frequency bands in two language (superior temporal and inferior frontal [ST and IF]) and one control (occipital [Occ]) region bilaterally. We tested whether language tasks elicited (1) changes in spectral power from baseline (significant ERSP) at any region or (2) asymmetric ERSPs between matched left and right regions.

RESULTS

Left IF beta power (-0.37±0.53, t = -3.12, P = 0.006) and gamma power in all regions decreased during verb generation. Asymmetric ERSPs (right > left) occurred between the (1) IF regions in the beta band (right vs. left difference of 0.23±0.37, t(19) = -2.80, P = 0.0114) and (2) ST regions in the alpha band (right vs. left difference of 0.48±0.63, t(19) = -3.36, P = 0.003). No changes from baseline or hemispheric asymmetries were noted in language regions during control tasks. On the individual level, 16 (80%) participants showed decreased left IF beta power from baseline, and 16 showed ST alpha asymmetry. Eighteen participants (90%) showed one of these two findings.

CONCLUSIONS

Spectral EEG analysis detects lateralized responses during language tasks in frontal and temporal regions. Spectral EEG analysis could be developed into a readily available language lateralization modality.

Neural consequences of binaural beat stimulation on auditory sentence comprehension: an EEG study

A growing literature has shown that binaural beat (BB)-generated by dichotic presentation of slightly mismatched pure tones-improves cognition. We recently found that BB stimulation of either beta (18 Hz) or gamma (40 Hz) frequencies enhanced auditory sentence comprehension. Here, we used electroencephalography (EEG) to characterize neural oscillations pertaining to the enhanced linguistic operations following BB stimulation. Sixty healthy young adults were randomly assigned to one of three listening groups: 18-Hz BB, 40-Hz BB, or pure-tone baseline, all embedded in music. After listening to the sound for 10 min (stimulation phase), participants underwent an auditory sentence comprehension task involving spoken sentences that contained either an object or subject relative clause (task phase). During the stimulation phase, 18-Hz BB yielded increased EEG power in a beta frequency range, while 40-Hz BB did not. During the task phase, only the 18-Hz BB resulted in significantly higher accuracy and faster response times compared with the baseline, especially on syntactically more complex object-relative sentences. The behavioral improvement by 18-Hz BB was accompanied by attenuated beta power difference between object- and subject-relative sentences. Altogether, our findings demonstrate beta oscillations as a neural correlate of improved syntactic operation following BB stimulation.

Development of Gamma Oscillation during Sentence Processing in Early Adolescence: Insights into the Maturation of Semantic Processing

Children's ability to retrieve word meanings and incorporate them into sentences, along with the neural structures that support these skills, continues to evolve throughout adolescence. Theta (4-8 Hz) activity that corresponds to word retrieval in children decreases in power and becomes more localized with age. This bottom-up word retrieval is often paired with changes in gamma (31-70 Hz), which are thought to reflect semantic unification in adults. Here, we studied gamma engagement during sentence processing using EEG time-frequency in children (ages 8-15) to unravel the developmental trajectory of the gamma network during sentence processing. Children heavily rely on semantic integration for sentence comprehension, but as they mature, semantic and syntactic processing units become distinct and localized. We observed a similar developmental shift in gamma oscillation around age 11, with younger groups (8-9 and 10-11) exhibiting broadly distributed gamma activity with higher amplitudes, while older groups (12-13 and 14-15) exhibited smaller and more localized gamma activity, especially over the left central and posterior regions. We interpret these findings as support for the argument that younger children rely more heavily on semantic processes for sentence comprehension than older children. And like adults, semantic processing in children is associated with gamma activity.

Neural oscillations and speech processing at birth

Are neural oscillations biologically endowed building blocks of the neural architecture for speech processing from birth, or do they require experience to emerge? In adults, delta, theta, and low-gamma oscillations support the simultaneous processing of phrasal, syllabic, and phonemic units in the speech signal, respectively. Using electroencephalography to investigate neural oscillations in the newborn brain we reveal that delta and theta oscillations differ for rhythmically different languages, suggesting that these bands underlie newborns' universal ability to discriminate languages on the basis of rhythm. Additionally, higher theta activity during post-stimulus as compared to pre-stimulus rest suggests that stimulation after-effects are present from birth.

Differences in electroencephalography oscillations between normal aging and mild cognitive impairment during semantic memory retrieval

Semantic memory remains relatively stable with normal cognitive aging and declines in early stages of neurodegenerative disease. We measured electroencephalography (EEG) oscillatory correlates of semantic memory retrieval to examine the effects of normal and pathological aging. Twenty-nine cognitively healthy young adults (YA), 22 cognitively healthy aging adults (HA) and 20 patients with mild cognitive impairment (MCI) completed a semantic memory retrieval task with concurrent EEG recording in which they judged whether two words (features of objects) led to retrieval of an object (retrieval) or not (non-retrieval). Event-related power changes contrasting the two conditions (retrieval vs. non-retrieval) within theta, alpha, low-beta and high-beta EEG frequency bands were examined for normal aging (YA vs. HA) and pathological aging effects (HA vs. MCI). With no behavioural differences between the two normal age groups, we found later theta and alpha event-related power differences between conditions only in YA and a high-beta event-related power difference only in HA. For pathological aging effects, with reduced accuracy in MCI, we found different EEG patterns of early event-related beta power differences between conditions in MCI compared with HA and an event-related low-beta power difference only in HA. Beta oscillations were correlated with behavioural performance only in HA. We conclude that the aging brain relies on faster (beta) oscillations during the semantic memory task. With pathological aging, retrieval accuracy declines and pattern of beta oscillation changes. The findings provide insights about age-related neural mechanisms underlying semantic memory and have implications for early detection of pathological aging.

Age and vocabulary knowledge differentially influence the N400 and theta responses during semantic retrieval

Using electroencephalography (EEG) to study the neural oscillations supporting language development is increasingly common; however, a clear understanding of the relationship between neural oscillations and traditional Event Related Potentials (ERPs) is needed to disentangle how maturation of language-related neural networks supports semantic processing throughout grade school. Theta and the N400 are both thought to index semantic retrieval but, in adults, are only weakly correlated with one another indicating they may measure somewhat unique aspects of retrieval. Here, we studied the relationship between the N400 amplitude and theta power during semantic retrieval with key indicators of language abilities including age, vocabulary, reading comprehension and phonological memory in 226 children ages 8-15 years. The N400 and theta responses were positively correlated over posterior areas, but negatively correlated over frontal areas. When controlling for the N400 amplitude, the amplitude of the theta response was predicted by age, but not by language measures. On the other hand, when controlling theta amplitude, the amplitude of the N400 was predicted by both vocabulary knowledge and age. These findings indicate that while there is a clear relationship between the N400 and theta responses, they may each index unique aspects of development related to semantic retrieval.

Language: Its Origin and Ongoing Evolution

With the present paper, we sought to use research findings to illustrate the following thesis: the evolution of language follows the principles of human evolution. We argued that language does not exist for its own sake, it is one of a multitude of skills that developed to achieve a shared communicative goal, and all its features are reflective of this. Ongoing emerging language adaptations strive to better fit the present state of the human species. Theories of language have evolved from a single-modality to multimodal, from human-specific to usage-based and goal-driven. We proposed that language should be viewed as a multitude of communication techniques that have developed and are developing in response to selective pressure. The precise nature of language is shaped by the needs of the species (arguably, uniquely H. sapiens) utilizing it, and the emergence of new situational adaptations, as well as new forms and types of human language, demonstrates that language includes an act driven by a communicative goal. This article serves as an overview of the current state of psycholinguistic research on the topic of language evolution.

Neural bases of predictions during natural reading of known statements: An EEG and eye movements co-registration study

Predictions of incoming words performed during reading have an impact on how the reader moves their eyes and on the electrical brain potentials. Eye tracking (ET) experiments show that less predictable words are fixated for longer periods of times. Electroencephalography (EEG) experiments show that these words elicit a more negative potential around 400ms (N400) after the word onset when reading one word at a time (foveated reading). Nevertheless, there was no N400 potential during the foveated reading of previously known sentences (memory-encoded), which suggests that the prediction of words from memory-encoded sentences is based on different mechanisms than predictions performed on common sentences. Here, we performed an ET-EEG co-registration experiment where participants read common and memory-encoded sentences. Our results show that the N400 potential disappear when the reader recognises the sentence. Furthermore, time-frequency analyses show a larger alpha lateralisation and a beta power increase for memory-encoded sentences. This suggests a more distributed attention and an active maintenance of the cognitive set, in concordance to the predictive coding framework.

fMRI BOLD and MEG theta power reflect complementary aspects of activity during lexicosemantic decision in adolescents with ASD

Neuroimaging studies of autism spectrum disorder (ASD) have been predominantly unimodal. While many fMRI studies have reported atypical activity patterns for diverse tasks, the MEG literature in ASD remains comparatively small. Our group recently reported atypically increased event-related theta power in individuals with ASD during lexicosemantic processing. The current multimodal study examined the relationship between fMRI BOLD signal and anatomically-constrained MEG (aMEG) theta power. Thirty-three adolescents with ASD and 23 typically developing (TD) peers took part in both fMRI and MEG scans, during which they distinguished between standard words (SW), animal words (AW), and pseudowords (PW). Regions-of-interest (ROIs) were derived based on task effects detected in BOLD signal and aMEG theta power. BOLD signal and theta power were extracted for each ROI and word condition. Compared to TD participants, increased theta power in the ASD group was found across several time windows and regions including left fusiform and inferior frontal, as well as right angular and anterior cingulate gyri, whereas BOLD signal was significantly increased in the ASD group only in right anterior cingulate gyrus. No significant correlations were observed between BOLD signal and theta power. Findings suggest that the common interpretation of increases in BOLD signal and theta power as 'activation' require careful differentiation, as these reflect largely distinct aspects of regional brain activity. Some group differences in dynamic neural processing detected with aMEG that are likely relevant for lexical processing may be obscured by the hemodynamic signal source and low temporal resolution of fMRI.

Oscillatory brain responses to processing code-switches in the presence of others

Code-switching, i.e. the alternation between languages in a conversation, is a typical, yet socially-constrained practice in bilingual communities. For instance, code-switching is permissible only when other conversation partners are fluent in both languages. Studying code-switching provides insight in the cognitive and neural mechanisms underlying language control, and their modulation by linguistic and non-linguistic factors. Using time-frequency representations, we analyzed brain oscillation changes in EEG data recorded in a prior study (Kaan et al., 2020). In this study, Spanish-English bilinguals read sentences with and without switches in the presence of a bilingual or monolingual partner. Consistent with prior studies, code-switches were associated with a power decrease in the lower beta band (15-18 Hz). In addition, code-switches were associated with a power decrease in the upper gamma band (40-50 Hz), but only when a bilingual partner was present, suggesting the semantic/pragmatic processing of code-switches differs depending on who is present.

The neural response at the fundamental frequency of speech is modulated by word-level acoustic and linguistic information

Spoken language comprehension requires rapid and continuous integration of information, from lower-level acoustic to higher-level linguistic features. Much of this processing occurs in the cerebral cortex. Its neural activity exhibits, for instance, correlates of predictive processing, emerging at delays of a few 100 ms. However, the auditory pathways are also characterized by extensive feedback loops from higher-level cortical areas to lower-level ones as well as to subcortical structures. Early neural activity can therefore be influenced by higher-level cognitive processes, but it remains unclear whether such feedback contributes to linguistic processing. Here, we investigated early speech-evoked neural activity that emerges at the fundamental frequency. We analyzed EEG recordings obtained when subjects listened to a story read by a single speaker. We identified a response tracking the speaker's fundamental frequency that occurred at a delay of 11 ms, while another response elicited by the high-frequency modulation of the envelope of higher harmonics exhibited a larger magnitude and longer latency of about 18 ms with an additional significant component at around 40 ms. Notably, while the earlier components of the response likely originate from the subcortical structures, the latter presumably involves contributions from cortical regions. Subsequently, we determined the magnitude of these early neural responses for each individual word in the story. We then quantified the context-independent frequency of each word and used a language model to compute context-dependent word surprisal and precision. The word surprisal represented how predictable a word is, given the previous context, and the word precision reflected the confidence about predicting the next word from the past context. We found that the word-level neural responses at the fundamental frequency were predominantly influenced by the acoustic features: the average fundamental frequency and its variability. Amongst the linguistic features, only context-independent word frequency showed a weak but significant modulation of the neural response to the high-frequency envelope modulation. Our results show that the early neural response at the fundamental frequency is already influenced by acoustic as well as linguistic information, suggesting top-down modulation of this neural response.

Attention and regulation during emotional word comprehension in older adults: Evidence from event-related potentials and brain oscillations

Older adults often show a positivity bias effect during picture processing, focusing more on positive than negative information. It is unclear whether this positivity bias effect generalizes to language and whether arousal matters. The present study investigated how age affects emotional word comprehension with varied valence (positive, negative) and arousal (high, low). We recorded older and younger participants' brainwaves (EEG) while they read positive/negative and high/low-arousing words and pseudowords, and made word/non-word judgments. Older adults showed increased N400s and left frontal alpha decreases (300-450 ms) for low-arousing positive as compared to low-arousing negative words, suggesting an arousal-dependent positivity bias during lexical retrieval. Both age groups showed similar LPPs to negative words. Older adults further showed a larger mid-frontal theta increase (500-700 ms) than younger adults for low-arousing negative words, possibly indicating down-regulation of negative meanings of low-arousing words. Altogether, our data supported the strength and vulnerability integration model of aging.

Social appropriateness perception of dynamic interactions

The current study explored the judgement of communicative appropriateness while processing a dialogue between two individuals. All stimuli were presented as audio-visual as well as audio-only vignettes and 24 young adults reported their social impression (appropriateness) of literal, blunt, sarcastic, and teasing statements. On average, teasing statements were rated as more appropriate when processing audiovisual statements compared to the audio-only version of a stimuli, while sarcastic statements were judged as less appropriate with additional visual information. These results indicate a rejection of the Tinge Hypothesis for audio-visual vignettes while confirming it for the reduced, audio-only counterparts. We also analyzed time-frequency EEG data of four frequency bands that have been related to language processing: alpha, beta, theta and low gamma. We found desynchronization in the alpha band literal versus nonliteral items, confirming the assumption that the alpha band reflects stimulus complexity. The analysis also revealed a power increase in the theta, beta and low gamma band, especially when comparing blunt and nonliteral statements in the audio-only condition. The time-frequency results corroborate the prominent role of the alpha and theta bands in language processing and offer new insights into the neural correlates of communicative appropriateness and social aspects of speech perception.

A comparison and classification of oscillatory characteristics in speech perception and covert speech

Covert speech, the mental imagery of speaking, has been studied increasingly to understand and decode thoughts in the context of brain-computer interfaces. In studies of speech comprehension, neural oscillations are thought to play a key role in the temporal encoding of speech. However, little is known about the role of oscillations in covert speech. In this study, we investigated the oscillatory involvements in covert speech and speech perception. Data were collected from 10 participants with 64 channel EEG. Participants heard the words, 'blue' and 'orange', and subsequently mentally rehearsed them. First, continuous wavelet transform was performed on epoched signals and subsequently two-tailed t-tests between two classes were conducted to determine statistical differences in frequency and time (t-CWT). Features were also extracted using t-CWT and subsequently classified using a support vector machine. θ and γ phase amplitude coupling (PAC) was also assessed within and between tasks. All binary classifications produced accuracies significantly greater (80-90%) than chance level, supporting the use of t-CWT in determining relative oscillatory involvements. While the perception task dynamically invoked all frequencies with more prominent θ and α activity, the covert task favoured higher frequencies with significantly higher γ activity than perception. Moreover, the perception condition produced significant θ-γ PAC, corroborating a reported linkage between syllabic and phonemic sampling. Although this coupling was found to be suppressed in the covert condition, we found significant cross-task coupling between perception θ and covert speech γ. Covert speech processing appears to be largely associated with higher frequencies of EEG. Importantly, the significant cross-task coupling between speech perception and covert speech, in the absence of within-task covert speech PAC, supports the notion that the γ- and θ-bands subserve, respectively, shared and unique encoding processes across tasks.

Neural oscillations reflect meaning identification for novel words in context

During language processing, people make rapid use of contextual information to promote comprehension of upcoming words. When new words are learned implicitly, information contained in the surrounding context can provide constraints on their possible meaning. In the current study, EEG was recorded as participants listened to a series of three sentences, each containing an identical target pseudoword, with the aim of using contextual information in the surrounding language to identify a meaning representation for the novel word. In half of trials, sentences were semantically coherent so that participants could develop a single representation for the novel word that fit all contexts. Other trials contained unrelated sentence contexts so that meaning associations were not possible. We observed greater theta band enhancement over the left-hemisphere across central and posterior electrodes in response to pseudowords processed across semantically related compared to unrelated contexts. Additionally, relative alpha and beta band suppression was increased prior to pseudoword onset in trials where contextual information more readily promoted pseudoword-meaning associations. Under the hypothesis that theta enhancement indexes processing demands during lexical access, the current study provides evidence for selective online memory retrieval to novel words learned implicitly in a spoken context.

Oscillatory electroencephalographic patterns of arithmetic problem solving in fourth graders

Numerous studies have identified neurophysiological correlates of performing arithmetic in adults. For example, oscillatory electroencephalographic (EEG) patterns associated with retrieval and procedural strategies are well established. Whereas fact retrieval has been linked to enhanced left-hemispheric theta ERS (event-related synchronization), procedural strategies are accompanied by increased bilateral alpha ERD (event-related desynchronization). It is currently not clear if these findings generalize to children. Our study is the first to investigate oscillatory EEG activity related to strategy use and arithmetic operations in children. We assessed ERD/ERS correlates of 31 children in fourth grade (aged between nine and ten years) during arithmetic problem solving. We presented multiplication and subtraction problems, which children solved with fact retrieval or a procedure. We analyzed these four problem categories (retrieved multiplications, retrieved subtractions, procedural multiplications, and procedural subtractions) in our study. In summary, we found similar strategy-related patterns to those reported in previous studies with adults. That is, retrieval problems elicited stronger left-hemispheric theta ERS and weaker alpha ERD as compared to procedural problems. Interestingly, we observed neurophysiological differences between multiplications and subtractions within retrieval problems. Although there were no response time or accuracy differences, retrieved multiplications were accompanied by larger theta ERS than retrieved subtractions. This finding could indicate that retrieval of multiplication and subtraction facts are distinct processes, and/or that multiplications are more frequently retrieved than subtractions in this age group.

Determinants of bilingualism predict dynamic changes in resting state EEG oscillations

This study uses resting state EEG data from 103 bilinguals to understand how determinants of bilingualism may reshape the mind/brain. Participants completed the LSBQ, which quantifies language use and crucially the division of labor of dual-language use in diverse activities and settings over the lifespan. We hypothesized correlations between the degree of active bilingualism with power of neural oscillations in specific frequency bands. Moreover, we anticipated levels of mean coherence (connectivity between brain regions) to vary by degree of bilingual language experience. Results demonstrated effects of Age of L2/2L1 onset on high beta and gamma powers. Higher usage of the non-societal language at home and society modulated indices of functional connectivity in theta, alpha and gamma frequencies. Results add to the emerging literature on the neuromodulatory effects of bilingualism for rs-EEG, and are in line with claims that bilingualism effects are modulated by degree of engagement with dual-language experiential factors.

Neurofeedback Training Based on Motor Imagery Strategies Increases EEG Complexity in Elderly Population

Neurofeedback training (NFT) has shown promising results in recent years as a tool to address the effects of age-related cognitive decline in the elderly. Since previous studies have linked reduced complexity of electroencephalography (EEG) signal to the process of cognitive decline, we propose the use of non-linear methods to characterise changes in EEG complexity induced by NFT. In this study, we analyse the pre- and post-training EEG from 11 elderly subjects who performed an NFT based on motor imagery (MI-NFT). Spectral changes were studied using relative power (RP) from classical frequency bands (delta, theta, alpha, and beta), whilst multiscale entropy (MSE) was applied to assess EEG-induced complexity changes. Furthermore, we analysed the subject's scores from Luria tests performed before and after MI-NFT. We found that MI-NFT induced a power shift towards rapid frequencies, as well as an increase of EEG complexity in all channels, except for C3. These improvements were most evident in frontal channels. Moreover, results from cognitive tests showed significant enhancement in intellectual and memory functions. Therefore, our findings suggest the usefulness of MI-NFT to improve cognitive functions in the elderly and encourage future studies to use MSE as a metric to characterise EEG changes induced by MI-NFT.

How the healthy ageing brain supports semantic binding during language comprehension

Semantic binding refers to constructing complex meaning based on elementary building blocks. Using electroencephalography (EEG), we investigated the age-related changes in modulations of oscillatory brain activity supporting lexical retrieval and semantic binding. Young and older adult participants were visually presented two-word phrases, which for the first word revealed a lexical retrieval signature (e.g., swift vs. swrfeq) and for the second word revealed a semantic binding signature (e.g., horse in a semantic binding "swift horse" vs. no binding "swrfeq horse" context). The oscillatory brain activity associated with lexical retrieval as well as semantic binding significantly differed between healthy older and young adults. Specifically for lexical retrieval, we found that different age groups exhibited opposite patterns of theta and alpha modulation, which as a combined picture suggest that lexical retrieval is associated with different and delayed signatures in older compared with young adults. For semantic binding, in young adults, we found a signature in the low-beta range centred around the target word onset (i.e., a smaller low-beta increase for binding relative to no binding), whereas in healthy older adults, we found an opposite binding signature about ~500 ms later in the low- and high-beta range (i.e., a smaller low- and high-beta decrease for binding relative to no binding). The novel finding of a different and delayed oscillatory signature for semantic binding in healthy older adults reflects that the integration of word meaning into the semantic context takes longer and relies on different mechanisms in healthy older compared with young adults.

Multisensory Input Modulates P200 and L2 Sentence Comprehension: A One-Week Consolidation Phase

Multisensory input is an aid to language comprehension; however, it remains to be seen to what extent various combinations of senses may affect the P200 component and attention-related cognitive processing associated with L2 sentence comprehension along with the N400 as a later component. To this aim, we provided some multisensory input (enriched with data from three (i.e., exvolvement) and five senses (i.e., involvement)) for a list of unfamiliar words to 18 subjects. Subsequently, the words were embedded in an acceptability judgment task with 360 pragmatically correct and incorrect sentences. The task, along with the ERP recording, was conducted after a 1-week consolidation period to track any possible behavioral and electrophysiological distinctions in the retrieval of information with various sense combinations. According to the behavioral results, we found that the combination of five senses leads to more accurate and quicker responses. Based on the electrophysiological results, the combination of five senses induced a larger P200 amplitude compared to the three-sense combination. The implication is that as the sensory weight of the input increases, vocabulary retrieval is facilitated and more attention is directed to the overall comprehension of L2 sentences which leads to more accurate and quicker responses. This finding was not, however, reflected in the neural activity of the N400 component.

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

Until recently, human behavioral data from reading has mainly been of interest to researchers to understand human cognition. However, these human language processing signals can also be beneficial in machine learning-based natural language processing tasks. Using EEG brain activity for this purpose is largely unexplored as of yet. In this paper, we present the first large-scale study of systematically analyzing the potential of EEG brain activity data for improving natural language processing tasks, with a special focus on which features of the signal are most beneficial. We present a multi-modal machine learning architecture that learns jointly from textual input as well as from EEG features. We find that filtering the EEG signals into frequency bands is more beneficial than using the broadband signal. Moreover, for a range of word embedding types, EEG data improves binary and ternary sentiment classification and outperforms multiple baselines. For more complex tasks such as relation detection, only the contextualized BERT embeddings outperform the baselines in our experiments, which raises the need for further research. Finally, EEG data shows to be particularly promising when limited training data is available.

The role of the posterior hypothalamic area in the generation of theta rhythm

Theta rhythm is one of the best synchronized patterns of the oscillatory activity recorded in the mammalian brain. In humans, this rhythm is associated with REM sleep, spatial navigation, memory functions, analytical and language processes. On the other hand, it can be treated as a non-specific marker of such pathological states of the central nervous system as Alzheimer’s disease or epilepsy. The hippocampal formation is the key structure involved in the generation of this bioelectric phenomenon, both in humans and rodents (the most commonly studied laboratory animals). Theta rhythm appearance in the hippocampus is dependent on the interaction of multiple different structures of the nervous system. One of them is the posterior hypothalamic area (PHa), which constitutes a crucial part of the neuronal system modulating the ability of the hippocampal formation to generate theta rhythm. Although the research results encompassed in this paper emphasize the essential role of the PHa as a modulator of the hippocampal theta rhythm, it was the authors’ intent to indicate that this area is also capable of generating local rhythmical theta oscillations, independently of the influence of other brain structures.

Functional brain plasticity during L1 training on complex sentences: Changes in gamma-band oscillatory activity

The adult human brain remains plastic even after puberty. However, whether first language (L1) training in adults can alter the language network is yet largely unknown. Thus, we conducted a longitudinal training experiment on syntactically complex German sentence comprehension. Sentence complexity was varied by the depth of the center embedded relative clauses (i.e., single or double embedded). Comprehension was tested after each sentence with a question on the thematic role assignment. Thirty adult, native German speakers were recruited for 4 days of training. Magnetoencephalography (MEG) data were recorded and subjected to spectral power analysis covering the classical frequency bands (i.e., theta, alpha, beta, low gamma, and gamma). Normalized spectral power, time‐locked to the final closure of the relative clause, was subjected to a two‐factor analysis (“sentence complexity” and “training days”). Results showed that for the more complex sentences, the interaction of sentence complexity and training days was observed in Brodmann area 44 (BA 44) as a decrease of gamma power with training. Moreover, in the gamma band (55–95 Hz) functional connectivity between BA 44 and other brain regions such as the inferior frontal sulcus and the inferior parietal cortex were correlated with behavioral performance increase due to training. These results show that even for native speakers, complex L1 sentence training improves language performance and alters neural activities of the left hemispheric language network. Training strengthens the use of the dorsal processing stream with working‐memory‐related brain regions for syntactically complex sentences, thereby demonstrating the brain's functional plasticity for L1 training.

Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing

In recent years, a growing number of studies have used cortical tracking methods to investigate auditory language processing. Although most studies that employ cortical tracking stem from the field of auditory signal processing, this approach should also be of interest to psycholinguistics-particularly the subfield of sentence processing-given its potential to provide insight into dynamic language comprehension processes. However, there has been limited collaboration between these fields, which we suggest is partly because of differences in theoretical background and methodological constraints, some mutually exclusive. In this paper, we first review the theories and methodological constraints that have historically been prioritized in each field and provide concrete examples of how some of these constraints may be reconciled. We then elaborate on how further collaboration between the two fields could be mutually beneficial. Specifically, we argue that the use of cortical tracking methods may help resolve long-standing debates in the field of sentence processing that commonly used behavioral and neural measures (e.g., ERPs) have failed to adjudicate. Similarly, signal processing researchers who use cortical tracking may be able to reduce noise in the neural data and broaden the impact of their results by controlling for linguistic features of their stimuli and by using simple comprehension tasks. Overall, we argue that a balance between the methodological constraints of the two fields will lead to an overall improved understanding of language processing as well as greater clarity on what mechanisms cortical tracking of speech reflects. Increased collaboration will help resolve debates in both fields and will lead to new and exciting avenues for research.

MEG Theta during Lexico-Semantic and Executive Processing Is Altered in High-Functioning Adolescents with Autism

Neuroimaging studies have revealed atypical activation during language and executive tasks in individuals with autism spectrum disorders (ASD). However, the spatiotemporal stages of processing associated with these dysfunctions remain poorly understood. Using an anatomically constrained magnetoencephalography approach, we examined event-related theta oscillations during a double-duty lexical decision task that combined demands on lexico-semantic processing and executive functions. Relative to typically developing peers, high-functioning adolescents with ASD had lower performance accuracy on trials engaging selective semantic retrieval and cognitive control. They showed an early overall theta increase in the left fusiform cortex followed by greater activity in the left-lateralized temporal (starting at ~250 ms) and frontal cortical areas (after ~450 ms) known to contribute to language processing. During response preparation and execution, the ASD group exhibited elevated theta in the anterior cingulate cortex, indicative of greater engagement of cognitive control. Simultaneously increased activity in the ipsilateral motor cortex may reflect a less lateralized and suboptimally organized motor circuitry. Spanning early sensory-specific and late response selection stages, the higher event-related theta responsivity in ASD may indicate compensatory recruitment to offset inefficient lexico-semantic retrieval under cognitively demanding conditions. Together, these findings provide further support for atypical language and executive functions in high-functioning ASD.

Development of Neural Response to Novel Sounds in Fragile X Syndrome: Potential Biomarkers

Auditory processing abnormalities in fragile X syndrome (FXS) may contribute to difficulties with language development, pattern identification, and contextual updating. Participants with FXS (N = 41) and controls (N = 27) underwent auditory event-related potentials during presentation of an oddball paradigm. Data was adequate for analysis for 33 participants with FXS and 27 controls (age 4-51 y, 13 females [FXS]; 4-54 y, 11 females [control]). Participants with FXS showed larger N1 and P2 amplitudes, abnormal lack of modulation of P1 and P2 amplitudes and P2 latency in response to oddball stimuli ) relative to controls: Females with FXS were more similar to controls. Participants with FXS showed a marginal speeding of the P2 latency, suggesting potentiation to oddball stimuli rather than habituation. Participants with FXS showed a heightened N1 habituation effect compared to controls. Gamma power was significantly higher for participants with FXS. Groups did not differ on mismatch negativity. Both controls and participants with FXS showed similar developmental trajectories in P1 and N1 amplitude, P2 latency, and gamma power, but not for P2 amplitude. One month retest analyses performed in 14 participants suggest strong test-retest reliability for most measures. Individuals with FXS show previously demonstrated increased response amplitude and high frequency neural activity. Despite an overall normal developmental trajectory for most measures, individuals with FXS show age-independent but gender-dependent decreases in complex processing of novel stimuli. Many markers show strong retest reliability even in children and thus are potential biomarkers for clinical trials in FXS.

Cerebellar Theta and Beta Noninvasive Stimulation Rhythms Differentially Influence Episodic Memory versus Semantic Prediction

The human cerebellum is thought to interact with distributed brain networks to support cognitive abilities such as episodic memory and semantic prediction. Hippocampal and fronto-temporo-parietal networks that respectively support episodic memory versus semantic prediction have been associated with distinct endogenous oscillatory activity frequency bands: theta (∼3-8 Hz) versus beta (∼13-30 Hz) respectively. We sought to test whether it is possible to toggle cerebellar participation in episodic memory versus semantic prediction by noninvasively stimulating with theta versus beta rhythmic transcranial magnetic stimulation. In human subjects of both sexes, cerebellar theta stimulation improved episodic memory encoding but did not influence neural signals of semantic prediction, whereas beta stimulation of the same cerebellar location increased neural signals of semantic prediction but did not influence episodic memory encoding. This constitutes evidence for double dissociation of cerebellar contributions to semantic prediction versus episodic memory based on stimulation rhythm, supporting the hypothesis that the cerebellum can be biased to support these distinct cognitive abilities at the command of network-specific rhythmic activity.SIGNIFICANCE STATEMENT The cerebellum interacts with several distinct large-scale brain networks for cognitive function, but the factors governing selectivity of such interactions for particular functions are not fully understood. We tested the hypothesis that cerebellar contributions to cognition are guided by neural oscillations with function-specific frequency bands. We demonstrated that matching noninvasive stimulation to network-specific frequencies selectively enhanced episodic memory versus semantic prediction. These findings suggest that cerebellar contributions to cognitive networks are selected based on corresponding activity rhythms and could be used to develop cerebellar stimulation interventions for specific neurocognitive impairments.

A Compositional Neural Architecture for Language

Hierarchical structure and compositionality imbue human language with unparalleled expressive power and set it apart from other perception–action systems. However, neither formal nor neurobiological models account for how these defining computational properties might arise in a physiological system. I attempt to reconcile hierarchy and compositionality with principles from cell assembly computation in neuroscience; the result is an emerging theory of how the brain could convert distributed perceptual representations into hierarchical structures across multiple timescales while representing interpretable incremental stages of (de)compositional meaning. The model's architecture—a multidimensional coordinate system based on neurophysiological models of sensory processing—proposes that a manifold of neural trajectories encodes sensory, motor, and abstract linguistic states. Gain modulation, including inhibition, tunes the path in the manifold in accordance with behavior and is how latent structure is inferred. As a consequence, predictive information about upcoming sensory input during production and comprehension is available without a separate operation. The proposed processing mechanism is synthesized from current models of neural entrainment to speech, concepts from systems neuroscience and category theory, and a symbolic-connectionist computational model that uses time and rhythm to structure information. I build on evidence from cognitive neuroscience and computational modeling that suggests a formal and mechanistic alignment between structure building and neural oscillations, and moves toward unifying basic insights from linguistics and psycholinguistics with the currency of neural computation.

Electrophysiological dynamics of false belief understanding and complementation syntax in school-aged children: Oscillatory brain activity and event-related potentials

A large body of research in developmental psychology has been devoted to the ongoing debate over which aspects of language are fundamental to false belief understanding (FBU). A key proposal from de Villiers and colleagues proposes the essential role of complementation syntax in FBU development. The current study, using scalp electroencephalography (EEG), addressed one opposing hypothesis purporting that complementation is redundant to FBU by characterizing the electrophysiological correlates of FBU and complementation syntax in school-age children. Time-frequency decomposition showed robust parieto-occipital low beta (12-16 Hz) power reduction in the belief versus complementation conditions. This divergence was also supported by event-related potentials (ERPs), with parieto-occipital late slow waves around 600 to 900 ms distinguishing belief and complementation conditions. The false belief condition generated the lowest behavioral response accuracy, suggesting that it is the most challenging condition. Together, the current findings provide evidence showing that complementation is not redundant to FBU.

Bilingual Language Experience Shapes Resting-State Brain Rhythms

An increasing body of research has investigated how bilingual language experience changes brain structure and function, including changes to task-free, or "resting-state" brain connectivity. Such findings provide important evidence about how the brain continues to be shaped by different language experiences throughout the lifespan. The neural effects of bilingual language experience can provide evidence about the additional processing demands placed on the linguistic and/or executive systems by dual-language use. While considerable research has used MRI to examine where these changes occur, such methods cannot reveal the temporal dynamics of functioning brain networks at rest. The current study used data from task-free EEGS to disentangle how the linguistic and cognitive demands of bilingual language use impact brain functioning. Data analyzed from 106 bilinguals and 91 monolinguals revealed that bilinguals had greater alpha power, and significantly greater and broader coherence in the alpha and beta frequency ranges than monolinguals. Follow-up analyses showed that higher alpha was related to language control: more second-language use, higher native-language proficiency, and earlier age of second-language acquisition. Bilateral beta power was related to native-language proficiency, whereas theta was related to native-language proficiency only in left-hemisphere electrodes. The results contribute to our understanding of how the linguistic and cognitive requirements of dual-language use shape intrinsic brain activity, and what the broader implications for information processing may be.

Frequency-based Dissociation of Symbolic and Nonsymbolic Numerical Processing during Numerical Comparison

Recent evidence suggests that during numerical calculation, symbolic and nonsymbolic processing are functionally distinct operations. Nevertheless, both roughly recruit the same brain areas (spatially overlapping networks in the parietal cortex) and happen at the same time (roughly 250 msec poststimulus onset). We tested the hypothesis that symbolic and nonsymbolic processing are segregated by means of functionally relevant networks in different frequency ranges: high gamma (above 50 Hz) for symbolic processing and lower beta (12–17 Hz) for nonsymbolic processing. EEG signals were quantified as participants compared either symbolic numbers or nonsymbolic quantities. Larger EEG gamma-band power was observed for more difficult symbolic comparisons (ratio of 0.8 between the two numbers) than for easier comparisons (ratio of 0.2) over frontocentral regions. Similarly, beta-band power was larger for more difficult nonsymbolic comparisons than for easier ones over parietal areas. These results confirm the existence of a functional dissociation in EEG oscillatory dynamics during numerical processing that is compatible with the notion of distinct linguistic processing of symbolic numbers and approximation of nonsymbolic numerical information.

Cortical Tracking of Surprisal during Continuous Speech Comprehension

Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as of the precision of such a prediction. However, they have focused on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension, a listener hears many successive words whose predictability and precision vary over a large range. Here, we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network and through relating these speech features to EEG responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies, including the delta band as well as in the higher frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension.

The Relation between Alpha/Beta Oscillations and the Encoding of Sentence induced Contextual Information

Pre-stimulus alpha (8-12 Hz) and beta (16-20 Hz) oscillations have been frequently linked to the prediction of upcoming sensory input. Do these frequency bands serve as a neural marker of linguistic prediction as well? We hypothesized that if pre-stimulus alpha and beta oscillations index language predictions, their power should monotonically relate to the degree of predictability of incoming words based on past context. We expected that the more predictable the last word of a sentence, the stronger the alpha and beta power modulation. To test this, we measured neural responses with magnetoencephalography of healthy individuals during exposure to a set of linguistically matched sentences featuring three levels of sentence context constraint (high, medium and low constraint). We observed fluctuations in alpha and beta power before last word onset, and modulations in M400 amplitude after last word onset. The M400 amplitude was monotonically related to the degree of context constraint, with a high constraining context resulting in the strongest amplitude decrease. In contrast, pre-stimulus alpha and beta power decreased more strongly for intermediate constraints, followed by high and low constraints. Therefore, unlike the M400, pre-stimulus alpha and beta dynamics were not indexing the degree of word predictability from sentence context.

THE POWER OF NEURAL OSCILLATIONS TO INFORM SENTENCE COMPREHENSION: A LINGUISTIC PERSPECTIVE

The field of psycholinguistics is currently experiencing an explosion of interest in the analysis of neural oscillations - rhythmic brain activity synchronized at different temporal and spatial levels. Given that language comprehension relies on a myriad of processes, which are carried out in parallel in distributed brain networks, there is hope that this methodology might bring the field closer to understanding some of the more basic (spatially and temporally distributed, yet at the same time often overlapping) neural computations that support language function. In this review we discuss existing proposals linking oscillatory dynamics in different frequency bands to basic neural computations, and review relevant theories suggesting associations between band-specific oscillations and higher-level cognitive processes. More or less consistent patterns of oscillatory activity related to certain types of linguistic processing can already be derived from the evidence that has accumulated over the past few decades. The centerpiece of the current review is a synthesis of such patterns grouped by linguistic phenomenon. We restrict our review to evidence linking measures of oscillatory power to the comprehension of sentences, as well as linguistically (and/or pragmatically) more complex structures. For each grouping, we provide a brief summary and a table of associated oscillatory signatures that a psycholinguist might expect to find when employing a particular linguistic task. Summarizing across different paradigms, we conclude that a handful of basic neural oscillatory mechanisms are likely recruited in different ways and at different times for carrying out a variety of linguistic computations.

Co-registration of eye movements and neuroimaging for studying contextual predictions in natural reading

Sixteen years ago, Sereno and Rayner (2003. Measuring word recognition in reading: eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489-493) illustrated how "by means of review and comparison" eye movement (EM) and event-related potential (ERP) studies may advance our understanding of visual word recognition. Attempts to simultaneously record EMs and ERPs soon followed. Recently, this co-registration approach has also been transferred to fMRI and oscillatory EEG. With experimental settings close to natural reading, co-registration enables us to directly integrate insights from EM and neuroimaging studies. This should extend current experimental paradigms by moving the field towards studying sentence-level processing including effects of context and parafoveal preview. This article will introduce the basic principles and applications of co-registration and selectively review how this approach may shed light on one of the most controversially discussed issues in reading research, contextual predictions in online language processing.

Gamma tACS over the temporal lobe increases the occurrence of Eureka! moments

The solution to a problem might manifest itself as a burst of unexpected, unpredictable clarity. Such Eureka! events, or Insight moments, are among the most fascinating mysteries of human cognition, whose neurophysiological substrate seems to include a role for oscillatory activity within the α and γ bands in the right parietal and temporal brain regions. We tested this hypothesis on thirty-one healthy participants using transcranial Alternating Current Stimulation (tACS) to externally amplify α (10 Hz) and γ (40 Hz) activity in the right parietal and temporal lobes, respectively. During γ-tACS over the right temporal lobe, we observed an increase in accuracy on a verbal insight task. Furthermore, electroencephalography (EEG) data revealed an increase in γ spectral power over bilateral temporal lobes after stimulation. Additionally, resting-state functional MRI data acquired before the stimulation session suggested a correlation between behavioral response to right temporal lobe tACS and functional connectivity of bilateral temporal lobes, in line with the bilateral increase in γ band revealed by EEG. Overall, results suggest the possibility of enhancing the probability of generating Eureka! moments in humans by means of frequency-specific noninvasive brain stimulation.

Written-Word Concreteness Effects in Non-attend Conditions: Evidence From Mismatch Responses and Cortical Oscillations

It has been widely reported that concrete words have processing advantages over abstract words in terms of speed and efficiency of processing, a phenomenon known as the concreteness effect. However, little is still known about the early time-course of processing concrete and abstract words and whether this concreteness effect can still persist in conditions where attention is not focused on the words presented (automatic processing). This study aimed to shed light on these issues by examining the electrophysiological brain responses to concrete and abstract words. While participants were engaged in a non-linguistic color tracking task presented in the center of the monitor screen, matched Chinese concrete and abstract single-character words appeared within a passive oddball paradigm, out of the focus of attention. In calculating visual Mismatch Negativity (vMMN), Event-related potentials (ERPs) to words of the same semantic category were compared when these words were presented as deviants and standards. Before 320 ms, both abstract and concrete words yielded vMMN with left-lateralized distribution, suggesting similar verbal processing at an initial processing stage. After 320 ms, only concrete words additionally elicited vMMN with a central distribution. Time frequency (TF) analysis of the results also revealed larger theta power increase (200–300 ms) and theta power phase locking (200–450 ms) for concrete than for abstract words. Interestingly, there was more alpha power decrease for abstract than for concrete words from 300 to 450 ms. This may reflect the greater difficulty in processing abstract meaning. Taken together, our ERP and TF results point to the existence of different neural mechanisms underlying non-attentive processing of abstract and concrete words.

Conceptual and Perceptual Dimensions of Word Meaning Are Recovered Rapidly and in Parallel during Reading

A single word (the noun “ elephant”) encapsulates a complex multidimensional meaning, including both perceptual (“ big”, “ gray”, “ trumpeting”) and conceptual (“ mammal”, “ can be found in India”) features. Opposing theories make different predictions as to whether different features (also conceivable as dimensions of the semantic space) are stored in similar neural regions and recovered with similar temporal dynamics during word reading. In this magnetoencephalography study, we tracked the brain activity of healthy human participants while reading single words varying orthogonally across three semantic dimensions: two perceptual ones (i.e., the average implied real-world size and the average strength of association with a prototypical sound) and a conceptual one (i.e., the semantic category). The results indicate that perceptual and conceptual representations are supported by partially segregated neural networks: Whereas visual and auditory dimensions are encoded in the phase coherence of low-frequency oscillations of occipital and superior temporal regions, respectively, semantic features are encoded in the power of low-frequency oscillations of anterior temporal and inferior parietal areas. However, despite the differences, these representations appear to emerge at the same latency: around 200 msec after stimulus onset. Taken together, these findings suggest that perceptual and conceptual dimensions of the semantic space are recovered automatically, rapidly, and in parallel during word reading.

MEG imaging of recurrent gliomas reveals functional plasticity of hemispheric language specialization

In patients with gliomas, changes in hemispheric specialization for language determined by magnetoencephalography (MEG) were analyzed to elucidate the impact of treatment and tumor recurrence on language networks. Demonstration of reorganization of language networks in these patients has significant implications on the prevention of postoperative functional loss and recovery. Whole-brain activity during an auditory verb generation task was estimated from MEG recordings in a group of 73 patients with recurrent gliomas. Hemisphere of language dominance was estimated using the language laterality index (LI), a measure derived from the task. The initial scan was performed prior to resection; patients subsequently underwent surgery and adjuvant treatment. A second scan was performed upon recurrence prior to repeat resection. The relationship between the shift in LI between scans and demographics, anatomic location, pathology, and adjuvant treatment was analyzed. Laterality shifts were observed between scans; the median percent change was 29.1% across all patients. Laterality shift magnitude and relative direction were associated with the initial position of language dominance; patients with increased lateralization experienced greater shifts than those presenting more bilateral representation. A change in LI from left or right to bilateral (or vice versa) occurred in 23.3% of patients; complete switch occurred in 5.5% of patients. Patients with tumors within the language-dominant hemisphere experienced significantly greater shifts than those with contralateral tumors. The majority of patients with glioma experience shifts in language network organization over time which correlate with the relative position of language lateralization and tumor location.

Successful Encoding during Natural Reading Is Associated with Fixation-Related Potentials and Large-Scale Network Deactivation

Reading literature (e.g., an entire book) is an enriching experience that qualitatively differs from reading a single sentence; however, the brain dynamics of such context-dependent memory remains unclear. This study aimed to elucidate mnemonic neural dynamics during natural reading of literature by performing electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI). Brain activities of human participants recruited on campus were correlated with their subsequent memory, which was quantified by semantic correlation between the read text and reports subsequently written by them based on state of the art natural language processing procedures. The results of the EEG data analysis showed a significant positive relationship between subsequent memory and fixation-related EEG. Sentence-length and paragraph-length mnemonic processes were associated with N1-P2 and P3 fixation-related potential (FRP) components and fixation-related θ-band (4-8 Hz) EEG power, respectively. In contrast, the results of fMRI analysis showed a significant negative relationship between subsequent memory and blood oxygenation level-dependent (BOLD) activation. Sentence-length and paragraph-length mnemonic processes were associated with networks of regions forming part of the salience network and the default mode network (DMN), respectively. Taken together with the EEG results, these memory-related deactivations in the salience network and the DMN were thought to reflect the reading of sentences characterized by low mnemonic load and the suppression of task-irreverent thoughts, respectively. It was suggested that the context-dependent mnemonic process during literature reading requires large-scale network deactivation, which might reflect coordination of a range of voluntary processes during reading.

Developmental differences in the neural oscillations underlying auditory sentence processing in children and adults

Although very young children seem to process ongoing language quickly and effortlessly, neuroimaging and behavioral studies reveal that children continue to mature in their language skills through adolescence. During this prolonged development, children likely engage the same basic cognitive processes and neural mechanisms to perform language tasks as adults, but in somewhat different ways. In this study we used time frequency analysis of EEG to identify developmental differences in the engagement of neural oscillations between children (ages 10-12) and adults while listening to naturally-paced sentences. Adults displayed consistent beta changes throughout the sentence compared to children, thought to be related to efficient syntactic integration, and children displayed more broadly distributed theta changes than adults, thought to be related to more effortful semantic integration. Few differences in alpha, related to verbal working memory, existed between groups. These findings shed new light on developmental changes in the neuronal processes underlying language comprehension.

Differences in Neurocognitive Mechanisms Underlying the Processing of Center-Embedded and Non–embedded Musical Structures

In music, chords are organized into hierarchical structures based on recursive or embedded syntax. How the brain extracts recursive grammar is a central question in musical cognition and other cognitive neuroscience, but the precise mechanism remains unclear. By analyzing event related potentials (ERPs) and neural oscillatory activity, the present study investigated neurocognitive mechanisms underlying the processing of center-embedded structure in music by examining the differences in center-embedded and non-embedded structure processing and evaluating how these differences are affected by musical proficiency. Based on Western musical proficiency, the subjects were divided into two groups, non-experts and experts. The results revealed that for non-experts, the processing of center-embedded structure elicited greater early right-anterior negativity (ERAN) and N5 components as well as, reduced alpha and gamma activities than did the non-embedded structure. For experts, no significant difference in the ERP response was observed between the processing of non-embedded and center-embedded structures; however, the processing of center-embedded structure elicited increased beta activity compared to non-embedded structure. These findings indicate that listeners different in proficiency would rely on different cognitive neural mechanisms in music processing with the syntactic complexity increases.

Unattended processing of hierarchical pitch variations in spoken sentences

An auditory oddball paradigm was employed to examine the unattended processing of pitch variation which functions to signal hierarchically different levels of meaning contrasts. Four oddball conditions were constructed by varying the pitch contour of critical words embedded in a Mandarin Chinese sentence. Two conditions included lexical-level word meaning contrasts (i.e. TONE condition) and the other two sentence-level information-status contrasts (i.e. ACCENTUATION condition). Both included stimuli with early vs. late acoustic cue divergence points. Results showed that the two early-cue conditions elicited earlier Mismatch Negativities, regardless of their functional hierarchy. The deviant stimuli induced theta-band power increases in the TONE condition but beta-band power decreases in the ACCENTUATIION condition, regardless of the timing of their acoustic cues. These results suggest that, in an unattentive state, the human brain can functionally disentangle hierarchically different levels of pitch variation, and the brain responses to these pitch variations are time-locked to the presence of the acoustic cues.

Identifying the relationship between oscillatory dynamics and event-related responses

Event related potentials (ERPs) and time frequency analysis of the EEG can identify the temporally distinct coordination of groups of neurons across brain regions during sentence processing. Although there are strong arguments that ERP components and neural oscillations are driven by the same changes in the neural signal, others argue that the lack of clear associations between the two suggests oscillatory dynamics are more than just time frequency representations of ERP components, making it unclear how the two are related. The current study seeks to examine the neural activity underlying auditory sentence processing of both semantic and syntactic errors to clarify if ERP and time frequency analyses identify the same or unique neural responses. Thirty-nine adults completed an auditory semantic judgment task and a grammaticality judgment task. As expected, the semantic judgment task elicited a larger N400 and greater increase in theta power for semantic errors compared to correct sentences and the syntactic judgment task elicited a greater P600 and beta power decrease for both grammatical error types compared to syntactically correct sentences. Importantly, we identified a significant relationship between the N400 and P600 ERPs and theta and beta oscillatory dynamics during semantic and syntactic processing. These findings suggest that ERPs and neural oscillations measure similar neural processes; however, unaccounted for variance may indicate that neural oscillations provide additional information regarding fluctuations in power within a given frequency band. Future studies that vary semantic and syntactic complexity are necessary to understand the cognitive processes that are indexed by these oscillations.

An oscillatory neural network model that demonstrates the benefits of multisensory learning

Since the world consists of objects that stimulate multiple senses, it is advantageous for a vertebrate to integrate all the sensory information available. However, the precise mechanisms governing the temporal dynamics of multisensory processing are not well understood. We develop a computational modeling approach to investigate these mechanisms. We present an oscillatory neural network model for multisensory learning based on sparse spatio-temporal encoding. Recently published results in cognitive science show that multisensory integration produces greater and more efficient learning. We apply our computational model to qualitatively replicate these results. We vary learning protocols and system dynamics, and measure the rate at which our model learns to distinguish superposed presentations of multisensory objects. We show that the use of multiple channels accelerates learning and recall by up to 80%. When a sensory channel becomes disabled, the performance degradation is less than that experienced during the presentation of non-congruent stimuli. This research furthers our understanding of fundamental brain processes, paving the way for multiple advances including the building of machines with more human-like capabilities.

Electrophysiological dynamics of Chinese phonology during visual word recognition in Chinese-English bilinguals

Silent word reading leads to the activation of orthographic (spelling), semantic (meaning), as well as phonological (sound) information. For bilinguals, native language information can also be activated automatically when they read words in their second language. For example, when Chinese-English bilinguals read words in their second language (English), the phonology of the Chinese translations is automatically activated. Chinese phonology, however, consists of consonants and vowels (segmental) and tonal information. To what extent these two aspects of Chinese phonology are activated is yet unclear. Here, we used behavioural measures, event-related potentials and oscillatory EEG to investigate Chinese segmental and tonal activation during word recognition. Evidence of Chinese segmental activation was found when bilinguals read English words (faster responses, reduced N400, gamma-band power reduction) and when they read Chinese words (increased LPC, gamma-band power reduction). In contrast, evidence for Chinese tonal activation was only found when bilinguals read Chinese words (gamma-band power increase). Together, our converging behavioural and electrophysiological evidence indicates that Chinese segmental information is activated during English word reading, whereas both segmental and tonal information are activated during Chinese word reading. Importantly, gamma-band oscillations are modulated differently by tonal and segmental activation, suggesting independent processing of Chinese tones and segments.