MEG and EEG data fusion: simultaneous localisation of face-evoked responses

Localized estimation of event-related neural source activity from simultaneous MEG-EEG with a recurrent neural network

Estimating intracranial current sources underlying the electromagnetic signals observed from extracranial sensors is a perennial challenge in non-invasive neuroimaging. Established solutions to this inverse problem treat time samples independently without considering the temporal dynamics of event-related brain processes. This paper describes current source estimation from simultaneously recorded magneto- and electro-encephalography (MEEG) using a recurrent neural network (RNN) that learns sequential relationships from neural data. The RNN was trained in two phases: (1) pre-training and (2) transfer learning with L1 regularization applied to the source estimation layer. Performance of using scaled labels derived from MEEG, magnetoencephalography (MEG), or electroencephalography (EEG) were compared, as were results from volumetric source space with free dipole orientation and surface source space with fixed dipole orientation. Exact low-resolution electromagnetic tomography (eLORETA) and mixed-norm L1/L2 (MxNE) source estimation methods were also applied to these data for comparison with the RNN method. The RNN approach outperformed other methods in terms of output signal-to-noise ratio, correlation and mean-squared error metrics evaluated against reference event-related field (ERF) and event-related potential (ERP) waveforms. Using MEEG labels with fixed-orientation surface sources produced the most consistent estimates. To estimate sources of ERF and ERP waveforms, the RNN generates temporal dynamics within its internal computational units, driven by sequential structure in neural data used as training labels. It thus provides a data-driven model of computational transformations from psychophysiological events into corresponding event-related neural signals, which is unique among MEEG source reconstruction solutions.

Multi-Modal Electrophysiological Source Imaging With Attention Neural Networks Based on Deep Fusion of EEG and MEG

The process of reconstructing underlying cortical and subcortical electrical activities from Electroencephalography (EEG) or Magnetoencephalography (MEG) recordings is called Electrophysiological Source Imaging (ESI). Given the complementarity between EEG and MEG in measuring radial and tangential cortical sources, combined EEG/MEG is considered beneficial in improving the reconstruction performance of ESI algorithms. Traditional algorithms mainly emphasize incorporating predesigned neurophysiological priors to solve the ESI problem. Deep learning frameworks aim to directly learn the mapping from scalp EEG/MEG measurements to the underlying brain source activities in a data-driven manner, demonstrating superior performance compared to traditional methods. However, most of the existing deep learning approaches for the ESI problem are performed on a single modality of EEG or MEG, meaning the complementarity of these two modalities has not been fully utilized. How to fuse the EEG and MEG in a more principled manner under the deep learning paradigm remains a challenging question. This study develops a Multi-Modal Deep Fusion (MMDF) framework using Attention Neural Networks (ANN) to fully leverage the complementary information between EEG and MEG for solving the ESI inverse problem, which is termed as MMDF-ANN. Specifically, our proposed brain source imaging approach consists of four phases, including feature extraction, weight generation, deep feature fusion, and source mapping. Our experimental results on both synthetic dataset and real dataset demonstrated that using a fusion of EEG and MEG can significantly improve the source localization accuracy compared to using a single-modality of EEG or MEG. Compared to the benchmark algorithms, MMDF-ANN demonstrated good stability when reconstructing sources with extended activation areas and situations of EEG/MEG measurements with a low signal-to-noise ratio.

Finding structure during incremental speech comprehension

A core aspect of human speech comprehension is the ability to incrementally integrate consecutive words into a structured and coherent interpretation, aligning with the speaker's intended meaning. This rapid process is subject to multidimensional probabilistic constraints, including both linguistic knowledge and non-linguistic information within specific contexts, and it is their interpretative coherence that drives successful comprehension. To study the neural substrates of this process, we extract word-by-word measures of sentential structure from BERT, a deep language model, which effectively approximates the coherent outcomes of the dynamic interplay among various types of constraints. Using representational similarity analysis, we tested BERT parse depths and relevant corpus-based measures against the spatiotemporally resolved brain activity recorded by electro-/magnetoencephalography when participants were listening to the same sentences. Our results provide a detailed picture of the neurobiological processes involved in the incremental construction of structured interpretations. These findings show when and where coherent interpretations emerge through the evaluation and integration of multifaceted constraints in the brain, which engages bilateral brain regions extending beyond the classical fronto-temporal language system. Furthermore, this study provides empirical evidence supporting the use of artificial neural networks as computational models for revealing the neural dynamics underpinning complex cognitive processes in the brain.

Parkinson's disease impairs cortical sensori-motor decision-making cascades

The transformation from perception to action requires a set of neuronal decisions about the nature of the percept, identification and selection of response options and execution of the appropriate motor response. The unfolding of such decisions is mediated by distributed representations of the decision variables-evidence and intentions-that are represented through oscillatory activity across the cortex. Here we combine magneto-electroencephalography and linear ballistic accumulator models of decision-making to reveal the impact of Parkinson's disease during the selection and execution of action. We used a visuomotor task in which we independently manipulated uncertainty in sensory and action domains. A generative accumulator model was optimized to single-trial neurophysiological correlates of human behaviour, mapping the cortical oscillatory signatures of decision-making, and relating these to separate processes accumulating sensory evidence and selecting a motor action. We confirmed the role of widespread beta oscillatory activity in shaping the feed-forward cascade of evidence accumulation from resolution of sensory inputs to selection of appropriate responses. By contrasting the spatiotemporal dynamics of evidence accumulation in age-matched healthy controls and people with Parkinson's disease, we identified disruption of the beta-mediated cascade of evidence accumulation as the hallmark of atypical decision-making in Parkinson's disease. In frontal cortical regions, there was inefficient processing and transfer of perceptual information. Our findings emphasize the intimate connection between abnormal visuomotor function and pathological oscillatory activity in neurodegenerative disease. We propose that disruption of the oscillatory mechanisms governing fast and precise information exchanges between the sensory and motor systems contributes to behavioural changes in people with Parkinson's disease.

Deep learning based source imaging provides strong sublobar localization of epileptogenic zone from MEG interictal spikes

Electromagnetic source imaging (ESI) offers unique capability of imaging brain dynamics for studying brain functions and aiding the clinical management of brain disorders. Challenges exist in ESI due to the ill-posedness of the inverse problem and thus the need of modeling the underlying brain dynamics for regularizations. Advances in generative models provide opportunities for more accurate and realistic source modeling that could offer an alternative approach to ESI for modeling the underlying brain dynamics beyond equivalent physical source models. However, it is not straightforward to explicitly formulate the knowledge arising from these generative models within the conventional ESI framework. Here we investigate a novel source imaging framework based on mesoscale neuronal modeling and deep learning (DL) that can learn the sensor-source mapping relationship directly from MEG data for ESI. Two DL-based ESI models were trained based on data generated by neural mass models and either generic or personalized head models. The robustness of the two DL models was evaluated by systematic computer simulations and clinical validation in a cohort of 29 drug-resistant focal epilepsy patients who underwent intracranial EEG (iEEG) evaluation or surgical resection. Results estimated from pre-operative MEG interictal spikes were quantified using the overlap with resection regions and the distance to the seizure-onset zone (SOZ) defined by iEEG recordings. The DL-based ESI provided robust results when no personalized head geometry is considered, reaching a spatial dispersion of 21.90 ± 19.03 mm, sublobar concordance of 83 %, and sublobar sensitivity and specificity of 66 and 97 % respectively. When using personalized head geometry derived from individual patients' MRI in the training data, personalized DL-based ESI model can further improve the performance and reached a spatial dispersion of 8.19 ± 8.14 mm, sublobar concordance of 93 %, and sublobar sensitivity and specificity of 77 and 99 % respectively. When compared to the SOZ, the localization error of the personalized approach is 15.78 ± 5.54 mm, outperforming the conventional benchmarks. This work demonstrates that combining generative models and deep learning enables an accurate and robust imaging of epileptogenic zone from MEG recordings with strong sublobar precision, suggesting its added value to enhancing MEG source localization and imaging, and to epilepsy source localization and other clinical applications.

Distinct roles for the anterior temporal lobe and angular gyrus in the spatiotemporal cortical semantic network

Semantic knowledge is supported by numerous brain regions, but the spatiotemporal configuration of the network that links these areas remains an open question. The hub-and-spokes model posits that a central semantic hub coordinates this network. In this study, we explored distinct aspects that define a semantic hub, as reflected in the spatiotemporal modulation of neural activity and connectivity by semantic variables, from the earliest stages of semantic processing. We used source-reconstructed electro/magnetoencephalography, and investigated the concreteness contrast across three tasks. In a whole-cortex analysis, the left anterior temporal lobe (ATL) was the only area that showed modulation of evoked brain activity from 100 ms post-stimulus. Furthermore, using Dynamic Causal Modeling of the evoked responses, we investigated effective connectivity amongst the candidate semantic hub regions, that is, left ATL, supramarginal/angular gyrus (SMG/AG), middle temporal gyrus, and inferior frontal gyrus. We found that models with a single semantic hub showed the highest Bayesian evidence, and the hub region was found to change from ATL (within 250 ms) to SMG/AG (within 450 ms) over time. Our results support a single semantic hub view, with ATL showing sustained modulation of neural activity by semantics, and both ATL and AG underlying connectivity depending on the stage of semantic processing.

Maturational trajectory of fusiform gyrus neural activity when viewing faces: From 4 months to 4 years old

Infant and young child electrophysiology studies have provided information regarding the maturation of face-encoding neural processes. A limitation of previous research is that very few studies have examined face-encoding processes in children 12-48 months of age, a developmental period characterized by rapid changes in the ability to encode facial information. The present study sought to fill this gap in the literature via a longitudinal study examining the maturation of a primary node in the face-encoding network-the left and right fusiform gyrus (FFG). Whole-brain magnetoencephalography (MEG) data were obtained from 25 infants with typical development at 4-12 months, and with follow-up MEG exams every ∼12 months until 3-4 years old. Children were presented with color images of Face stimuli and visual noise images (matched on spatial frequency, color distribution, and outer contour) that served as Non-Face stimuli. Using distributed source modeling, left and right face-sensitive FFG evoked waveforms were obtained from each child at each visit, with face-sensitive activity identified via examining the difference between the Non-Face and Face FFG timecourses. Before 24 months of age (Visits 1 and 2) the face-sensitive FFG M290 response was the dominant response, observed in the left and right FFG ∼250-450 ms post-stimulus. By 3-4 years old (Visit 4), the left and right face-sensitive FFG response occurred at a latency consistent with a face-sensitive M170 response ∼100-250 ms post-stimulus. Face-sensitive left and right FFG peak latencies decreased as a function of age (with age explaining greater than 70% of the variance in face-sensitive FFG latency), and with an adult-like FFG latency observed at 3-4 years old. Study findings thus showed face-sensitive FFG maturational changes across the first 4 years of life. Whereas a face-sensitive M290 response was observed under 2 years of age, by 3-4 years old, an adult-like face-sensitive M170 response was observed bilaterally. Future studies evaluating the maturation of face-sensitive FFG activity in infants at risk for neurodevelopmental disorders are of interest, with the present findings suggesting age-specific face-sensitive neural markers of a priori interest.

Combining electro- and magnetoencephalography data using directional archetypal analysis

Metastable microstates in electro- and magnetoencephalographic (EEG and MEG) measurements are usually determined using modified k-means accounting for polarity invariant states. However, hard state assignment approaches assume that the brain traverses microstates in a discrete rather than continuous fashion. We present multimodal, multisubject directional archetypal analysis as a scale and polarity invariant extension to archetypal analysis using a loss function based on the Watson distribution. With this method, EEG/MEG microstates are modeled using subject- and modality-specific archetypes that are representative, distinct topographic maps between which the brain continuously traverses. Archetypes are specified as convex combinations of unit norm input data based on a shared generator matrix, thus assuming that the timing of neural responses to stimuli is consistent across subjects and modalities. The input data is reconstructed as convex combinations of archetypes using a subject- and modality-specific continuous archetypal mixing matrix. We showcase the model on synthetic data and an openly available face perception event-related potential data set with concurrently recorded EEG and MEG. In synthetic and unimodal experiments, we compare our model to conventional Euclidean multisubject archetypal analysis. We also contrast our model to a directional clustering model with discrete state assignments to highlight the advantages of modeling state trajectories rather than hard assignments. We find that our approach successfully models scale and polarity invariant data, such as microstates, accounting for intersubject and intermodal variability. The model is readily extendable to other modalities ensuring component correspondence while elucidating spatiotemporal signal variability.

Multimodal neuroimaging with optically pumped magnetometers: A simultaneous MEG-EEG-fNIRS acquisition system

Multimodal neuroimaging plays an important role in neuroscience research. Integrated noninvasive neuroimaging modalities, such as magnetoencephalography (MEG), electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), allow neural activity and related physiological processes in the brain to be precisely and comprehensively depicted, providing an effective and advanced platform to study brain function. Noncryogenic optically pumped magnetometer (OPM) MEG has high signal power due to its on-scalp sensor layout and enables more flexible configurations than traditional commercial superconducting MEG. Here, we integrate OPM-MEG with EEG and fNIRS to develop a multimodal neuroimaging system that can simultaneously measure brain electrophysiology and hemodynamics. We conducted a series of experiments to demonstrate the feasibility and robustness of our MEG-EEG-fNIRS acquisition system. The complementary neural and physiological signals simultaneously collected by our multimodal imaging system provide opportunities for a wide range of potential applications in neurovascular coupling, wearable neuroimaging, hyperscanning and brain-computer interfaces.

Neurophysiological and Brain Structural Markers of Cognitive Frailty Differ from Alzheimer's Disease

With increasing life span and prevalence of dementia, it is important to understand the mechanisms of cognitive aging. Here, we focus on a subgroup of the population we term "cognitively frail," defined by reduced cognitive function in the absence of subjective memory complaints, or a clinical diagnosis of dementia. Cognitive frailty is distinct from cognitive impairment caused by physical frailty. It has been proposed to be a precursor to Alzheimer's disease, but may alternatively represent one end of a nonpathologic spectrum of cognitive aging. We test these hypotheses in humans of both sexes, by comparing the structural and neurophysiological properties of a community-based cohort of cognitive frail adults, to people presenting clinically with diagnoses of Alzheimer's disease or mild cognitive impairment, and community-based cognitively typical older adults. Cognitive performance of the cognitively frail was similar to those with mild cognitive impairment. We used a novel cross-modal paired-associates task that presented images followed by sounds, to induce physiological responses of novelty and associative mismatch, recorded by EEG/MEG. Both controls and cognitively frail showed stronger mismatch responses and larger temporal gray matter volume, compared with people with mild cognitive impairment and Alzheimer's disease. Our results suggest that community-based cognitively frail represents a spectrum of normal aging rather than incipient Alzheimer's disease, despite similar cognitive function. Lower lifelong cognitive reserve, hearing impairment, and cardiovascular comorbidities might contribute to the etiology of the cognitive frailty. Critically, community-based cohorts of older adults with low cognitive performance should not be interpreted as representing undiagnosed Alzheimer's disease.SIGNIFICANCE STATEMENT The current study investigates the neural signatures of cognitive frailty in relation to healthy aging and Alzheimer's disease. We focus on the cognitive aspect of frailty and show that, despite performing similarly to the patients with mild cognitive impairment, a cohort of community-based adults with poor cognitive performance do not show structural atrophy or neurophysiological signatures of Alzheimer's disease. Our results call for caution before assuming that cognitive frailty represents latent Alzheimer's disease. Instead, the cognitive underperformance of cognitively frail adults could result in cumulative effects of multiple psychosocial risk factors over the lifespan, and medical comorbidities.

Imaging the extent and location of spatiotemporally distributed epileptiform sources from MEG measurements

Non-invasive MEG/EEG source imaging provides valuable information about the epileptogenic brain areas which can be used to aid presurgical planning in focal epilepsy patients suffering from drug-resistant seizures. However, the source extent estimation for electrophysiological source imaging remains to be a challenge and is usually largely dependent on subjective choice. Our recently developed algorithm, fast spatiotemporal iteratively reweighted edge sparsity minimization (FAST-IRES) strategy, has been shown to objectively estimate extended sources from EEG recording, while it has not been applied to MEG recordings. In this work, through extensive numerical experiments and real data analysis in a group of focal drug-resistant epilepsy patients' interictal spikes, we demonstrated the ability of FAST-IRES algorithm to image the location and extent of underlying epilepsy sources from MEG measurements. Our results indicate the merits of FAST-IRES in imaging the location and extent of epilepsy sources for pre-surgical evaluation from MEG measurements.

Face-selective responses in combined EEG/MEG recordings with fast periodic visual stimulation (FPVS)

Fast periodic visual stimulation (FPVS) allows the recording of objective brain responses of human face categorization (i.e., generalizable face-selective responses) with high signal-to-noise ratio. This approach has been successfully employed in a number of scalp electroencephalography (EEG) studies but has not been used with magnetoencephalography (MEG) yet, let alone with combined MEG/EEG recordings and distributed source estimation. Here, we presented various natural images of faces periodically (1.2 Hz) among natural images of objects (base frequency 6 Hz) whilst recording simultaneous EEG and MEG in 15 participants. Both measurement modalities showed face-selective responses at 1.2 Hz and harmonics across participants, with high and comparable signal-to-noise ratio (SNR) in about 3 min of stimulation. The correlation of face categorization responses between EEG and two MEG sensor types was lower than between the two MEG sensor types, indicating that the two sensor modalities provide independent information about the sources of face-selective responses. Face-selective EEG responses were right-lateralized as reported previously, and were numerically but non-significantly right-lateralized in MEG data. Distributed source estimation based on combined EEG/MEG signals confirmed a more bilateral face-selective response in visual brain regions located anteriorly to the common response to all stimuli at 6 Hz and harmonics. Conventional sensor and source space analyses of evoked responses in the time domain further corroborated this result. Our results demonstrate that FPVS in combination with simultaneously recorded EEG and MEG may serve as an efficient localizer paradigm for human face categorization.

Data-point-wise spatiotemporal mapping of human ventral visual areas: Use of spatial frequency/luminance-modulated chromatic faces

Visual information involving facial identity and expression is crucial for social communication. Although the influence of facial features such as spatial frequency (SF) and luminance on face processing in visual areas has been studied extensively using grayscale stimuli, the combined effects of other features in this process have not been characterized. To determine the combined effects of different SFs and color, we created chromatic stimuli with low, high or no SF components, which bring distinct SF and color information into the ventral stream simultaneously. To obtain neural activity data with high spatiotemporal resolution we recorded face-selective responses (M170) using magnetoencephalography. We used a permutation test procedure with threshold-free cluster enhancement to assess statistical significance while resolving problems related to multiple comparisons and arbitrariness found in traditional statistical methods. We found that time windows with statistically significant threshold levels were distributed differently among the stimulus conditions. Face stimuli containing any SF components evoked M170 in the fusiform gyrus (FG), whereas a significant emotional effect on M170 was only observed with the original images. Low SF faces elicited larger activation of the FG and the inferior occipital gyrus than the original images, suggesting an interaction between low and high SF information processing. Interestingly, chromatic face stimuli without SF first activated color-selective regions and then the FG, indicating that facial color was processed according to a hierarchy in the ventral stream. These findings suggest complex effects of SFs in the presence of color information, reflected in M170, and unveil the detailed spatiotemporal dynamics of face processing in the human brain.

Ant Colony System Optimization for Spatiotemporal Modelling of Combined EEG and MEG Data

Electroencephalography/Magnetoencephalography (EEG/MEG) source localization involves the estimation of neural activity inside the brain volume that underlies the EEG/MEG measures observed at the sensor array. In this paper, we consider a Bayesian finite spatial mixture model for source reconstruction and implement Ant Colony System (ACS) optimization coupled with Iterated Conditional Modes (ICM) for computing estimates of the neural source activity. Our approach is evaluated using simulation studies and a real data application in which we implement a nonparametric bootstrap for interval estimation. We demonstrate improved performance of the ACS-ICM algorithm as compared to existing methodology for the same spatiotemporal model.

Maturation of hemispheric specialization for face encoding during infancy and toddlerhood

•

Using infant magnetoencephalography (MEG), study findings show maturational changes to fusiform gyrus (FFG) activity when viewing faces.

•

Earlier right FFG activity to face stimuli is associated with better social and cognitive ability.

•

Stronger right- than left-hemisphere FFG responses to face stimuli are most evident after 1 year of age.

Little is known about the neural processes associated with attending to social stimuli during infancy and toddlerhood. Using infant magnetoencephalography (MEG), fusiform gyrus (FFG) activity while processing Face and Non-Face stimuli was examined in 46 typically developing infants 3 to 24 months old (28 males). Several findings indicated FFG maturation throughout the first two years of life. First, right FFG responses to Face stimuli decreased as a function of age. Second, hemispheric specialization to the face stimuli developed somewhat slowly, with earlier right than left FFG peak activity most evident after 1 year of age. Right FFG activity to Face stimuli was of clinical interest, with an earlier right FFG response associated with better performance on tests assessing social and cognitive ability. Building on the above, clinical studies examining maturational change in FFG activity (e.g., lateralization and speed) in infants at-risk for childhood disorders associated with social deficits are of interest to identify atypical FFG maturation before a formal diagnosis is possible.

Empirical Bayes evaluation of fused EEG-MEG source reconstruction: Application to auditory mismatch evoked responses

We here turn the general and theoretical question of the complementarity of EEG and MEG for source reconstruction, into a practical empirical one. Precisely, we address the challenge of evaluating multimodal data fusion on real data. For this purpose, we build on the flexibility of Parametric Empirical Bayes, namely for EEG-MEG data fusion, group level inference and formal hypothesis testing. The proposed approach follows a two-step procedure by first using unimodal or multimodal inference to derive a cortical solution at the group level; and second by using this solution as a prior model for single subject level inference based on either unimodal or multimodal data. Interestingly, for inference based on the same data (EEG, MEG or both), one can then formally compare, as alternative hypotheses, the relative plausibility of the two unimodal and the multimodal group priors. Using auditory data, we show that this approach enables to draw important conclusions, namely on (i) the superiority of multimodal inference, (ii) the greater spatial sensitivity of MEG compared to EEG, (iii) the ability of EEG data alone to source reconstruct temporal lobe activity, (iv) the usefulness of EEG to improve MEG based source reconstruction. Importantly, we largely reproduce those findings over two different experimental conditions. We here focused on Mismatch Negativity (MMN) responses for which generators have been extensively investigated with little homogeneity in the reported results. Our multimodal inference at the group level revealed spatio-temporal activity within the supratemporal plane with a precision which, to our knowledge, has never been achieved before with non-invasive recordings.

Tau pathology in early Alzheimer's disease is linked to selective disruptions in neurophysiological network dynamics

Understanding the role of Tau protein aggregation in the pathogenesis of Alzheimer's disease is critical for the development of new Tau-based therapeutic strategies to slow or prevent dementia. We tested the hypothesis that Tau pathology is associated with functional organization of widespread neurophysiological networks. We used electro-magnetoencephalography with [¹⁸F]AV-1451 PET scanning to quantify Tau-dependent network changes. Using a graph theoretical approach to brain connectivity, we quantified nodal measures of functional segregation, centrality, and the efficiency of information transfer and tested them against levels of [¹⁸F]AV-1451. Higher Tau burden in early Alzheimer's disease was associated with a shift away from the optimal small-world organization and a more fragmented network in the beta and gamma bands, whereby parieto-occipital areas were disconnected from the anterior parts of the network. Similarly, higher Tau burden was associated with decreases in both local and global efficiency, especially in the gamma band. The results support the translational development of neurophysiological "signatures" of Alzheimer's disease, to understand disease mechanisms in humans and facilitate experimental medicine studies.

Anterior temporal lobe is necessary for efficient lateralised processing of spoken word identity

In the healthy human brain, the processing of language is strongly lateralised, usually to the left hemisphere, while the processing of complex non-linguistic sounds recruits brain regions bilaterally. Here we asked whether the anterior temporal lobes, strongly implicated in semantic processing, are critical to this special treatment of spoken words. Nine patients with semantic dementia (SD) and fourteen age-matched controls underwent magnetoencephalography and structural MRI. Voxel based morphometry demonstrated the stereotypical pattern of SD: severe grey matter loss restricted to the anterior temporal lobes, with the left side more affected. During magnetoencephalography, participants listened to word sets in which identity and meaning were ambiguous until word completion, for example PLAYED versus PLATE. Whereas left-hemispheric responses were similar across groups, patients demonstrated increased right hemisphere activity 174-294 msec after stimulus disambiguation. Source reconstructions confirmed recruitment of right-sided analogues of language regions in SD: atrophy of anterior temporal lobes was associated with increased activity in right temporal pole, middle temporal gyrus, inferior frontal gyrus and supramarginal gyrus. Overall, the results indicate that anterior temporal lobes are necessary for normal and efficient lateralised processing of word identity by the language network.

The Neural Time Course of Semantic Ambiguity Resolution in Speech Comprehension

Semantically ambiguous words challenge speech comprehension, particularly when listeners must select a less frequent (subordinate) meaning at disambiguation. Using combined magnetoencephalography (MEG) and EEG, we measured neural responses associated with distinct cognitive operations during semantic ambiguity resolution in spoken sentences: (i) initial activation and selection of meanings in response to an ambiguous word and (ii) sentence reinterpretation in response to subsequent disambiguation to a subordinate meaning. Ambiguous words elicited an increased neural response approximately 400-800 msec after their acoustic offset compared with unambiguous control words in left frontotemporal MEG sensors, corresponding to sources in bilateral frontotemporal brain regions. This response may reflect increased demands on processes by which multiple alternative meanings are activated and maintained until later selection. Disambiguating words heard after an ambiguous word were associated with marginally increased neural activity over bilateral temporal MEG sensors and a central cluster of EEG electrodes, which localized to similar bilateral frontal and left temporal regions. This later neural response may reflect effortful semantic integration or elicitation of prediction errors that guide reinterpretation of previously selected word meanings. Across participants, the amplitude of the ambiguity response showed a marginal positive correlation with comprehension scores, suggesting that sentence comprehension benefits from additional processing around the time of an ambiguous word. Better comprehenders may have increased availability of subordinate meanings, perhaps due to higher quality lexical representations and reflected in a positive correlation between vocabulary size and comprehension success.

Rhythmic Temporal Expectation Boosts Neural Activity by Increasing Neural Gain

Temporal orienting improves sensory processing, akin to other top-down biases. However, it is unknown whether these improvements reflect increased neural gain to any stimuli presented at expected time points, or specific tuning to task-relevant stimulus aspects. Furthermore, while other top-down biases are selective, the extent of trade-offs across time is less well characterized. Here, we tested whether gain and/or tuning of auditory frequency processing in humans is modulated by rhythmic temporal expectations, and whether these modulations are specific to time points relevant for task performance. Healthy participants (N = 23) of either sex performed an auditory discrimination task while their brain activity was measured using magnetoencephalography/electroencephalography (M/EEG). Acoustic stimulation consisted of sequences of brief distractors interspersed with targets, presented in a rhythmic or jittered way. Target rhythmicity not only improved behavioral discrimination accuracy and M/EEG-based decoding of targets, but also of irrelevant distractors preceding these targets. To explain this finding in terms of increased sensitivity and/or sharpened tuning to auditory frequency, we estimated tuning curves based on M/EEG decoding results, with separate parameters describing gain and sharpness. The effect of rhythmic expectation on distractor decoding was linked to gain increase only, suggesting increased neural sensitivity to any stimuli presented at relevant time points.SIGNIFICANCE STATEMENT Being able to predict when an event may happen can improve perception and action related to this event, likely due to the alignment of neural activity to the temporal structure of stimulus streams. However, it is unclear whether rhythmic increases in neural sensitivity are specific to task-relevant targets, and whether they competitively impair stimulus processing at unexpected time points. By combining magnetoencephalography and encephalographic recordings, neural decoding of auditory stimulus features, and modeling, we found that rhythmic expectation improved neural decoding of both relevant targets and irrelevant distractors presented and expected time points, but did not competitively impair stimulus processing at unexpected time points. Using a quantitative model, these results were linked to nonspecific neural gain increases due to rhythmic expectation.

Neural dynamics of semantic composition

Human speech comprehension is remarkable for its immediacy and rapidity. The listener interprets an incrementally delivered auditory input, millisecond by millisecond as it is heard, in terms of complex multilevel representations of relevant linguistic and nonlinguistic knowledge. Central to this process are the neural computations involved in semantic combination, whereby the meanings of words are combined into more complex representations, as in the combination of a verb and its following direct object (DO) noun (e.g., "eat the apple"). These combinatorial processes form the backbone for incremental interpretation, enabling listeners to integrate the meaning of each word as it is heard into their dynamic interpretation of the current utterance. Focusing on the verb-DO noun relationship in simple spoken sentences, we applied multivariate pattern analysis and computational semantic modeling to source-localized electro/magnetoencephalographic data to map out the specific representational constraints that are constructed as each word is heard, and to determine how these constraints guide the interpretation of subsequent words in the utterance. Comparing context-independent semantic models of the DO noun with contextually constrained noun models reflecting the semantic properties of the preceding verb, we found that only the contextually constrained model showed a significant fit to the brain data. Pattern-based measures of directed connectivity across the left hemisphere language network revealed a continuous information flow among temporal, inferior frontal, and inferior parietal regions, underpinning the verb's modification of the DO noun's activated semantics. These results provide a plausible neural substrate for seamless real-time incremental interpretation on the observed millisecond time scales.

Multimodal Integration of M/EEG and f/MRI Data in SPM12

We describe the steps involved in analysis of multi-modal, multi-subject human neuroimaging data using the SPM12 free and open source software (https://www.fil.ion.ucl.ac.uk/spm/) and a publically-available dataset organized according to the Brain Imaging Data Structure (BIDS) format (https://openneuro.org/datasets/ds000117/). The dataset contains electroencephalographic (EEG), magnetoencephalographic (MEG), and functional and structural magnetic resonance imaging (MRI) data from 16 subjects who undertook multiple runs of a simple task performed on a large number of famous, unfamiliar and scrambled faces. We demonstrate: (1) batching and scripting of preprocessing of multiple runs/subjects of combined MEG and EEG data, (2) creation of trial-averaged evoked responses, (3) source-reconstruction of the power (induced and evoked) across trials within a time-frequency window around the "N/M170" evoked component, using structural MRI for forward modeling and simultaneous inversion (fusion) of MEG and EEG data, (4) group-based optimisation of spatial priors during M/EEG source reconstruction using fMRI data on the same paradigm, and (5) statistical mapping across subjects of cortical source power increases for faces vs. scrambled faces.

The inferior occipital gyrus is a major cortical source of the face-evoked N170: Evidence from simultaneous scalp and intracerebral human recordings

The sudden onset of a face image leads to a prominent face-selective response in human scalp electroencephalographic (EEG) recordings, peaking 170 ms after stimulus onset at occipito-temporal (OT) scalp sites: the N170 (or M170 in magnetoencephalography). According to a widely held view, the main cortical source of the N170 lies in the fusiform gyrus (FG), whereas the posteriorly located inferior occipital gyrus (IOG) would rather generate earlier face-selective responses. Here, we report neural responses to upright and inverted faces recorded in a unique patient using multicontact intracerebral electrodes implanted in the right IOG and in the OT sulcus above the right lateral FG (LFG). Simultaneous EEG recordings on the scalp identified the N170 over the right OT scalp region. The latency and amplitude of this scalp N170 were correlated at the single-trial level with the N170 recorded in the lateral IOG, close to the scalp lateral occipital surface. In addition, a positive component maximal around the latency of the N170 (a P170) was prominent above the internal LFG, whereas this region typically generates an N170 (or "N200") over its external/ventral surface. This suggests that electrophysiological responses in the LFG manifest as an equivalent dipole oriented mostly along the vertical axis with likely minimal projection to the lateral OT scalp region. Altogether, these observations provide evidence that the IOG is a major cortical generator of the face-selective scalp N170, qualifying the potential contribution of the FG and questioning a strict serial spatiotemporal organization of the human cortical face network.

Differential temporal dynamics during visual imagery and perception

Visual perception and imagery rely on similar representations in the visual cortex. During perception, visual activity is characterized by distinct processing stages, but the temporal dynamics underlying imagery remain unclear. Here, we investigated the dynamics of visual imagery in human participants using magnetoencephalography. Firstly, we show that, compared to perception, imagery decoding becomes significant later and representations at the start of imagery already overlap with later time points. This suggests that during imagery, the entire visual representation is activated at once or that there are large differences in the timing of imagery between trials. Secondly, we found consistent overlap between imagery and perceptual processing around 160 ms and from 300 ms after stimulus onset. This indicates that the N170 gets reactivated during imagery and that imagery does not rely on early perceptual representations. Together, these results provide important insights for our understanding of the neural mechanisms of visual imagery.

Mapping Brain Activity with Electrocorticography: Resolution Properties and Robustness of Inverse Solutions

Electrocorticography (ECoG) is an electrophysiological technique that records brain activity directly from the cortical surface with high temporal (ms) and spatial (mm) resolution. Its major limitations are in the high invasiveness and in the restricted field-of-view of the electrode grid, which partially covers the cortex. To infer brain activity at locations different from just below the electrodes, it is necessary to solve the electromagnetic inverse problem. Limitations in the performance of source reconstruction algorithms from ECoG have been, to date, only partially addressed in the literature, and a systematic evaluation is still lacking. The main goal of this study is to provide a quantitative evaluation of resolution properties of widely used inverse methods (eLORETA and MNE) for various ECoG grid sizes, in terms of localization error, spatial dispersion, and overall amplitude. Additionally, this study aims at evaluating how the use of simultaneous electroencephalography (EEG) affects the above properties. For these purposes, we take advantage of a unique dataset in which a monkey underwent a simultaneous recording with a 128 channel ECoG grid and an 18 channel EEG grid. Our results show that, in general conditions, the reconstruction of cortical activity located more than 1 cm away from the ECoG grid is not accurate, since the localization error increases linearly with the distance from the electrodes. This problem can be partially overcome by recording simultaneously ECoG and EEG. However, this analysis enlightens the necessity to design inverse algorithms specifically targeted at taking into account the limited field-of-view of the ECoG grid.

Evidence for causal top-down frontal contributions to predictive processes in speech perception

Perception relies on the integration of sensory information and prior expectations. Here we show that selective neurodegeneration of human frontal speech regions results in delayed reconciliation of predictions in temporal cortex. These temporal regions were not atrophic, displayed normal evoked magnetic and electrical power, and preserved neural sensitivity to manipulations of sensory detail. Frontal neurodegeneration does not prevent the perceptual effects of contextual information; instead, prior expectations are applied inflexibly. The precision of predictions correlates with beta power, in line with theoretical models of the neural instantiation of predictive coding. Fronto-temporal interactions are enhanced while participants reconcile prior predictions with degraded sensory signals. Excessively precise predictions can explain several challenging phenomena in frontal aphasias, including agrammatism and subjective difficulties with speech perception. This work demonstrates that higher-level frontal mechanisms for cognitive and behavioural flexibility make a causal functional contribution to the hierarchical generative models underlying speech perception.

Choice of Magnetometers and Gradiometers after Signal Space Separation

BACKGROUND

Modern Elekta Neuromag MEG devices include 102 sensor triplets containing one magnetometer and two planar gradiometers. The first processing step is often a signal space separation (SSS), which provides a powerful noise reduction. A question commonly raised by researchers and reviewers relates to which data should be employed in analyses: (1) magnetometers only, (2) gradiometers only, (3) magnetometers and gradiometers together. The MEG community is currently divided with regard to the proper answer.

METHODS

First, we provide theoretical evidence that both gradiometers and magnetometers result from the backprojection of the same SSS components. Then, we compare resting state and task-related sensor and source estimations from magnetometers and gradiometers in real MEG recordings before and after SSS.

RESULTS

SSS introduced a strong increase in the similarity between source time series derived from magnetometers and gradiometers (r² = 0.3-0.8 before SSS and r² > 0.80 after SSS). After SSS, resting state power spectrum and functional connectivity, as well as visual evoked responses, derived from both magnetometers and gradiometers were highly similar (Intraclass Correlation Coefficient > 0.8, r² > 0.8).

CONCLUSIONS

After SSS, magnetometer and gradiometer data are estimated from a single set of SSS components (usually ≤ 80). Equivalent results can be obtained with both sensor types in typical MEG experiments.

Reproducibility of EEG-MEG fusion source analysis of interictal spikes: Relevance in presurgical evaluation of epilepsy

Fusion of electroencephalography (EEG) and magnetoencephalography (MEG) data using maximum entropy on the mean method (MEM-fusion) takes advantage of the complementarities between EEG and MEG to improve localization accuracy. Simulation studies demonstrated MEM-fusion to be robust especially in noisy conditions such as single spike source localizations (SSSL). Our objective was to assess the reliability of SSSL using MEM-fusion on clinical data. We proposed to cluster SSSL results to find the most reliable and consistent source map from the reconstructed sources, the so-called consensus map. Thirty-four types of interictal epileptic discharges (IEDs) were analyzed from 26 patients with well-defined epileptogenic focus. SSSLs were performed on EEG, MEG, and fusion data and consensus maps were estimated using hierarchical clustering. Qualitative (spike-to-spike reproducibility rate, SSR) and quantitative (localization error and spatial dispersion) assessments were performed using the epileptogenic focus as clinical reference. Fusion SSSL provided significantly better results than EEG or MEG alone. Fusion found at least one cluster concordant with the clinical reference in all cases. This concordant cluster was always the one involving the highest number of spikes. Fusion yielded highest reproducibility (SSR EEG = 55%, MEG = 71%, fusion = 90%) and lowest localization error. Also, using only few channels from either modality (21EEG + 272MEG or 54EEG + 25MEG) was sufficient to reach accurate fusion. MEM-fusion with consensus map approach provides an objective way of finding the most reliable and concordant generators of IEDs. We, therefore, suggest the pertinence of SSSL using MEM-fusion as a valuable clinical tool for presurgical evaluation of epilepsy.

Adaptive cortical parcellations for source reconstructed EEG/MEG connectomes

There is growing interest in the rich temporal and spectral properties of the functional connectome of the brain that are provided by Electro- and Magnetoencephalography (EEG/MEG). However, the problem of leakage between brain sources that arises when reconstructing brain activity from EEG/MEG recordings outside the head makes it difficult to distinguish true connections from spurious connections, even when connections are based on measures that ignore zero-lag dependencies. In particular, standard anatomical parcellations for potential cortical sources tend to over- or under-sample the real spatial resolution of EEG/MEG. By using information from cross-talk functions (CTFs) that objectively describe leakage for a given sensor configuration and distributed source reconstruction method, we introduce methods for optimising the number of parcels while simultaneously minimising the leakage between them. More specifically, we compare two image segmentation algorithms: 1) a split-and-merge (SaM) algorithm based on standard anatomical parcellations and 2) a region growing (RG) algorithm based on all the brain vertices with no prior parcellation. Interestingly, when applied to minimum-norm reconstructions for EEG/MEG configurations from real data, both algorithms yielded approximately 70 parcels despite their different starting points, suggesting that this reflects the resolution limit of this particular sensor configuration and reconstruction method. Importantly, when compared against standard anatomical parcellations, resolution matrices of adaptive parcellations showed notably higher sensitivity and distinguishability of parcels. Furthermore, extensive simulations of realistic networks revealed significant improvements in network reconstruction accuracies, particularly in reducing false leakage-induced connections. Adaptive parcellations therefore allow a more accurate reconstruction of functional EEG/MEG connectomes.

•

We introduce adaptive cortical parcellation algorithms for E/MEG source estimation.

•

Optimum number, size and locations of parcels are found based on cross-talk functions

•

Algorithms yielded ∼70 distinguishable parcels regardless of the starting point.

•

Parcel resolution matrices were notably improved compared to anatomical atlases.

•

Network reconstruction accuracies of simulated connectomes improved significantly.

Backward renormalization-group inference of cortical dipole sources and neural connectivity efficacy

Proper neural connectivity inference has become essential for understanding cognitive processes associated with human brain function. Its efficacy is often hampered by the curse of dimensionality. In the electroencephalogram case, which is a noninvasive electrophysiological monitoring technique to record electrical activity of the brain, a possible way around this is to replace multichannel electrode information with dipole reconstructed data. We use a method based on maximum entropy and the renormalization group to infer the position of the sources, whose success hinges on transmitting information from low- to high-resolution representations of the cortex. The performance of this method compares favorably to other available source inference algorithms, which are ranked here in terms of their performance with respect to directed connectivity inference by using artificially generated dynamic data. We examine some representative scenarios comprising different numbers of dynamically connected dipoles over distinct cortical surface positions and under different sensor noise impairment levels. The overall conclusion is that inverse problem solutions do not affect the correct inference of the direction of the flow of information as long as the equivalent dipole sources are correctly found.

A Review of Issues Related to Data Acquisition and Analysis in EEG/MEG Studies

Electroencephalography (EEG) and magnetoencephalography (MEG) are non-invasive electrophysiological methods, which record electric potentials and magnetic fields due to electric currents in synchronously-active neurons. With MEG being more sensitive to neural activity from tangential currents and EEG being able to detect both radial and tangential sources, the two methods are complementary. Over the years, neurophysiological studies have changed considerably: high-density recordings are becoming de rigueur; there is interest in both spontaneous and evoked activity; and sophisticated artifact detection and removal methods are available. Improved head models for source estimation have also increased the precision of the current estimates, particularly for EEG and combined EEG/MEG. Because of their complementarity, more investigators are beginning to perform simultaneous EEG/MEG studies to gain more complete information about neural activity. Given the increase in methodological complexity in EEG/MEG, it is important to gather data that are of high quality and that are as artifact free as possible. Here, we discuss some issues in data acquisition and analysis of EEG and MEG data. Practical considerations for different types of EEG and MEG studies are also discussed.

Processing of gaze direction within the N170/M170 time window: A combined EEG/MEG study

Gaze direction is an important social signal for human beings. Beside the role of gaze in attention orienting, direct gaze (that is, gaze directed toward an observer) is a highly relevant biological stimulus that elicits attention capture and increases face encoding. Brain imaging studies have emphasized the role of the superior temporal sulcus (STS) in the coding of gaze direction and in the integration of gaze and head cues of social attention. The dynamics of the processing and integration of these cues remains, however, unclear. In order to address this question, we used deviated and frontal faces with averted and direct gaze in a combined electro- and magneto- encephalography (EEG-MEG) study. We showed distinct effects of gaze direction on the N170 and M170 responses. There was an interaction between gaze direction and head orientation between 134 and 162ms in MEG and a main effect of gaze direction between 171 and 186ms in EEG. These effects involved the posterior and anterior regions of the STS respectively. Both effects also emphasized the sensitivity to direct gaze. These data highlight the central role of the STS in gaze processing.

A Spiking Neurocomputational Model of High-Frequency Oscillatory Brain Responses to Words and Pseudowords

Experimental evidence indicates that neurophysiological responses to well-known meaningful sensory items and symbols (such as familiar objects, faces, or words) differ from those to matched but novel and senseless materials (unknown objects, scrambled faces, and pseudowords). Spectral responses in the high beta- and gamma-band have been observed to be generally stronger to familiar stimuli than to unfamiliar ones. These differences have been hypothesized to be caused by the activation of distributed neuronal circuits or cell assemblies, which act as long-term memory traces for learned familiar items only. Here, we simulated word learning using a biologically constrained neurocomputational model of the left-hemispheric cortical areas known to be relevant for language and conceptual processing. The 12-area spiking neural-network architecture implemented replicates physiological and connectivity features of primary, secondary, and higher-association cortices in the frontal, temporal, and occipital lobes of the human brain. We simulated elementary aspects of word learning in it, focussing specifically on semantic grounding in action and perception. As a result of spike-driven Hebbian synaptic plasticity mechanisms, distributed, stimulus-specific cell-assembly (CA) circuits spontaneously emerged in the network. After training, presentation of one of the learned "word" forms to the model correlate of primary auditory cortex induced periodic bursts of activity within the corresponding CA, leading to oscillatory phenomena in the entire network and spontaneous across-area neural synchronization. Crucially, Morlet wavelet analysis of the network's responses recorded during presentation of learned meaningful "word" and novel, senseless "pseudoword" patterns revealed stronger induced spectral power in the gamma-band for the former than the latter, closely mirroring differences found in neurophysiological data. Furthermore, coherence analysis of the simulated responses uncovered dissociated category specific patterns of synchronous oscillations in distant cortical areas, including indirectly connected primary sensorimotor areas. Bridging the gap between cellular-level mechanisms, neuronal-population behavior, and cognitive function, the present model constitutes the first spiking, neurobiologically, and anatomically realistic model able to explain high-frequency oscillatory phenomena indexing language processing on the basis of dynamics and competitive interactions of distributed cell-assembly circuits which emerge in the brain as a result of Hebbian learning and sensorimotor experience.

Spatio-temporal reconstruction of brain dynamics from EEG with a Markov prior

Electroencephalography (EEG) can capture brain dynamics in high temporal resolution. By projecting the scalp EEG signal back to its origin in the brain also high spatial resolution can be achieved. Source localized EEG therefore has potential to be a very powerful tool for understanding the functional dynamics of the brain. Solving the inverse problem of EEG is however highly ill-posed as there are many more potential locations of the EEG generators than EEG measurement points. Several well-known properties of brain dynamics can be exploited to alleviate this problem. More short ranging connections exist in the brain than long ranging, arguing for spatially focal sources. Additionally, recent work (Delorme et al., 2012) argues that EEG can be decomposed into components having sparse source distributions. On the temporal side both short and long term stationarity of brain activation are seen. We summarize these insights in an inverse solver, the so-called "Variational Garrote" (Kappen and Gómez, 2013). Using a Markov prior we can incorporate flexible degrees of temporal stationarity. Through spatial basis functions spatially smooth distributions are obtained. Sparsity of these are inherent to the Variational Garrote solver. We name our method the MarkoVG and demonstrate its ability to adapt to the temporal smoothness and spatial sparsity in simulated EEG data. Finally a benchmark EEG dataset is used to demonstrate MarkoVG's ability to recover non-stationary brain dynamics.

Perceptual learning of degraded speech by minimizing prediction error

Human perception is shaped by past experience on multiple timescales. Sudden and dramatic changes in perception occur when prior knowledge or expectations match stimulus content. These immediate effects contrast with the longer-term, more gradual improvements that are characteristic of perceptual learning. Despite extensive investigation of these two experience-dependent phenomena, there is considerable debate about whether they result from common or dissociable neural mechanisms. Here we test single- and dual-mechanism accounts of experience-dependent changes in perception using concurrent magnetoencephalographic and EEG recordings of neural responses evoked by degraded speech. When speech clarity was enhanced by prior knowledge obtained from matching text, we observed reduced neural activity in a peri-auditory region of the superior temporal gyrus (STG). Critically, longer-term improvements in the accuracy of speech recognition following perceptual learning resulted in reduced activity in a nearly identical STG region. Moreover, short-term neural changes caused by prior knowledge and longer-term neural changes arising from perceptual learning were correlated across subjects with the magnitude of learning-induced changes in recognition accuracy. These experience-dependent effects on neural processing could be dissociated from the neural effect of hearing physically clearer speech, which similarly enhanced perception but increased rather than decreased STG responses. Hence, the observed neural effects of prior knowledge and perceptual learning cannot be attributed to epiphenomenal changes in listening effort that accompany enhanced perception. Instead, our results support a predictive coding account of speech perception; computational simulations show how a single mechanism, minimization of prediction error, can drive immediate perceptual effects of prior knowledge and longer-term perceptual learning of degraded speech.

Resource efficient data compression algorithms for demanding, WSN based biomedical applications

During the last few years, medical research areas of critical importance such as Epilepsy monitoring and study, increasingly utilize wireless sensor network technologies in order to achieve better understanding and significant breakthroughs. However, the limited memory and communication bandwidth offered by WSN platforms comprise a significant shortcoming to such demanding application scenarios. Although, data compression can mitigate such deficiencies there is a lack of objective and comprehensive evaluation of relative approaches and even more on specialized approaches targeting specific demanding applications. The research work presented in this paper focuses on implementing and offering an in-depth experimental study regarding prominent, already existing as well as novel proposed compression algorithms. All algorithms have been implemented in a common Matlab framework. A major contribution of this paper, that differentiates it from similar research efforts, is the employment of real world Electroencephalography (EEG) and Electrocardiography (ECG) datasets comprising the two most demanding Epilepsy modalities. Emphasis is put on WSN applications, thus the respective metrics focus on compression rate and execution latency for the selected datasets. The evaluation results reveal significant performance and behavioral characteristics of the algorithms related to their complexity and the relative negative effect on compression latency as opposed to the increased compression rate. It is noted that the proposed schemes managed to offer considerable advantage especially aiming to achieve the optimum tradeoff between compression rate-latency. Specifically, proposed algorithm managed to combine highly completive level of compression while ensuring minimum latency thus exhibiting real-time capabilities. Additionally, one of the proposed schemes is compared against state-of-the-art general-purpose compression algorithms also exhibiting considerable advantages as far as the compression rate is concerned.

EEG-MEG Integration Enhances the Characterization of Functional and Effective Connectivity in the Resting State Network

At the sensor level many aspects, such as spectral power, functional and effective connectivity as well as relative-power-ratio ratio (RPR) and spatial resolution have been comprehensively investigated through both electroencephalography (EEG) and magnetoencephalography (MEG). Despite this, differences between both modalities have not yet been systematically studied by direct comparison. It remains an open question as to whether the integration of EEG and MEG data would improve the information obtained from the above mentioned parameters. Here, EEG (64-channel system) and MEG (275 sensor system) were recorded simultaneously in conditions with eyes open (EO) and eyes closed (EC) in 29 healthy adults. Spectral power, functional and effective connectivity, RPR, and spatial resolution were analyzed at five different frequency bands (delta, theta, alpha, beta and gamma). Networks of functional and effective connectivity were described using a spatial filter approach called the dynamic imaging of coherent sources (DICS) followed by the renormalized partial directed coherence (RPDC). Absolute mean power at the sensor level was significantly higher in EEG than in MEG data in both EO and EC conditions. At the source level, there was a trend towards a better performance of the combined EEG+MEG analysis compared with separate EEG or MEG analyses for the source mean power, functional correlation, effective connectivity for both EO and EC. The network of coherent sources and the spatial resolution were similar for both the EEG and MEG data if they were analyzed separately. Results indicate that the combined approach has several advantages over the separate analyses of both EEG and MEG. Moreover, by a direct comparison of EEG and MEG, EEG was characterized by significantly higher values in all measured parameters in both sensor and source level. All the above conclusions are specific to the resting state task and the specific analysis used in this study to have general conclusion multi-center studies would be helpful.

Brain Network Connectivity During Language Comprehension: Interacting Linguistic and Perceptual Subsystems

The dynamic neural processes underlying spoken language comprehension require the real-time integration of general perceptual and specialized linguistic information. We recorded combined electro- and magnetoencephalographic measurements of participants listening to spoken words varying in perceptual and linguistic complexity. Combinatorial linguistic complexity processing was consistently localized to left perisylvian cortices, whereas competition-based perceptual complexity triggered distributed activity over both hemispheres. Functional connectivity showed that linguistically complex words engaged a distributed network of oscillations in the gamma band (20-60 Hz), which only partially overlapped with the network supporting perceptual analysis. Both processes enhanced cross-talk between left temporal regions and bilateral pars orbitalis (BA47). The left-lateralized synchrony between temporal regions and pars opercularis (BA44) was specific to the linguistically complex words, suggesting a specific role of left frontotemporal cross-cortical interactions in morphosyntactic computations. Synchronizations in oscillatory dynamics reveal the transient coupling of functional networks that support specific computational processes in language comprehension.

Predicting the Time Course of Individual Objects with MEG

To respond appropriately to objects, we must process visual inputs rapidly and assign them meaning. This involves highly dynamic, interactive neural processes through which information accumulates and cognitive operations are resolved across multiple time scales. However, there is currently no model of object recognition which provides an integrated account of how visual and semantic information emerge over time; therefore, it remains unknown how and when semantic representations are evoked from visual inputs. Here, we test whether a model of individual objects--based on combining the HMax computational model of vision with semantic-feature information--can account for and predict time-varying neural activity recorded with magnetoencephalography. We show that combining HMax and semantic properties provides a better account of neural object representations compared with the HMax alone, both through model fit and classification performance. Our results show that modeling and classifying individual objects is significantly improved by adding semantic-feature information beyond ∼200 ms. These results provide important insights into the functional properties of visual processing across time.

Interacting parallel pathways associate sounds with visual identity in auditory cortices

Spatial and non-spatial information of sound events is presumably processed in parallel auditory cortex (AC) "what" and "where" streams, which are modulated by inputs from the respective visual-cortex subsystems. How these parallel processes are integrated to perceptual objects that remain stable across time and the source agent's movements is unknown. We recorded magneto- and electroencephalography (MEG/EEG) data while subjects viewed animated video clips featuring two audiovisual objects, a black cat and a gray cat. Adaptor-probe events were either linked to the same object (the black cat meowed twice in a row in the same location) or included a visually conveyed identity change (the black and then the gray cat meowed with identical voices in the same location). In addition to effects in visual (including fusiform, middle temporal or MT areas) and frontoparietal association areas, the visually conveyed object-identity change was associated with a release from adaptation of early (50-150ms) activity in posterior ACs, spreading to left anterior ACs at 250-450ms in our combined MEG/EEG source estimates. Repetition of events belonging to the same object resulted in increased theta-band (4-8Hz) synchronization within the "what" and "where" pathways (e.g., between anterior AC and fusiform areas). In contrast, the visually conveyed identity changes resulted in distributed synchronization at higher frequencies (alpha and beta bands, 8-32Hz) across different auditory, visual, and association areas. The results suggest that sound events become initially linked to perceptual objects in posterior AC, followed by modulations of representations in anterior AC. Hierarchical what and where pathways seem to operate in parallel after repeating audiovisual associations, whereas the resetting of such associations engages a distributed network across auditory, visual, and multisensory areas.

Anticipation of electric shocks modulates low beta power and event-related fields during memory encoding

In humans, the temporal and oscillatory dynamics of pain anticipation and its effects on long-term memory are largely unknown. Here, we investigated this open question by using a previously established behavioral paradigm in combination with magnetoencephalography (MEG). Healthy human subjects encoded a series of scene images, which was combined with cues predicting an aversive electric shock with different probabilities (0.2, 0.5 or 0.8). After encoding, memory for the studied images was tested using a remember/know recognition task. Behaviorally, pain anticipation did not modulate recollection-based recognition memory per se, but interacted with the perceived unpleasantness of the electric shock [visual analogue scale rating from 1 (not unpleasant) to 10 (highly unpleasant)]. More precisely, the relationship between pain anticipation and recollection followed an inverted u-shaped function the more unpleasant the shocks were rated by a subject. At the physiological level, this quadratic effect was mimicked in the event-related magnetic fields associated with successful memory formation ('DM-effect') ∼450ms after image onset at left frontal sensors. Importantly, across all subjects, shock anticipation modulated oscillatory power in the low beta frequency range (13-20Hz) in a linear fashion at left temporal sensors. Taken together, our findings indicate that beta oscillations provide a generic mechanism underlying pain anticipation; the effect on subsequent long-term memory, on the other hand, is much more variable and depends on the level of individual pain perception. As such, our findings give new and important insights into how aversive motivational states can drive memory formation.

MEG-EEG Information Fusion and Electromagnetic Source Imaging: From Theory to Clinical Application in Epilepsy

The purpose of this study is to develop and quantitatively assess whether fusion of EEG and MEG (MEEG) data within the maximum entropy on the mean (MEM) framework increases the spatial accuracy of source localization, by yielding better recovery of the spatial extent and propagation pathway of the underlying generators of inter-ictal epileptic discharges (IEDs). The key element in this study is the integration of the complementary information from EEG and MEG data within the MEM framework. MEEG was compared with EEG and MEG when localizing single transient IEDs. The fusion approach was evaluated using realistic simulation models involving one or two spatially extended sources mimicking propagation patterns of IEDs. We also assessed the impact of the number of EEG electrodes required for an efficient EEG–MEG fusion. MEM was compared with minimum norm estimate, dynamic statistical parametric mapping, and standardized low-resolution electromagnetic tomography. The fusion approach was finally assessed on real epileptic data recorded from two patients showing IEDs simultaneously in EEG and MEG. Overall the localization of MEEG data using MEM provided better recovery of the source spatial extent, more sensitivity to the source depth and more accurate detection of the onset and propagation of IEDs than EEG or MEG alone. MEM was more accurate than the other methods. MEEG proved more robust than EEG and MEG for single IED localization in low signal-to-noise ratio conditions. We also showed that only few EEG electrodes are required to bring additional relevant information to MEG during MEM fusion.

Early Visual Word Processing Is Flexible: Evidence from Spatiotemporal Brain Dynamics

Visual word recognition is often described as automatic, but the functional locus of top-down effects is still a matter of debate. Do task demands modulate how information is retrieved, or only how it is used? We used EEG/MEG recordings to assess whether, when, and how task contexts modify early retrieval of specific psycholinguistic information in occipitotemporal cortex, an area likely to contribute to early stages of visual word processing. Using a parametric approach, we analyzed the spatiotemporal response patterns of occipitotemporal cortex for orthographic, lexical, and semantic variables in three psycholinguistic tasks: silent reading, lexical decision, and semantic decision. Task modulation of word frequency and imageability effects occurred simultaneously in ventral occipitotemporal regions-in the vicinity of the putative visual word form area-around 160 msec, following task effects on orthographic typicality around 100 msec. Frequency and typicality also produced task-independent effects in anterior temporal lobe regions after 200 msec. The early task modulation for several specific psycholinguistic variables indicates that occipitotemporal areas integrate perceptual input with prior knowledge in a task-dependent manner. Still, later task-independent effects in anterior temporal lobes suggest that word recognition eventually leads to retrieval of semantic information irrespective of task demands. We conclude that even a highly overlearned visual task like word recognition should be described as flexible rather than automatic.