Localizing Event-Related Potentials Using Multi-source Minimum Variance Beamformers: A Validation Study

XDL-ESI: Electrophysiological Sources Imaging via explainable deep learning framework with validation on simultaneous EEG and iEEG

Electroencephalography (EEG) or Magnetoencephalography (MEG) source imaging aims to estimate the underlying activated brain sources to explain the observed EEG/MEG recordings. Solving the inverse problem of EEG/MEG Source Imaging (ESI) is challenging due to its ill-posed nature. To achieve a unique solution, it is essential to apply sophisticated regularization constraints to restrict the solution space. Traditionally, the design of regularization terms is based on assumptions about the spatiotemporal structure of the underlying source dynamics. In this paper, we propose a novel paradigm for ESI via an Explainable Deep Learning framework, termed as XDL-ESI, which connects the iterative optimization algorithm with deep learning architecture by unfolding the iterative updates with neural network modules. The proposed framework has the advantages of (1) establishing a data-driven approach to model the source solution structure instead of using hand-crafted regularization terms; (2) improving the robustness of source solutions by introducing a topological loss that leverages the geometric spatial information applying varying penalties on distinct localization errors; (3) improving the reconstruction efficiency and interpretability as it inherits the advantages from both the iterative optimization algorithms (interpretability) and deep learning approaches (function approximation). The proposed XDL-ESI framework provides an efficient, accurate, and interpretable paradigm to solve the ESI inverse problem with satisfactory performance in both simulated data and real clinical data. Specially, this approach is further validated using simultaneous EEG and intracranial EEG (iEEG).

Multi-Modal Electrophysiological Source Imaging With Attention Neural Networks Based on Deep Fusion of EEG and MEG

The process of reconstructing underlying cortical and subcortical electrical activities from Electroencephalography (EEG) or Magnetoencephalography (MEG) recordings is called Electrophysiological Source Imaging (ESI). Given the complementarity between EEG and MEG in measuring radial and tangential cortical sources, combined EEG/MEG is considered beneficial in improving the reconstruction performance of ESI algorithms. Traditional algorithms mainly emphasize incorporating predesigned neurophysiological priors to solve the ESI problem. Deep learning frameworks aim to directly learn the mapping from scalp EEG/MEG measurements to the underlying brain source activities in a data-driven manner, demonstrating superior performance compared to traditional methods. However, most of the existing deep learning approaches for the ESI problem are performed on a single modality of EEG or MEG, meaning the complementarity of these two modalities has not been fully utilized. How to fuse the EEG and MEG in a more principled manner under the deep learning paradigm remains a challenging question. This study develops a Multi-Modal Deep Fusion (MMDF) framework using Attention Neural Networks (ANN) to fully leverage the complementary information between EEG and MEG for solving the ESI inverse problem, which is termed as MMDF-ANN. Specifically, our proposed brain source imaging approach consists of four phases, including feature extraction, weight generation, deep feature fusion, and source mapping. Our experimental results on both synthetic dataset and real dataset demonstrated that using a fusion of EEG and MEG can significantly improve the source localization accuracy compared to using a single-modality of EEG or MEG. Compared to the benchmark algorithms, MMDF-ANN demonstrated good stability when reconstructing sources with extended activation areas and situations of EEG/MEG measurements with a low signal-to-noise ratio.

Comparison of beamformer implementations for MEG source localization

Beamformers are applied for estimating spatiotemporal characteristics of neuronal sources underlying measured MEG/EEG signals. Several MEG analysis toolboxes include an implementation of a linearly constrained minimum-variance (LCMV) beamformer. However, differences in implementations and in their results complicate the selection and application of beamformers and may hinder their wider adoption in research and clinical use. Additionally, combinations of different MEG sensor types (such as magnetometers and planar gradiometers) and application of preprocessing methods for interference suppression, such as signal space separation (SSS), can affect the results in different ways for different implementations. So far, a systematic evaluation of the different implementations has not been performed. Here, we compared the localization performance of the LCMV beamformer pipelines in four widely used open-source toolboxes (MNE-Python, FieldTrip, DAiSS (SPM12), and Brainstorm) using datasets both with and without SSS interference suppression.

We analyzed MEG data that were i) simulated, ii) recorded from a static and moving phantom, and iii) recorded from a healthy volunteer receiving auditory, visual, and somatosensory stimulation. We also investigated the effects of SSS and the combination of the magnetometer and gradiometer signals. We quantified how localization error and point-spread volume vary with the signal-to-noise ratio (SNR) in all four toolboxes.

When applied carefully to MEG data with a typical SNR (3–15 ​dB), all four toolboxes localized the sources reliably; however, they differed in their sensitivity to preprocessing parameters. As expected, localizations were highly unreliable at very low SNR, but we found high localization error also at very high SNRs for the first three toolboxes while Brainstorm showed greater robustness but with lower spatial resolution. We also found that the SNR improvement offered by SSS led to more accurate localization.

•

Different beamformer implementations are reported to sometimes yield differing source estimates for the same MEG data.

•

We compared beamformers in four major open-source MEG analysis toolboxes.

•

All toolboxes provide consistent and accurate results with 3–15-dB input SNR.

•

However, localization errors are high at very high input SNR for the tested scalar beamformers.

•

We discuss the critical differences between the implementations.

Mapping neural dynamics underlying saccade preparation and execution and their relation to reaction time and direction errors

Our ability to control and inhibit automatic behaviors is crucial for negotiating complex environments, all of which require rapid communication between sensory, motor, and cognitive networks. Here, we measured neuromagnetic brain activity to investigate the neural timing of cortical areas needed for inhibitory control, while 14 healthy young adults performed an interleaved prosaccade (look at a peripheral visual stimulus) and antisaccade (look away from stimulus) task. Analysis of how neural activity relates to saccade reaction time (SRT) and occurrence of direction errors (look at stimulus on antisaccade trials) provides insight into inhibitory control. Neuromagnetic source activity was used to extract stimulus‐aligned and saccade‐aligned activity to examine temporal differences between prosaccade and antisaccade trials in brain regions associated with saccade control. For stimulus‐aligned antisaccade trials, a longer SRT was associated with delayed onset of neural activity within the ipsilateral parietal eye field (PEF) and bilateral frontal eye field (FEF). Saccade‐aligned activity demonstrated peak activation 10ms before saccade‐onset within the contralateral PEF for prosaccade trials and within the bilateral FEF for antisaccade trials. In addition, failure to inhibit prosaccades on anti‐saccade trials was associated with increased activity prior to saccade onset within the FEF contralateral to the peripheral stimulus. This work on dynamic activity adds to our knowledge that direction errors were due, at least in part, to a failure to inhibit automatic prosaccades. These findings provide novel evidence in humans regarding the temporal dynamics within oculomotor areas needed for saccade programming and the role frontal brain regions have on top‐down inhibitory control.