Single-trial detection in EEG and MEG: Keeping it linear

Representational dissimilarity component analysis (ReDisCA)

The principle of Representational Similarity Analysis (RSA) posits that neural representations reflect the structure of encoded information, allowing exploration of spatial and temporal organization of brain information processing. Traditional RSA when applied to EEG or MEG data faces challenges in accessing activation time series at the brain source level due to modeling complexities and insufficient geometric/anatomical data. To overcome this, we introduce Representational Dissimilarity Component Analysis (ReDisCA), a method for estimating spatial-temporal components in EEG or MEG responses aligned with a target representational dissimilarity matrix (RDM). ReDisCA yields informative spatial filters and associated topographies, offering insights into the location of "representationally relevant" sources. Applied to evoked response time series, ReDisCA produces temporal source activation profiles with the desired RDM. Importantly, while ReDisCA does not require inverse modeling its output is consistent with EEG and MEG observation equation and can be used as an input to rigorous source localization procedures. Demonstrating ReDisCA's efficacy through simulations and comparison with conventional methods, we show superior source localization accuracy and apply the method to real EEG and MEG datasets, revealing physiologically plausible representational structures without inverse modeling. ReDisCA adds to the family of inverse modeling free methods such as independent component analysis (Makeig, 1995), Spatial spectral decomposition (Nikulin, 2011), and Source power comodulation (Dähne, 2014) designed for extraction sources with desired properties from EEG or MEG data. Extending its utility beyond EEG and MEG analysis, ReDisCA is likely to find application in fMRI data analysis and exploration of representational structures emerging in multilayered artificial neural networks.

Dissociation of early and late face-related processes in autism spectrum disorder and Williams syndrome

Background

Williams syndrome (WS) and Autism Spectrum Disorders (ASD) are neurodevelopmental conditions associated with atypical but opposite face-to-face interactions patterns: WS patients overly stare at others, ASD individuals escape eye contact. Whether these behaviors result from dissociable visual processes within the occipito-temporal pathways is unknown.

Using high-density electroencephalography, multivariate signal processing algorithms and a protocol designed to identify and extract evoked activities sensitive to facial cues, we investigated how WS (N = 14), ASD (N = 14) and neurotypical subjects (N = 14) decode the information content of a face stimulus.

Results

We found two neural components in neurotypical participants, both strongest when the eye region was projected onto the subject's fovea, simulating a direct eye contact situation, and weakest over more distant regions, reaching a minimum when the focused region was outside the stimulus face. The first component peaks at 170 ms, an early signal known to be implicated in low-level face features. The second is identified later, 260 ms post-stimulus onset and is implicated in decoding salient face social cues.

Remarkably, both components were found distinctly impaired and preserved in WS and ASD. In WS, we could weakly decode the 170 ms signal based on our regressor relative to facial features, probably due to their relatively poor ability to process faces’ morphology, while the late 260 ms component was highly significant. The reverse pattern was observed in ASD participants who showed neurotypical like early 170 ms evoked activity but impaired late evoked 260 ms signal.

Conclusions

Our study reveals a dissociation between WS and ASD patients and points at different neural origins for their social impairments.

Neural tracking in infants - An analytical tool for multisensory social processing in development

Humans are born into a social environment and from early on possess a range of abilities to detect and respond to social cues. In the past decade, there has been a rapidly increasing interest in investigating the neural responses underlying such early social processes under naturalistic conditions. However, the investigation of neural responses to continuous dynamic input poses the challenge of how to link neural responses back to continuous sensory input. In the present tutorial, we provide a step-by-step introduction to one approach to tackle this issue, namely the use of linear models to investigate neural tracking responses in electroencephalographic (EEG) data. While neural tracking has gained increasing popularity in adult cognitive neuroscience over the past decade, its application to infant EEG is still rare and comes with its own challenges. After introducing the concept of neural tracking, we discuss and compare the use of forward vs. backward models and individual vs. generic models using an example data set of infant EEG data. Each section comprises a theoretical introduction as well as a concrete example using MATLAB code. We argue that neural tracking provides a promising way to investigate early (social) processing in an ecologically valid setting.

Decoding and interpreting cortical signals with a compact convolutional neural network

OBJECTIVE

Brain-computer interfaces (BCIs) decode information from neural activity and send it to external devices. The use of Deep Learning approaches for decoding allows for automatic feature engineering within the specific decoding task. Physiologically plausible interpretation of the network parameters ensures the robustness of the learned decision rules and opens the exciting opportunity for automatic knowledge discovery.

APPROACH

We describe a compact convolutional network-based architecture for adaptive decoding of electrocorticographic (ECoG) data into finger kinematics. We also propose a novel theoretically justified approach to interpreting the spatial and temporal weights in the architectures that combine adaptation in both space and time. The obtained spatial and frequency patterns characterizing the neuronal populations pivotal to the specific decoding task can then be interpreted by fitting appropriate spatial and dynamical models.

MAIN RESULTS

We first tested our solution using realistic Monte-Carlo simulations. Then, when applied to the ECoG data from Berlin BCI competition IV dataset, our architecture performed comparably to the competition winners without requiring explicit feature engineering. Using the proposed approach to the network weights interpretation we could unravel the spatial and the spectral patterns of the neuronal processes underlying the successful decoding of finger kinematics from an ECoG dataset. Finally we have also applied the entire pipeline to the analysis of a 32-channel EEG motor-imagery dataset and observed physiologically plausible patterns specific to the task.

SIGNIFICANCE

We described a compact and interpretable CNN architecture derived from the basic principles and encompassing the knowledge in the field of neural electrophysiology. For the first time in the context of such multibranch architectures with factorized spatial and temporal processing we presented theoretically justified weights interpretation rules. We verified our recipes using simulations and real data and demonstrated that the proposed solution offers a good decoder and a tool for investigating motor control neural mechanisms.

Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. Linear classifiers are widely employed in the brain decoding paradigm to discriminate among experimental conditions. Then, the derived linear weights are visualized in the form of multivariate brain maps to further study spatio-temporal patterns of underlying neural activities. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed definition, we exemplify a heuristic for approximating the interpretability in multivariate analysis of evoked magnetoencephalography (MEG) responses. Third, we propose to combine the approximated interpretability and the generalization performance of the brain decoding into a new multi-objective criterion for model selection. Our results, for the simulated and real MEG data, show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future.

Single-Trial Detection With Magnetoencephalography During a Dual-Rapid Serial Visual Presentation Task

GOAL

The detection of brain responses corresponding to the presentation of a particular class of images is a challenge in brain-machine interface. Current systems based on the detection of brain responses during rapid serial visual presentation (RSVP) tasks possess advantages for both healthy and disabled people, as they are gaze independent and can offer a high throughput.

METHODS

We propose a novel paradigm based on a dual-RSVP task that assumes a low target probability. Two streams of images are presented simultaneously on the screen, the second stream is identical to the first one, but delayed in time. Participants were asked to detect images containing a person. They follow the first stream until they see a target image, then change their attention to the second stream until the target image reappears, finally they change their attention back to the first stream.

RESULTS

The performance of single-trial detection was evaluated on both streams and their combination of the decisions with signal recorded with magnetoencephalography (MEG) during the dual-RSVP task. We compare classification performance across different sets of channels (magnetometers, gradiometers) with a BLDA classifier with inputs obtained after spatial filtering.

CONCLUSION

The results suggest that single-trial detection can be obtained with an area under the ROC curve superior to 0.95, and that an almost perfect accuracy can be obtained with some subjects thanks to the combination of the decisions from two trials, without doubling the duration of the experiment.

SIGNIFICANCE

The present results show that a reliable accuracy can be obtained with the MEG for target detection during a dual-RSVP task.

Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering

Accurate detection of single-trial event-related potentials (ERPs) in the electroencephalogram (EEG) is a difficult problem that requires efficient signal processing and machine learning techniques. Supervised spatial filtering methods that enhance the discriminative information in EEG data are commonly used to improve single-trial ERP detection. We propose a convolutional neural network (CNN) with a layer dedicated to spatial filtering for the detection of ERPs and with training based on the maximization of the area under the receiver operating characteristic curve (AUC). The CNN is compared with three common classifiers: 1) Bayesian linear discriminant analysis; 2) multilayer perceptron (MLP); and 3) support vector machines. Prior to classification, the data were spatially filtered with xDAWN (for the maximization of the signal-to-signal-plus-noise ratio), common spatial pattern, or not spatially filtered. The 12 analytical techniques were tested on EEG data recorded in three rapid serial visual presentation experiments that required the observer to discriminate rare target stimuli from frequent nontarget stimuli. Classification performance discriminating targets from nontargets depended on both the spatial filtering method and the classifier. In addition, the nonlinear classifier MLP outperformed the linear methods. Finally, training based AUC maximization provided better performance than training based on the minimization of the mean square error. The results support the conclusion that the choice of the systems architecture is critical and both spatial filtering and classification must be considered together.

On the interpretation of weight vectors of linear models in multivariate neuroimaging

The increase in spatiotemporal resolution of neuroimaging devices is accompanied by a trend towards more powerful multivariate analysis methods. Often it is desired to interpret the outcome of these methods with respect to the cognitive processes under study. Here we discuss which methods allow for such interpretations, and provide guidelines for choosing an appropriate analysis for a given experimental goal: For a surgeon who needs to decide where to remove brain tissue it is most important to determine the origin of cognitive functions and associated neural processes. In contrast, when communicating with paralyzed or comatose patients via brain-computer interfaces, it is most important to accurately extract the neural processes specific to a certain mental state. These equally important but complementary objectives require different analysis methods. Determining the origin of neural processes in time or space from the parameters of a data-driven model requires what we call a forward model of the data; such a model explains how the measured data was generated from the neural sources. Examples are general linear models (GLMs). Methods for the extraction of neural information from data can be considered as backward models, as they attempt to reverse the data generating process. Examples are multivariate classifiers. Here we demonstrate that the parameters of forward models are neurophysiologically interpretable in the sense that significant nonzero weights are only observed at channels the activity of which is related to the brain process under study. In contrast, the interpretation of backward model parameters can lead to wrong conclusions regarding the spatial or temporal origin of the neural signals of interest, since significant nonzero weights may also be observed at channels the activity of which is statistically independent of the brain process under study. As a remedy for the linear case, we propose a procedure for transforming backward models into forward models. This procedure enables the neurophysiological interpretation of the parameters of linear backward models. We hope that this work raises awareness for an often encountered problem and provides a theoretical basis for conducting better interpretable multivariate neuroimaging analyses.

A survey of signal processing algorithms in brain-computer interfaces based on electrical brain signals

Brain-computer interfaces (BCIs) aim at providing a non-muscular channel for sending commands to the external world using the electroencephalographic activity or other electrophysiological measures of the brain function. An essential factor in the successful operation of BCI systems is the methods used to process the brain signals. In the BCI literature, however, there is no comprehensive review of the signal processing techniques used. This work presents the first such comprehensive survey of all BCI designs using electrical signal recordings published prior to January 2006. Detailed results from this survey are presented and discussed. The following key research questions are addressed: (1) what are the key signal processing components of a BCI, (2) what signal processing algorithms have been used in BCIs and (3) which signal processing techniques have received more attention?

EMG and EOG artifacts in brain computer interface systems: A survey

It is widely accepted in the brain computer interface (BCI) research community that neurological phenomena are the only source of control in any BCI system. Artifacts are undesirable signals that can interfere with neurological phenomena. They may change the characteristics of neurological phenomena or even be mistakenly used as the source of control in BCI systems. Electrooculography (EOG) and electromyography (EMG) artifacts are considered among the most important sources of physiological artifacts in BCI systems. Currently, however, there is no comprehensive review of EMG and EOG artifacts in BCI literature. This paper reviews EOG and EMG artifacts associated with BCI systems and the current methods for dealing with them. More than 250 refereed journal and conference papers are reviewed and categorized based on the type of neurological phenomenon used and the methods employed for handling EOG and EMG artifacts. This study reveals weaknesses in BCI studies related to reporting the methods of handling EMG and EOG artifacts. Most BCI papers do not report whether or not they have considered the presence of EMG and EOG artifacts in the brain signals. Only a small percentage of BCI papers report automated methods for rejection or removal of artifacts in their systems. As the lack of dealing with artifacts may result in the deterioration of the performance of a particular BCI system during practical applications, it is necessary to develop automatic methods to handle artifacts or to design BCI systems whose performance is robust to the presence of artifacts.

Blind Estimation of Channel Parameters and Source Components for EEG Signals: A Sparse Factorization Approach

In this paper, we use a two-stage sparse factorization approach for blindly estimating the channel parameters and then estimating source components for electroencephalogram (EEG) signals. EEG signals are assumed to be linear mixtures of source components, artifacts, etc. Therefore, a raw EEG data matrix can be factored into the product of two matrices, one of which represents the mixing matrix and the other the source component matrix. Furthermore, the components are sparse in the time-frequency domain, i.e., the factorization is a sparse factorization in the time frequency domain. It is a challenging task to estimate the mixing matrix. Our extensive analysis and computational results, which were based on many sets of EEG data, not only provide firm evidences supporting the above assumption, but also prompt us to propose a new algorithm for estimating the mixing matrix. After the mixing matrix is estimated, the source components are estimated in the time frequency domain using a linear programming method. In an example of the potential applications of our approach, we analyzed the EEG data that was obtained from a modified Sternberg memory experiment. Two almost uncorrelated components obtained by applying the sparse factorization method were selected for phase synchronization analysis. Several interesting findings were obtained, especially that memory-related synchronization and desynchronization appear in the alpha band, and that the strength of alpha band synchronization is related to memory performance.