An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth

Recent advancements in machine learning and deep learning (DL) based neural decoders have significantly improved decoding capabilities using scalp electroencephalography (EEG). However, the interpretability of DL models remains an under-explored area. In this study, we compared multiple model explanation methods to identify the most suitable method for EEG and understand when some of these approaches might fail. A simulation framework was developed to evaluate the robustness and sensitivity of twelve back-propagation-based visualization methods by comparing to ground truth features. Multiple methods tested here showed reliability issues after randomizing either model weights or labels: e.g., the saliency approach, which is the most used visualization technique in EEG, was not class or model-specific. We found that DeepLift was consistently accurate as well as robust to detect the three key attributes tested here (temporal, spatial, and spectral precision). Overall, this study provides a review of model explanation methods for DL-based neural decoders and recommendations to understand when some of these methods fail and what they can capture in EEG.

Decoding Neural Activity Preceding Balance Loss During Standing with a Lower-limb Exoskeleton using an Interpretable Deep Learning Model

Falls are a leading cause of death in adults 65 and older. Recent efforts to restore lower-limb function in these populations have seen an increase in the use of wearable robotic systems; however, fall prevention measures in these systems require early detection of balance loss to be effective. Prior studies have investigated whether kinematic variables contain information about an impending fall, but few have examined the potential of using electroencephalography (EEG) as a fall-predicting signal and how the brain responds to avoid a fall. To address this, we decoded neural activity in a balance perturbation task while wearing an exoskeleton. We acquired EEG, electromyography (EMG), and center of pressure (COP) data from 7 healthy participants during mechanical perturbations while standing. The timing of the perturbations was randomized in all trials. We found perturbation evoked potentials (PEP) components as early as 75-134 ms after the onset of the external perturbation, which preceded both the peak in EMG (∼ 180 ms) and the COP (∼ 350 ms). A convolutional neural network trained to predict balance perturbations from single-trial EEG had a mean F-score of 75.0 ± 4.3 %. Clustering GradCAM-based model explanations demonstrated that the model utilized components in the PEP and was not driven by artifacts. Additionally, dynamic functional connectivity results agreed with model explanations; the nodal connectivity measured using phase difference derivative was higher in the occipital-parietal region in the early stage of perturbations, before shifting to the parietal, motor, and back to the frontal-parietal channels. Continuous-time decoding of COP trajectories from EEG, using a gated recurrent unit model, achieved a mean Pearson's correlation coefficient of 0.7 ± 0.06. Overall, our findings suggest that EEG signals contain short-latency neural information related to an impending fall, which may be useful for developing brain-machine interface systems for fall prevention in robotic exoskeletons.

Characterization of the Stages of Creative Writing With Mobile EEG Using Generalized Partial Directed Coherence

Two stages of the creative writing process were characterized through mobile scalp electroencephalography (EEG) in a 16-week creative writing workshop. Portable dry EEG systems (four channels: TP09, AF07, AF08, TP10) with synchronized head acceleration, video recordings, and journal entries, recorded mobile brain-body activity of Spanish heritage students. Each student's brain-body activity was recorded as they experienced spaces in Houston, Texas (“Preparation” stage), and while they worked on their creative texts (“Generation” stage). We used Generalized Partial Directed Coherence (gPDC) to compare the functional connectivity among both stages. There was a trend of higher gPDC in the Preparation stage from right temporo-parietal (TP10) to left anterior-frontal (AF07) brain scalp areas within 1–50 Hz, not reaching statistical significance. The opposite directionality was found for the Generation stage, with statistical significant differences (p < 0.05) restricted to the delta band (1–4 Hz). There was statistically higher gPDC observed for the inter-hemispheric connections AF07–AF08 in the delta and theta bands (1–8 Hz), and AF08 to TP09 in the alpha and beta (8–30 Hz) bands. The left anterior-frontal (AF07) recordings showed higher power localized to the gamma band (32–50 Hz) for the Generation stage. An ancillary analysis of Sample Entropy did not show significant difference. The information transfer from anterior-frontal to temporal-parietal areas of the scalp may reflect multisensory interpretation during the Preparation stage, while brain signals originating at temporal-parietal toward frontal locations during the Generation stage may reflect the final decision making process to translate the multisensory experience into a creative text.

A Roadmap Towards Standards for Neurally Controlled End Effectors

The control and manipulation of various types of end effectors such as powered exoskeletons, prostheses, and ‘neural’ cursors by brain-machine interface (BMI) systems has been the target of many research projects. A seamless “plug and play” interface between any BMI and end effector is desired, wherein similar user's intent cause similar end effectors to behave identically. This report is based on the outcomes of an IEEE Standards Association Industry Connections working group on End Effectors for Brain-Machine Interfacing that convened to identify and address gaps in the existing standards for BMI-based solutions with a focus on the end-effector component. A roadmap towards standardization of end effectors for BMI systems is discussed by identifying current device standards that are applicable for end effectors. While current standards address basic electrical and mechanical safety, and to some extent, performance requirements, several gaps exist pertaining to unified terminologies, data communication protocols, patient safety and risk mitigation.

An empirical comparison of neural networks and machine learning algorithms for EEG gait decoding

Previous studies of Brain Computer Interfaces (BCI) based on scalp electroencephalography (EEG) have demonstrated the feasibility of decoding kinematics for lower limb movements during walking. In this computational study, we investigated offline decoding analysis with different models and conditions to assess how they influence the performance and stability of the decoder. Specifically, we conducted three computational decoding experiments that investigated decoding accuracy: (1) based on delta band time-domain features, (2) when downsampling data, (3) of different frequency band features. In each experiment, eight different decoder algorithms were compared including the current state-of-the-art. Different tap sizes (sample window sizes) were also evaluated for a real-time applicability assessment. A feature of importance analysis was conducted to ascertain which features were most relevant for decoding; moreover, the stability to perturbations was assessed to quantify the robustness of the methods. Results indicated that generally the Gated Recurrent Unit (GRU) and Quasi Recurrent Neural Network (QRNN) outperformed other methods in terms of decoding accuracy and stability. Previous state-of-the-art Unscented Kalman Filter (UKF) still outperformed other decoders when using smaller tap sizes, with fast convergence in performance, but occurred at a cost to noise vulnerability. Downsampling and the inclusion of other frequency band features yielded overall improvement in performance. The results suggest that neural network-based decoders with downsampling or a wide range of frequency band features could not only improve decoder performance but also robustness with applications for stable use of BCIs.

Assaying neural activity of children during video game play in public spaces: a deep learning approach

OBJECTIVE

Understanding neural activity patterns in the developing brain remains one of the grand challenges in neuroscience. Developing neural networks are likely to be endowed with functionally important variability associated with the environmental context, age, gender, and other variables. Therefore, we conducted experiments with typically developing children in a stimulating museum setting and tested the feasibility of using deep learning techniques to help identify patterns of brain activity associated with different conditions.

APPROACH

A four-channel dry EEG-based Mobile brain-body imaging data of children at rest and during videogame play (VGP) was acquired at the Children's Museum of Houston. A data-driven approach based on convolutional neural networks (CNN) was used to describe underlying feature representations in the EEG and their ability to discern task and gender. The variability of the spectral features of EEG during the rest condition as a function of age was also analyzed.

MAIN RESULTS

Alpha power (7-13 Hz) was higher during rest whereas theta power (4-7 Hz) was higher during VGP. Beta (13-18 Hz) power was the most significant feature, higher in females, when differentiating between males and females. Using data from both temporoparietal channels to classify between VGP and rest condition, leave-one-subject-out cross-validation accuracy of 67% was obtained. Age-related changes in EEG spectral content during rest were consistent with previous developmental studies conducted in laboratory settings showing an inverse relationship between age and EEG power.

SIGNIFICANCE

These findings are the first to acquire, quantify and explain brain patterns observed during VGP and rest in freely behaving children in a museum setting using a deep learning framework. The study shows how deep learning can be used as a data driven approach to identify patterns in the data and explores the issues and the potential of conducting experiments involving children in a natural and engaging environment.