Pre-Trial EEG-Based Single-Trial Motor Performance Prediction to Enhance Neuroergonomics for a Hand Force Task

Decoding subjective emotional arousal from EEG during an immersive virtual reality experience

Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation: 37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a long short-term memory recurrent neural network. We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience.

Predicting motor behavior: an efficient EEG signal processing pipeline to detect brain states with potential therapeutic relevance for VR-based neurorehabilitation

Virtual reality (VR)-based motor therapy is an emerging approach in neurorehabilitation. The combination of VR with electroencephalography (EEG) presents further opportunities to improve therapeutic efficacy by personalizing the paradigm. Specifically, the idea is to synchronize the choice and timing of stimuli in the perceived virtual world with fluctuating brain states relevant to motor behavior. Here, we present an open source EEG single-trial based classification pipeline that is designed to identify ongoing brain states predictive of the planning and execution of movements. 9 healthy volunteers each performed 1080 trials of a repetitive reaching task with an implicit two-alternative forced choice, i.e., use of the right or left hand, in response to the appearance of a visual target. The performance of the EEG decoding pipeline was assessed with respect to classification accuracy of right vs. left arm use, based on the EEG signal at the time of the stimulus. Different features, feature extraction methods, and classifiers were compared at different time windows; the number and location of informative EEG channels and the number of calibration trials needed were also quantified, as well as any benefits from individual-level optimization of pipeline parameters. This resulted in a set of recommended parameters that achieved an average 83.3% correct prediction on never-before-seen testing data, and a state-of-the-art 77.1% in a real-time simulation. Neurophysiological plausibility of the resulting classifiers was assessed by time-frequency and event-related potential analyses, as well as by Independent Component Analysis topographies and cortical source localization. We expect that this pipeline will facilitate the identification of relevant brain states as prospective therapeutic targets in closed-loop EEG-VR motor neurorehabilitation.

Manipulating Single-Trial Motor Performance in Chronic Stroke Patients by Closed-Loop Brain State Interaction

Motor impaired patients performing repetitive motor tasks often reveal large single-trial performance variations. Based on a data-driven framework, we extracted robust oscillatory brain states from pre-trial intervals, which are predictive for the upcoming motor performance on the level of single trials. Based on the brain state estimate, i.e. whether the brain state predicts a good or bad upcoming performance, we implemented a novel gating strategy for the start of trials by selecting specifically suitable or unsuitable trial starting time points. In a pilot study with four chronic stroke patients with hand motor impairments, we conducted a total of 41 sessions. After few initial calibration sessions, patients completed approximately 15 hours of effective hand motor training during eight online sessions using the gating strategy. Patients' reaction times were significantly reduced for suitable trials compared to unsuitable trials and shorter overall trial durations under suitable states were found in two patients. Overall, this successful proof-of-concept pilot study motivates to transfer this closed-loop training framework to a clinical study and to other application fields, such as cognitive rehabilitation, sport sciences or systems neuroscience.

A Pilot Study on Data-Driven Adaptive Deep Brain Stimulation in Chronically Implanted Essential Tremor Patients

Deep brain stimulation (DBS) is an established therapy for Parkinson's disease (PD) and essential-tremor (ET). In adaptive DBS (aDBS) systems, online tuning of stimulation parameters as a function of neural signals may improve treatment efficacy and reduce side-effects. State-of-the-art aDBS systems use symptom surrogates derived from neural signals-so-called neural markers (NMs)-defined on the patient-group level, and control strategies assuming stationarity of symptoms and NMs. We aim at improving these aDBS systems with (1) a data-driven approach for identifying patient- and session-specific NMs and (2) a control strategy coping with short-term non-stationary dynamics. The two building blocks are implemented as follows: (1) The data-driven NMs are based on a machine learning model estimating tremor intensity from electrocorticographic signals. (2) The control strategy accounts for local variability of tremor statistics. Our study with three chronically implanted ET patients amounted to five online sessions. Tremor quantified from accelerometer data shows that symptom suppression is at least equivalent to that of a continuous DBS strategy in 3 out-of 4 online tests, while considerably reducing net stimulation (at least 24%). In the remaining online test, symptom suppression was not significantly different from either the continuous strategy or the no treatment condition. We introduce a novel aDBS system for ET. It is the first aDBS system based on (1) a machine learning model to identify session-specific NMs, and (2) a control strategy coping with short-term non-stationary dynamics. We show the suitability of our aDBS approach for ET, which opens the door to its further study in a larger patient population.

Identifying controllable cortical neural markers with machine learning for adaptive deep brain stimulation in Parkinson's disease

The identification of oscillatory neural markers of Parkinson's disease (PD) can contribute not only to the understanding of functional mechanisms of the disorder, but may also serve in adaptive deep brain stimulation (DBS) systems. These systems seek online adaptation of stimulation parameters in closed-loop as a function of neural markers, aiming at improving treatment's efficacy and reducing side effects. Typically, the identification of PD neural markers is based on group-level studies. Due to the heterogeneity of symptoms across patients, however, such group-level neural markers, like the beta band power of the subthalamic nucleus, are not present in every patient or not informative about every patient's motor state. Instead, individual neural markers may be preferable for providing a personalized solution for the adaptation of stimulation parameters. Fortunately, data-driven bottom-up approaches based on machine learning may be utilized. These approaches have been developed and applied successfully in the field of brain-computer interfaces with the goal of providing individuals with means of communication and control. In our contribution, we present results obtained with a novel supervised data-driven identification of neural markers of hand motor performance based on a supervised machine learning model. Data of 16 experimental sessions obtained from seven PD patients undergoing DBS therapy show that the supervised patient-specific neural markers provide improved decoding accuracy of hand motor performance, compared to group-level neural markers reported in the literature. We observed that the individual markers are sensitive to DBS therapy and thus, may represent controllable variables in an adaptive DBS system.

Modulatory effects of dynamic fMRI-based neurofeedback on emotion regulation networks in adolescent females

Research has shown that difficulties with emotion regulation abilities in childhood and adolescence increase the risk for developing symptoms of mental disorders, e.g anxiety. We investigated whether functional magnetic resonance imaging (fMRI)-based neurofeedback (NF) can modulate brain networks supporting emotion regulation abilities in adolescent females.

We performed three experiments (Experiment 1: N ​= ​18; Experiment 2: N ​= ​30; Experiment 3: N ​= ​20). We first compared different NF implementations regarding their effectiveness of modulating prefrontal cortex (PFC)-amygdala functional connectivity (fc). Further we assessed the effects of fc-NF on neural measures, emotional/metacognitive measures and their associations. Finally, we probed the mechanism underlying fc-NF by examining concentrations of inhibitory and excitatory neurotransmitters.

Results showed that NF implementations differentially modulate PFC-amygdala fc. Using the most effective NF implementation we observed important relationships between neural and emotional/metacognitive measures, such as practice-related change in fc was related with change in thought control ability. Further, we found that the relationship between state anxiety prior to the MRI session and the effect of fc-NF was moderated by GABA concentrations in the PFC and anterior cingulate cortex.

To conclude, we were able to show that fc-NF can be used in adolescent females to shape neural and emotional/metacognitive measures underlying emotion regulation. We further show that neurotransmitter concentrations moderate fc–NF–effects.

•

Neurofeedback implementations differentially modulate PFC-amygdala connectivity.

•

Functional connectivity neurofeedback affect measures of emotion regulation.

•

Neurotransmitter concentrations moderate neurofeedback effects.

Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems : Guidelines Derived from Simulation and Real-World Data

We report on novel supervised algorithms for single-trial brain state decoding. Their reliability and robustness are essential to efficiently perform neurotechnological applications in closed-loop. When brain activity is assessed by multichannel recordings, spatial filters computed by the source power comodulation (SPoC) algorithm allow identifying oscillatory subspaces. They regress to a known continuous trial-wise variable reflecting, e.g. stimulus characteristics, cognitive processing or behavior. In small dataset scenarios, this supervised method tends to overfit to its training data as the involved recordings via electroencephalogram (EEG), magnetoencephalogram or local field potentials generally provide a low signal-to-noise ratio. To improve upon this, we propose and characterize three types of regularization techniques for SPoC: approaches using Tikhonov regularization (which requires model selection via cross-validation), combinations of Tikhonov regularization and covariance matrix normalization as well as strategies exploiting analytical covariance matrix shrinkage. All proposed techniques were evaluated both in a novel simulation framework and on real-world data. Based on the simulation findings, we saw our expectations fulfilled, that SPoC regularization generally reveals the largest benefit for small training sets and under severe label noise conditions. Relevant for practitioners, we derived operating ranges of regularization hyperparameters for cross-validation based approaches and offer open source code. Evaluating all methods additionally on real-world data, we observed an improved regression performance mainly for datasets from subjects with initially poor performance. With this proof-of-concept paper, we provided a generalizable regularization framework for SPoC which may serve as a starting point for implementing advanced techniques in the future.

Post-hoc Labeling of Arbitrary M/EEG Recordings for Data-Efficient Evaluation of Neural Decoding Methods

Many cognitive, sensory and motor processes have correlates in oscillatory neural source activity, which is embedded as a subspace in the recorded brain signals. Decoding such processes from noisy magnetoencephalogram/electroencephalogram (M/EEG) signals usually requires data-driven analysis methods. The objective evaluation of such decoding algorithms on experimental raw signals, however, is a challenge: the amount of available M/EEG data typically is limited, labels can be unreliable, and raw signals often are contaminated with artifacts. To overcome some of these problems, simulation frameworks have been introduced which support the development of data-driven decoding algorithms and their benchmarking. For generating artificial brain signals, however, most of the existing frameworks make strong and partially unrealistic assumptions about brain activity. This limits the generalization of results observed in the simulation to real-world scenarios. In the present contribution, we show how to overcome several shortcomings of existing simulation frameworks. We propose a versatile alternative, which allows for an objective evaluation and benchmarking of novel decoding algorithms using real neural signals. It allows to generate comparatively large datasets with labels being deterministically recoverable from the arbitrary M/EEG recordings. A novel idea to generate these labels is central to this framework: we determine a subspace of the true M/EEG recordings and utilize it to derive novel labels. These labels contain realistic information about the oscillatory activity of some underlying neural sources. For two categories of subspace-defining methods, we showcase how such labels can be obtained-either by an exclusively data-driven approach (independent component analysis-ICA), or by a method exploiting additional anatomical constraints (minimum norm estimates-MNE). We term our framework post-hoc labeling of M/EEG recordings. To support the adoption of the framework by practitioners, we have exemplified its use by benchmarking three standard decoding methods-i.e., common spatial patterns (CSP), source power-comodulation (SPoC), and convolutional neural networks (ConvNets)-wrt. Varied dataset sizes, label noise, and label variability. Source code and data are made available to the reader for facilitating the application of our post-hoc labeling framework.

Mining Within-Trial Oscillatory Brain Dynamics to Address the Variability of Optimized Spatial Filters

Data-driven spatial filtering algorithms optimize scores, such as the contrast between two conditions to extract oscillatory brain signal components. Most machine learning approaches for the filter estimation, however, disregard within-trial temporal dynamics and are extremely sensitive to changes in training data and involved hyperparameters. This leads to highly variable solutions and impedes the selection of a suitable candidate for, e.g., neurotechnological applications. Fostering component introspection, we propose to embrace this variability by condensing the functional signatures of a large set of oscillatory components into homogeneous clusters, each representing specific within-trial envelope dynamics. The proposed method is exemplified by and evaluated on a complex hand force task with a rich within-trial structure. Based on electroencephalography data of 18 healthy subjects, we found that the components' distinct temporal envelope dynamics are highly subject-specific. On average, we obtained seven clusters per subject, which were strictly confined regarding their underlying frequency bands. As the analysis method is not limited to a specific spatial filtering algorithm, it could be utilized for a wide range of neurotechnological applications, e.g., to select and monitor functionally relevant features for brain-computer interface protocols in stroke rehabilitation.

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.

Transcranial Direct Current Stimulation Enhances Motor Skill Learning but Not Generalization in Chronic Stroke

BACKGROUND

Motor training alone or combined with transcranial direct current stimulation (tDCS) positioned over the motor cortex (M1) improves motor function in chronic stroke. Currently, understanding of how tDCS influences the process of motor skill learning after stroke is lacking.

OBJECTIVE

To assess the effects of tDCS on the stages of motor skill learning and on generalization to untrained motor function.

METHODS

In this randomized, sham-controlled, blinded study of 56 mildly impaired chronic stroke patients, tDCS (anode over the ipsilesional M1 and cathode on the contralesional forehead) was applied during 5 days of training on an unfamiliar, challenging fine motor skill task (sequential visual isometric pinch force task). We assessed online and offline learning during the training period and retention over the following 4 months. We additionally assessed the generalization to untrained tasks.

RESULTS

With training alone (sham tDCS group), patients acquired a novel motor skill. This skill improved online, remained stable during the offline periods and was largely retained at follow-up. When tDCS was added to training (real tDCS group), motor skill significantly increased relative to sham, mostly in the online stage. Long-term retention was not affected by tDCS. Training effects generalized to untrained tasks, but those performance gains were not enhanced further by tDCS.

CONCLUSIONS

Training of an unfamiliar skill task represents a strategy to improve fine motor function in chronic stroke. tDCS augments motor skill learning, but its additive effect is restricted to the trained skill.

Deep learning with convolutional neural networks for EEG decoding and visualization

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp 38:5391-5420, 2017. © 2017 Wiley Periodicals, Inc.