Effects of user mental state on EEG-BCI performance

Comparing a BCI communication system in a patient with Multiple System Atrophy, with an animal model

Paralysis affects many people worldwide, and the people affected often suffer from impaired communication. We developed a microelectrode-based Brain-Computer Interface (BCI) for enabling communication in patients affected by paralysis, and implanted it in a patient with Multiple System Atrophy (MSA), a neurodegenerative disease that causes widespread neural symptoms including paralysis. To verify the effectiveness of the BCI system, it was also tested by implanting it in a non-human primate (NHP). Data from the human and NHP were used to train binary classifiers two different types of machine learning models: a Linear Discriminant Analysis (LDA) model, and a Long Short-Term Memory (LSTM)-based Artificial Neural Network (ANN). The LDA model performed at up to 72.7 % accuracy for binary decoding in the human patient, however, performance was highly variable and was much lower on most recording days. The BCI system was able to accurately decode movement vs non-movement in the NHP (accuracy using LDA: 82.7 ± 3.3 %, LSTM: 83.7 ± 2.2 %, 95 % confidence intervals), however it was not able to with recordings from the human patient (accuracy using LDA: 47.0 ± 5.1 %, LSTM: 44.6 ± 9.9 %, 95 % confidence intervals). We discuss how neurodegenerative diseases such as MSA can impede BCI-based communication, and postulate on the mechanisms by which this may occur.

Towards developing brain-computer interfaces for people with Multiple Sclerosis

BACKGROUND

Multiple Sclerosis (MS) can be a severely disabling condition that leads to various neurological symptoms. A Brain-Computer Interface (BCI) may substitute some lost function; however, there is a lack of BCI research in people with MS. Present BCI designs have also overlooked the unique pathological changes associated with MS and have not considered needs of users within their home environments. To progress this research area effectively and efficiently, we aimed to evaluate user needs and assess the feasibility and user-centric requirements of a BCI for people with MS. We hypothesised that (i) people with MS would be interested in adopting BCI technology and (ii) those with reduced independence would prefer a higher-performing invasive BCI.

METHODS

We conducted an online survey of people with MS to describe user preferences and establish the initial steps of user-centred design. The survey aimed to understand their interest in BCI applications, bionic applications, device preferences, and development considerations and related these to symptoms and assistance needs.

RESULTS

We demonstrated widespread interest for BCI applications in all stages of MS, with a preference for a non-invasive (n = 12) or minimally invasive (n = 15) BCI over carer assistance (n = 6). Descriptive analysis indicated that level of independence did not influence preference towards the higher performing but highly invasive BCI.

CONCLUSIONS

The needs of end users reported in this study are crucial for efficient development of BCI systems that can be effectively translated into the home environment. Considering the potential to enhance independence and quality of life for people living with MS, the results emphasise the importance of user-centred design for future advancement of BCIs that account for the unique pathological changes associated with MS.

Spatial and spectral changes in cortical surface potentials during pinching versusthumb and index finger flexion

Electrocorticographic (ECoG) signals provide high-fidelity representations of sensorimotor cortex activation during contralateral hand movements. Understanding the relationship between independent and coordinated finger movements along with their corresponding ECoG signals is crucial for precise brain mapping and neural prosthetic development. We analyzed subdural ECoG signals from three adult epilepsy patients with subdural electrode arrays implanted for seizure foci identification. Patients performed a cue-based task consisting of thumb flexion, index finger flexion or a pinching movement of both fingers together. Broadband power changes were estimated using principal component analysis of the power spectrum. All patients showed significant increases in broadband power during each movement compared to rest. We created topological maps for each movement type on brain renderings and quantified spatial overlap between movement types using a resampling metric. Pinching exhibited the highest spatial overlap with index flexion, followed by superimposed index and thumb flexion, with the least overlap observed for thumb flexion alone. This analysis provides practical insights into the complex overlap of finger representations in the motor cortex during various movement types and may help guide more nuanced approaches to brain-computer interfaces and neural prosthetics.

A Systematic Review of Bimanual Motor Coordination in Brain-Computer Interface

Advancements in neuroscience and artificial intelligence are propelling rapid progress in brain-computer interfaces (BCIs). These developments hold significant potential for decoding motion intentions from brain signals, enabling direct control commands without reliance on conventional neural pathways. Growing interest exists in decoding bimanual motor tasks, crucial for activities of daily living. This stems from the need to restore motor function, especially in individuals with deficits. This review aims to summarize neurological advancements in bimanual BCIs, encompassing neuroimaging techniques, experimental paradigms, and analysis algorithms. Thirty-six articles were reviewed, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The literature search result revealed diverse experimental paradigms, protocols, and research directions, including enhancing the decoding accuracy, advancing versatile prosthesis robots, and enabling real-time applications. Notably, within BCI studies on bimanual movement coordination, a shared objective is to achieve naturalistic movement and practical applications with neurorehabilitation potential.

Motor task-to-task transfer learning for motor imagery brain-computer interfaces

Motor imagery (MI) is one of the popular control paradigms in the non-invasive brain-computer interface (BCI) field. MI-BCI generally requires users to conduct the imagination of movement (e.g., left or right hand) to collect training data for generating a classification model during the calibration phase. However, this calibration phase is generally time-consuming and tedious, as users conduct the imagination of hand movement several times without being given feedback for an extended period. This obstacle makes MI-BCI non user-friendly and hinders its use. On the other hand, motor execution (ME) and motor observation (MO) are relatively easier tasks, yield lower fatigue than MI, and share similar neural mechanisms to MI. However, few studies have integrated these three tasks into BCIs. In this study, we propose a new task-to-task transfer learning approach of 3-motor tasks (ME, MO, and MI) for building a better user-friendly MI-BCI. For this study, 28 subjects participated in 3-motor tasks experiment, and electroencephalography (EEG) was acquired. User opinions regarding the 3-motor tasks were also collected through questionnaire survey. The 3-motor tasks showed a power decrease in the alpha rhythm, known as event-related desynchronization, but with slight differences in the temporal patterns. In the classification analysis, the cross-validated accuracy (within-task) was 67.05 % for ME, 65.93 % for MI, and 73.16 % for MO on average. Consistently with the results, the subjects scored MI (3.16) as the most difficult task compared with MO (1.42) and ME (1.41), with p < 0.05. In the analysis of task-to-task transfer learning, where training and testing are performed using different task datasets, the ME-trained model yielded an accuracy of 65.93 % (MI test), which is statistically similar to the within-task accuracy (p > 0.05). The MO-trained model achieved an accuracy of 60.82 % (MI test). On the other hand, combining two datasets yielded interesting results. ME and 50 % of the MI-trained model (50-shot) classified MI with a 69.21 % accuracy, which outperformed the within-task accuracy (p < 0.05), and MO and 50 % of the MI-trained model showed an accuracy of 66.75 %. Of the low performers with a within-task accuracy of 70 % or less, 90 % (n = 21) of the subjects improved in training with ME, and 76.2 % (n = 16) improved in training with MO on the MI test at 50-shot. These results demonstrate that task-to-task transfer learning is possible and could be a promising approach to building a user-friendly training protocol in MI-BCI.

Utility of Electroencephalograms for Enhancing Clinical Care and Rehabilitation of Children with Acquired Brain Injury

The objective of this literature review was to present evidence from recent studies and applications focused on employing electroencephalogram (EEG) monitoring and methodological approaches during the rehabilitation of children with acquired brain injuries and their related effects. We describe acquired brain injury (ABI) as one of the most common reasons for cognitive and motor disabilities in children that significantly impact their safety, independence, and overall quality of life. These disabilities manifest as dysfunctions in cognition, gait, balance, upper-limb coordination, and hand dexterity. Rehabilitation treatment aims to restore and optimize these impaired functions to help children regain autonomy and enhance their quality of life. Recent advancements in monitoring technologies such as EEG measurements are increasingly playing a role in clinical diagnosis and management. A significant advantage of incorporating EEG technology in pediatric rehabilitation is its ability to provide continuous and objective quantitative monitoring of a child's neurological status. This allows for the real-time assessment of improvement or deterioration in brain function, including, but not limited to, a significant impact on motor function. EEG monitoring enables healthcare providers to tailor and adjust interventions-both pharmacological and rehabilitative-based on the child's current neurological status.

The effect of brain-computer interface controlled functional electrical stimulation training on rehabilitation of upper limb after stroke: a systematic review and meta-analysis

Introduction

Several clinical studies have demonstrated that brain-computer interfaces (BCIs) controlled functional electrical stimulation (FES) facilitate neurological recovery in patients with stroke. This review aims to evaluate the effectiveness of BCI-FES training on upper limb functional recovery in stroke patients.

Methods

PubMed, Embase, Cochrane Library, Science Direct and Web of Science were systematically searched from inception to October 2023. Randomized controlled trials (RCTs) employing BCI-FES training were included. The methodological quality of the RCTs was assessed using the PEDro scale. Meta-analysis was conducted using RevMan 5.4.1 and STATA 18.

Results

The meta-analysis comprised 290 patients from 10 RCTs. Results showed a moderate effect size in upper limb function recovery through BCI-FES training (SMD = 0.50, 95% CI: 0.26-0.73, I2 = 0%, p < 0.0001). Subgroup analysis revealed that BCI-FES training significantly enhanced upper limb motor function in BCI-FES vs. FES group (SMD = 0.37, 95% CI: 0.00-0.74, I2 = 21%, p = 0.05), and the BCI-FES + CR vs. CR group (SMD = 0.61, 95% CI: 0.28-0.95, I2 = 0%, p = 0.0003). Moreover, BCI-FES training demonstrated effectiveness in both subacute (SMD = 0.56, 95% CI: 0.25-0.87, I2 = 0%, p = 0.0004) and chronic groups (SMD = 0.42, 95% CI: 0.05-0.78, I2 = 45%, p = 0.02). Subgroup analysis showed that both adjusting (SMD = 0.55, 95% CI: 0.24-0.87, I2 = 0%, p = 0.0006) and fixing (SMD = 0.43, 95% CI: 0.07-0.78, I2 = 46%, p = 0.02). BCI thresholds before training significantly improved motor function in stroke patients. Both motor imagery (MI) (SMD = 0.41 95% CI: 0.12-0.71, I2 = 13%, p = 0.006) and action observation (AO) (SMD = 0.73, 95% CI: 0.26-1.20, I2 = 0%, p = 0.002) as mental tasks significantly improved upper limb function in stroke patients.

Discussion

BCI-FES has significant immediate effects on upper limb function in subacute and chronic stroke patients, but evidence for its long-term impact remains limited. Using AO as the mental task may be a more effective BCI-FES training strategy.

Systematic review registration

Identifier: CRD42023485744, https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023485744.

EEG-based motor imagery channel selection and classification using hybrid optimization and two-tier deep learning

Brain-computer interface (BCI) technology holds promise for individuals with profound motor impairments, offering the potential for communication and control. Motor imagery (MI)-based BCI systems are particularly relevant in this context. Despite their potential, achieving accurate and robust classification of MI tasks using electroencephalography (EEG) data remains a significant challenge. In this paper, we employed the Minimum Redundancy Maximum Relevance (MRMR) algorithm to optimize channel selection. Furthermore, we introduced a hybrid optimization approach that combines the War Strategy Optimization (WSO) and Chimp Optimization Algorithm (ChOA). This hybridization significantly enhances the classification model's overall performance and adaptability. A two-tier deep learning architecture is proposed for classification, consisting of a Convolutional Neural Network (CNN) and a modified Deep Neural Network (M-DNN). The CNN focuses on capturing temporal correlations within EEG data, while the M-DNN is designed to extract high-level spatial characteristics from selected EEG channels. Integrating optimal channel selection, hybrid optimization, and the two-tier deep learning methodology in our BCI framework presents an enhanced approach for precise and effective BCI control. Our model got 95.06% accuracy with high precision. This advancement has the potential to significantly impact neurorehabilitation and assistive technology applications, facilitating improved communication and control for individuals with motor impairments.

A Study on Changes in Estimation Accuracy for EEG Data During Calibration and Operation in MI-BCI

Changes in psychological factors have been suggested to cause variations in brain-computer interface (BCI) performance. More specifically, differences in psychological variables between the calibration and operation phases may cause a decrease in accuracy during operation, presenting a potential challenge for the adoption of BCI technology. The purpose of this study is to analyze the differences in accuracy between the calibration and operation phases of a BCI using a deep learning model. We structured tasks to simulate the calibration and operation phases, and participants performed motor imagery tasks under both conditions. The analysis revealed a significant decrease in accuracy for data obtained under the operation condition, highlighting the need for techniques capable of adapting to the electroencephalography signal data produced when users execute operations.

Motor Imagery Decoding from EEG under Visual Distraction via Feature Map Attention EEGNet

The investigation of motor imagery (MI)-based brain-computer interface (BCI) is vital to the domains of human-computer interaction and rehabilitation. Few existing studies on electroencephalogram(EEG) signals decoding based on MI consider any distractions. However, it is difficult for users to do a single MI task in real life, which is especially affected by visual distraction. In this paper, we aim to investigate the effects of visual distraction on MI decoding performance. We first design a noval MI paradigm under visual distraction and observe distinct patterns of event-related desynchronization (ERD) and event-related synchronization (ERS) in MI under visual distraction. Then, we propose a robust decoding method of MI under visual distraction from EEG signals by using the feature map attention EEGNet (named FMA-EEGNet) and use EEG data under conditions without and with distraction to compare the decoding performance of five methods (including the proposed method and other methods). The results demonstrate that FMA-EEGNet achieved mean accuracy of 89.1% and 82.2% without and with visual distraction, respectively, indicating superior performance compared to other methods while exhibiting minimal degradation in performance. This work contributes significantly to the advancement of practical applications in MI-BCI technology.

The Effect of Caffeine on Movement-Related Cortical Potential Morphology and Detection

Movement-related cortical potential (MRCP) is observed in EEG recordings prior to a voluntary movement. It has been used for e.g., quantifying motor learning and for brain-computer interfacing (BCIs). The MRCP amplitude is affected by various factors, but the effect of caffeine is underexplored. The aim of this study was to investigate if a cup of coffee with 85 mg caffeine modulated the MRCP amplitude and the classification of MRCPs versus idle activity, which estimates BCI performance. Twenty-six healthy participants performed 2 × 100 ankle dorsiflexion separated by a 10-min break before a cup of coffee was consumed, followed by another 100 movements. EEG was recorded during the movements and divided into epochs, which were averaged to extract three average MRCPs that were compared. Also, idle activity epochs were extracted. Features were extracted from the epochs and classified using random forest analysis. The MRCP amplitude did not change after consuming caffeine. There was a slight increase of two percentage points in the classification accuracy after consuming caffeine. In conclusion, a cup of coffee with 85 mg caffeine does not affect the MRCP amplitude, and improves MRCP-based BCI performance slightly. The findings suggest that drinking coffee is only a minor confounder in MRCP-related studies.

Large scale investigation of the effect of gender on mu rhythm suppression in motor imagery brain-computer interfaces

The utmost issue in Motor Imagery Brain-Computer Interfaces (MI-BCI) is the BCI poor performance known as 'BCI inefficiency'. Although past research has attempted to find a solution by investigating factors influencing users' MI-BCI performance, the issue persists. One of the factors that has been studied in relation to MI-BCI performance is gender. Research regarding the influence of gender on a user's ability to control MI-BCIs remains inconclusive, mainly due to the small sample size and unbalanced gender distribution in past studies. To address these issues and obtain reliable results, this study combined four MI-BCI datasets into one large dataset with 248 subjects and equal gender distribution. The datasets included EEG signals from healthy subjects from both gender groups who had executed a right- vs. left-hand motor imagery task following the Graz protocol. The analysis consisted of extracting the Mu Suppression Index from C3 and C4 electrodes and comparing the values between female and male participants. Unlike some of the previous findings which reported an advantage for female BCI users in modulating mu rhythm activity, our results did not show any significant difference between the Mu Suppression Index of both groups, indicating that gender may not be a predictive factor for BCI performance.

Cortical activation and BCI performance during brief tactile imagery: A comparative study with motor imagery

Brain-computer interfaces (BCIs) rely heavily on motor imagery (MI) for operation, yet tactile imagery (TI) presents a novel approach that may be advantageous in situations where visual feedback is impractical. The current study aimed to compare the cortical activity and digit classification performance induced by TI and MI to assess the viability of TI for use in BCIs. Twelve right-handed participants engaged in trials of TI and MI, focusing on their left and right index digits. Event-related desynchronization (ERD) in the mu and beta bands was analyzed, and classification accuracy was determined through an artificial neural network (ANN). Comparable ERD patterns were observed in both TI and MI, with significant decreases in ERD during imagery tasks. The ANN demonstrated high classification accuracy, with TI achieving a mean±SD of 79.30 ± 3.91 % and MI achieving 81.10 ± 2.96 %, with no significant difference between the two (p = 0.11). The study found that TI induces substantial ERD comparable to MI and maintains high classification accuracy, supporting its potential as an effective mental strategy for BCIs. This suggests that TI could be a valuable alternative in BCI applications, particularly for individuals unable to rely on visual cues.

Personalized motor imagery prediction model based on individual difference of ERP

Objective. Motor imagery-based brain-computer interaction (MI-BCI) is a novel method of achieving human and external environment interaction that can assist individuals with motor disorders to rehabilitate. However, individual differences limit the utility of the MI-BCI. In this study, a personalized MI prediction model based on the individual difference of event-related potential (ERP) is proposed to solve the MI individual difference.Approach.A novel paradigm named action observation-based multi-delayed matching posture task evokes ERP during a delayed matching posture task phase by retrieving picture stimuli and videos, and generates MI electroencephalogram through action observation and autonomous imagery in an action observation-based motor imagery phase. Based on the correlation between the ERP and MI, a logistic regression-based personalized MI prediction model is built to predict each individual's suitable MI action. 32 subjects conducted the MI task with or without the help of the prediction model to select the MI action. Then classification accuracy of the MI task is used to evaluate the proposed model and three traditional MI methods.Main results.The personalized MI prediction model successfully predicts suitable action among 3 sets of daily actions. Under suitable MI action, the individual's ERP amplitude and event-related desynchronization (ERD) intensity are the largest, which helps to improve the accuracy by 14.25%.Significance.The personalized MI prediction model that uses the temporal ERP features to predict the classification accuracy of MI is feasible for improving the individual's MI-BCI performance, providing a new personalized solution for the individual difference and practical BCI application.

Towards user-centric BCI design: Markov chain-based user assessment for mental imagery EEG-BCIs

Objective.While electroencephalography (EEG)-based brain-computer interfaces (BCIs) have many potential clinical applications, their use is impeded by poor performance for many users. To improve BCI performance, either via enhanced signal processing or user training, it is critical to understand and describe each user's ability to perform mental control tasks and produce discernible EEG patterns. While classification accuracy has predominantly been used to assess user performance, limitations and criticisms of this approach have emerged, thus prompting the need to develop novel user assessment approaches with greater descriptive capability. Here, we propose a combination of unsupervised clustering and Markov chain models to assess and describe user skill.Approach.Using unsupervisedK-means clustering, we segmented the EEG signal space into regions representing pattern states that users could produce. A user's movement through these pattern states while performing different tasks was modeled using Markov chains. Finally, using the steady-state distributions and entropy rates of the Markov chains, we proposed two metricstaskDistinctandrelativeTaskInconsistencyto assess, respectively, a user's ability to (i) produce distinct task-specific patterns for each mental task and (ii) maintain consistent patterns during individual tasks.Main results.Analysis of data from 14 adolescents using a three-class BCI revealed significant correlations between thetaskDistinctandrelativeTaskInconsistencymetrics and classification F1 score. Moreover, analysis of the pattern states and Markov chain models yielded descriptive information regarding user performance not immediately apparent from classification accuracy.Significance.Our proposed user assessment method can be used in concert with classifier-based analysis to further understand the extent to which users produce task-specific, time-evolving EEG patterns. In turn, this information could be used to enhance user training or classifier design.

EEG Amplitude Modulation Analysis across Mental Tasks: Towards Improved Active BCIs

Brain-computer interface (BCI) technology has emerged as an influential communication tool with extensive applications across numerous fields, including entertainment, marketing, mental state monitoring, and particularly medical neurorehabilitation. Despite its immense potential, the reliability of BCI systems is challenged by the intricacies of data collection, environmental factors, and noisy interferences, making the interpretation of high-dimensional electroencephalogram (EEG) data a pressing issue. While the current trends in research have leant towards improving classification using deep learning-based models, our study proposes the use of new features based on EEG amplitude modulation (AM) dynamics. Experiments on an active BCI dataset comprised seven mental tasks to show the importance of the proposed features, as well as their complementarity to conventional power spectral features. Through combining the seven mental tasks, 21 binary classification tests were explored. In 17 of these 21 tests, the addition of the proposed features significantly improved classifier performance relative to using power spectral density (PSD) features only. Specifically, the average kappa score for these classifications increased from 0.57 to 0.62 using the combined feature set. An examination of the top-selected features showed the predominance of the AM-based measures, comprising over 77% of the top-ranked features. We conclude this paper with an in-depth analysis of these top-ranked features and discuss their potential for use in neurophysiology.

Online Ternary Classification of Covert Speech by Leveraging the Passive Perception of Speech

Brain-computer interfaces (BCIs) provide communicative alternatives to those without functional speech. Covert speech (CS)-based BCIs enable communication simply by thinking of words and thus have intuitive appeal. However, an elusive barrier to their clinical translation is the collection of voluminous examples of high-quality CS signals, as iteratively rehearsing words for long durations is mentally fatiguing. Research on CS and speech perception (SP) identifies common spatiotemporal patterns in their respective electroencephalographic (EEG) signals, pointing towards shared encoding mechanisms. The goal of this study was to investigate whether a model that leverages the signal similarities between SP and CS can differentiate speech-related EEG signals online. Ten participants completed a dyadic protocol where in each trial, they listened to a randomly selected word and then subsequently mentally rehearsed the word. In the offline sessions, eight words were presented to participants. For the subsequent online sessions, the two most distinct words (most separable in terms of their EEG signals) were chosen to form a ternary classification problem (two words and rest). The model comprised a functional mapping derived from SP and CS signals of the same speech token (features are extracted via a Riemannian approach). An average ternary online accuracy of 75.3% (60% chance level) was achieved across participants, with individual accuracies as high as 93%. Moreover, we observed that the signal-to-noise ratio (SNR) of CS signals was enhanced by perception-covert modeling according to the level of high-frequency ([Formula: see text]-band) correspondence between CS and SP. These findings may lead to less burdensome data collection for training speech BCIs, which could eventually enhance the rate at which the vocabulary can grow.

Enhancing motor imagery detection efficacy using multisensory virtual reality priming

Brain-computer interfaces (BCI) have been developed to allow users to communicate with the external world by translating brain activity into control signals. Motor imagery (MI) has been a popular paradigm in BCI control where the user imagines movements of e.g., their left and right limbs and classifiers are then trained to detect such intent directly from electroencephalography (EEG) signals. For some users, however, it is difficult to elicit patterns in the EEG signal that can be detected with existing features and classifiers. As such, new user control strategies and training paradigms have been highly sought-after to help improve motor imagery performance. Virtual reality (VR) has emerged as one potential tool where improvements in user engagement and level of immersion have shown to improve BCI accuracy. Motor priming in VR, in turn, has shown to further enhance BCI accuracy. In this pilot study, we take the first steps to explore if multisensory VR motor priming, where haptic and olfactory stimuli are present, can improve motor imagery detection efficacy in terms of both improved accuracy and faster detection. Experiments with 10 participants equipped with a biosensor-embedded VR headset, an off-the-shelf scent diffusion device, and a haptic glove with force feedback showed that significant improvements in motor imagery detection could be achieved. Increased activity in the six common spatial pattern filters used were also observed and peak accuracy could be achieved with analysis windows that were 2 s shorter. Combined, the results suggest that multisensory motor priming prior to motor imagery could improve detection efficacy.

BCI-activated electrical stimulation in children with perinatal stroke and hemiparesis: A pilot study

Background

Perinatal stroke (PS) causes most hemiparetic cerebral palsy (CP) and results in lifelong disability. Children with severe hemiparesis have limited rehabilitation options. Brain computer interface- activated functional electrical stimulation (BCI-FES) of target muscles may enhance upper extremity function in hemiparetic adults. We conducted a pilot clinical trial to assess the safety and feasibility of BCI-FES in children with hemiparetic CP.

Methods

Thirteen participants (mean age = 12.2 years, 31% female) were recruited from a population-based cohort. Inclusion criteria were: (1) MRI-confirmed PS, (2) disabling hemiparetic CP, (3) age 6–18 years, (4) informed consent/assent. Those with neurological comorbidities or unstable epilepsy were excluded. Participants attended two BCI sessions: training and rehabilitation. They wore an EEG-BCI headset and two forearm extensor stimulation electrodes. Participants’ imagination of wrist extension was classified on EEG, after which muscle stimulation and visual feedback were provided when the correct visualization was detected.

Results

No serious adverse events or dropouts occurred. The most common complaints were mild headache, headset discomfort and muscle fatigue. Children ranked the experience as comparable to a long car ride and none reported as unpleasant. Sessions lasted a mean of 87 min with 33 min of stimulation delivered. Mean classification accuracies were (M = 78.78%, SD = 9.97) for training and (M = 73.48, SD = 12.41) for rehabilitation. Mean Cohen’s Kappa across rehabilitation trials was M = 0.43, SD = 0.29, range = 0.019–1.00, suggesting BCI competency.

Conclusion

Brain computer interface-FES was well -tolerated and feasible in children with hemiparesis. This paves the way for clinical trials to optimize approaches and test efficacy.

Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training

Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.

The Berlin Bimanual Test for Tetraplegia (BeBiTT): development, psychometric properties, and sensitivity to change in assistive hand exoskeleton application

BACKGROUND

Assistive hand exoskeletons are promising tools to restore hand function after cervical spinal cord injury (SCI) but assessing their specific impact on bimanual hand and arm function is limited due to lack of reliable and valid clinical tests. Here, we introduce the Berlin Bimanual Test for Tetraplegia (BeBiTT) and demonstrate its psychometric properties and sensitivity to assistive hand exoskeleton-related improvements in bimanual task performance.

METHODS

Fourteen study participants with subacute cervical SCI performed the BeBiTT unassisted (baseline). Thereafter, participants repeated the BeBiTT while wearing a brain/neural hand exoskeleton (B/NHE) (intervention). Online control of the B/NHE was established via a hybrid sensorimotor rhythm-based brain-computer interface (BCI) translating electroencephalographic (EEG) and electrooculographic (EOG) signals into open/close commands. For reliability assessment, BeBiTT scores were obtained by four independent observers. Besides internal consistency analysis, construct validity was assessed by correlating baseline BeBiTT scores with the Spinal Cord Independence Measure III (SCIM III) and Quadriplegia Index of Function (QIF). Sensitivity to differences in bimanual task performance was assessed with a bootstrapped paired t-test.

RESULTS

The BeBiTT showed excellent interrater reliability (intraclass correlation coefficients > 0.9) and internal consistency (α = 0.91). Validity of the BeBiTT was evidenced by strong correlations between BeBiTT scores and SCIM III as well as QIF. Wearing a B/NHE (intervention) improved the BeBiTT score significantly (p < 0.05) with high effect size (d = 1.063), documenting high sensitivity to intervention-related differences in bimanual task performance.

CONCLUSION

The BeBiTT is a reliable and valid test for evaluating bimanual task performance in persons with tetraplegia, suitable to assess the impact of assistive hand exoskeletons on bimanual function.

A Review of Online Classification Performance in Motor Imagery-Based Brain–Computer Interfaces for Stroke Neurorehabilitation

Motor imagery (MI)-based brain–computer interfaces (BCI) have shown increased potential for the rehabilitation of stroke patients; nonetheless, their implementation in clinical practice has been restricted due to their low accuracy performance. To date, although a lot of research has been carried out in benchmarking and highlighting the most valuable classification algorithms in BCI configurations, most of them use offline data and are not from real BCI performance during the closed-loop (or online) sessions. Since rehabilitation training relies on the availability of an accurate feedback system, we surveyed articles of current and past EEG-based BCI frameworks who report the online classification of the movement of two upper limbs in both healthy volunteers and stroke patients. We found that the recently developed deep-learning methods do not outperform the traditional machine-learning algorithms. In addition, patients and healthy subjects exhibit similar classification accuracy in current BCI configurations. Lastly, in terms of neurofeedback modality, functional electrical stimulation (FES) yielded the best performance compared to non-FES systems.

A pediatric near-infrared spectroscopy brain-computer interface based on the detection of emotional valence

Brain-computer interfaces (BCIs) are being investigated as an access pathway to communication for individuals with physical disabilities, as the technology obviates the need for voluntary motor control. However, to date, minimal research has investigated the use of BCIs for children. Traditional BCI communication paradigms may be suboptimal given that children with physical disabilities may face delays in cognitive development and acquisition of literacy skills. Instead, in this study we explored emotional state as an alternative access pathway to communication. We developed a pediatric BCI to identify positive and negative emotional states from changes in hemodynamic activity of the prefrontal cortex (PFC). To train and test the BCI, 10 neurotypical children aged 8–14 underwent a series of emotion-induction trials over four experimental sessions (one offline, three online) while their brain activity was measured with functional near-infrared spectroscopy (fNIRS). Visual neurofeedback was used to assist participants in regulating their emotional states and modulating their hemodynamic activity in response to the affective stimuli. Child-specific linear discriminant classifiers were trained on cumulatively available data from previous sessions and adaptively updated throughout each session. Average online valence classification exceeded chance across participants by the last two online sessions (with 7 and 8 of the 10 participants performing better than chance, respectively, in Sessions 3 and 4). There was a small significant positive correlation with online BCI performance and age, suggesting older participants were more successful at regulating their emotional state and/or brain activity. Variability was seen across participants in regards to BCI performance, hemodynamic response, and discriminatory features and channels. Retrospective offline analyses yielded accuracies comparable to those reported in adult affective BCI studies using fNIRS. Affective fNIRS-BCIs appear to be feasible for school-aged children, but to further gauge the practical potential of this type of BCI, replication with more training sessions, larger sample sizes, and end-users with disabilities is necessary.

Competing at the Cybathlon championship for people with disabilities: long-term motor imagery brain-computer interface training of a cybathlete who has tetraplegia

BACKGROUND

The brain-computer interface (BCI) race at the Cybathlon championship, for people with disabilities, challenges teams (BCI researchers, developers and pilots with spinal cord injury) to control an avatar on a virtual racetrack without movement. Here we describe the training regime and results of the Ulster University BCI Team pilot who has tetraplegia and was trained to use an electroencephalography (EEG)-based BCI intermittently over 10 years, to compete in three Cybathlon events.

METHODS

A multi-class, multiple binary classifier framework was used to decode three kinesthetically imagined movements (motor imagery of left arm, right arm, and feet), and relaxed state. Three game paradigms were used for training i.e., NeuroSensi, Triad, and Cybathlon Race: BrainDriver. An evaluation of the pilot's performance is presented for two Cybathlon competition training periods-spanning 20 sessions over 5 weeks prior to the 2019 competition, and 25 sessions over 5 weeks in the run up to the 2020 competition.

RESULTS

Having participated in BCI training in 2009 and competed in Cybathlon 2016, the experienced pilot achieved high two-class accuracy on all class pairs when training began in 2019 (decoding accuracy > 90%, resulting in efficient NeuroSensi and Triad game control). The BrainDriver performance (i.e., Cybathlon race completion time) improved significantly during the training period, leading up to the competition day, ranging from 274-156 s (255 ± 24 s to 191 ± 14 s mean ± std), over 17 days (10 sessions) in 2019, and from 230-168 s (214 ± 14 s to 181 ± 4 s), over 18 days (13 sessions) in 2020. However, on both competition occasions, towards the race date, the performance deteriorated significantly.

CONCLUSIONS

The training regime and framework applied were highly effective in achieving competitive race completion times. The BCI framework did not cope with significant deviation in electroencephalography (EEG) observed in the sessions occurring shortly before and during the race day. Changes in cognitive state as a result of stress, arousal level, and fatigue, associated with the competition challenge and performance pressure, were likely contributing factors to the non-stationary effects that resulted in the BCI and pilot achieving suboptimal performance on race day. Trial registration not registered.

Effect of time windows in LSTM networks for EEG-based BCIs

People with impaired motor function could be helped by an effective brain–computer interface (BCI) based on a real-time electroencephalogram (EEG) and artificial intelligence algorithms. However, current methodologies for interpreting patient instructions from an EEG are not accurate enough to be completely safe in a real-world situation , where a poor decision would place their physical integrity at risk, such as when traveling in an electric wheelchair in a city. For various reasons, such as the low signal-to-noise ratio of portable EEGs or the effects of signal contamination (disturbances due to user movement, temporal variation of the features of EEG signals, etc.), a long short-term memory network (LSTM) (a type of recurrent neural network) that is able to learn data flow patterns from EEG signals could improve the classification of the actions taken by the user. In this paper, the effectiveness of using an LSTM with a low-cost wireless EEG device in real time is tested, and the time window that maximizes its classification accuracy is studied. The goal is to be able to implement it in the BCI of a smart wheelchair with a simple coded command protocol, such as opening or closing the eyes, which could be executed by patients with reduced mobility. Results show a higher resolution of the LSTM with an accuracy range between 77.61 and 92.14% compared to traditional classifiers (59.71%), and an optimal time window of around 7 s for the task done by users in this work. In addition, tests in real-life contexts show that a trade-off between accuracy and response times is necessary to ensure detection.

Automatic Cognitive Fatigue Detection Using Wearable fNIRS and Machine Learning

Wearable sensors have increasingly been applied in healthcare to generate data and monitor patients unobtrusively. Their application for Brain-Computer Interfaces (BCI) allows for unobtrusively monitoring one's cognitive state over time. A particular state relevant in multiple domains is cognitive fatigue, which may impact performance and attention, among other capabilities. The monitoring of this state will be applied in real learning settings to detect and advise on effective break periods. In this study, two functional near-infrared spectroscopy (fNIRS) wearable devices were employed to build a BCI to automatically detect the state of cognitive fatigue using machine learning algorithms. An experimental procedure was developed to effectively induce cognitive fatigue that included a close-to-real digital lesson and two standard cognitive tasks: Corsi-Block task and a concentration task. Machine learning models were user-tuned to account for the individual dynamics of each participant, reaching classification accuracy scores of around 70.91 ± 13.67 %. We concluded that, although effective for some subjects, the methodology needs to be individually validated before being applied. Moreover, time on task was not a particularly determining factor for classification, i.e., to induce cognitive fatigue. Further research will include other physiological signals and human-computer interaction variables.

Is There a Difference in Brain Functional Connectivity between Chinese Coal Mine Workers Who Have Engaged in Unsafe Behavior and Those Who Have Not?

(1) Background: As a world-recognized high-risk occupation, coal mine workers need various cognitive functions to process the surrounding information to cope with a large number of perceived hazards or risks. Therefore, it is necessary to explore the connection between coal mine workers’ neural activity and unsafe behavior from the perspective of cognitive neuroscience. This study explored the functional brain connectivity of coal mine workers who have engaged in unsafe behaviors (EUB) and those who have not (NUB). (2) Methods: Based on functional near-infrared spectroscopy (fNIRS), a total of 106 workers from the Hongliulin coal mine of Shaanxi North Mining Group, one of the largest modern coal mines in China, completed the test. Pearson’s Correlation Coefficient (COR) analysis, brain network analysis, and two-sample t-test were used to investigate the difference in brain functional connectivity between the two groups. (3) Results: The results showed that there were significant differences in functional brain connectivity between EUB and NUB among the frontopolar area (p = 0.002325), orbitofrontal area (p = 0.02102), and pars triangularis Broca’s area (p = 0.02888). Small-world properties existed in the brain networks of both groups, and the dorsolateral prefrontal cortex had significant differences in clustering coefficient (p = 0.0004), nodal efficiency (p = 0.0384), and nodal local efficiency (p = 0.0004). (4) Conclusions: This study is the first application of fNIRS to the field of coal mine safety. The fNIRS brain functional connectivity analysis is a feasible method to investigate the neuropsychological mechanism of unsafe behavior in coal mine workers in the view of brain science.

Multi-Parameter Physiological State Monitoring in Target Detection Under Real-World Settings

Mental state changes induced by stimuli under experimental settings or by daily events in real life affect task performance and are entwined with physical and mental health. In this study, we developed a physiological state indicator with five parameters that reflect the subject's real-time physiological states based on online EEG signal processing. These five parameters are attention, fatigue, stress, and the brain activity shifts of the left and right hemispheres. We designed a target detection experiment modified by a cognitive attention network test for validating the effectiveness of the proposed indicator, as such conditions would better approximate a real chaotic environment. Results demonstrated that attention levels while performing the target detection task were significantly higher than during rest periods, but also exhibited a decay over time. In contrast, the fatigue level increased gradually and plateaued by the third rest period. Similar to attention levels, the stress level decreased as the experiment proceeded. These parameters are therefore shown to be highly correlated to different stages of the experiment, suggesting their usage as primary factors in passive brain-computer interfaces (BCI). In addition, the left and right brain activity indexes reveal the EEG neural modulations of the corresponding hemispheres, which set a feasible reference of activation for an active BCI control system, such as one executing motor imagery tasks. The proposed indicator is applicable to potential passive and active BCI applications for monitoring the subject's physiological state change in real-time, along with providing a means of evaluating the associated signal quality to enhance the BCI performance.

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user’s error expectation of the robot’s current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user’s preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user’s preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.

Teaching an assistive robotic manipulator to move objects in a cluttered table requires demonstrations from expert operators, but what if the experts are individuals with motor disabilities? Batzianoulis et al. propose a learning approach which combines robot autonomy and a brain-computer interfacing that decodes whether the generated trajectories meet the user’s criteria, and show how their system enables the robot to learn individual user’s preferred behaviors using less than five demonstrations that are not necessarily optimal.

Post-stroke Rehabilitation of Severe Upper Limb Paresis in Germany - Toward Long-Term Treatment With Brain-Computer Interfaces

Severe upper limb paresis can represent an immense burden for stroke survivors. Given the rising prevalence of stroke, restoration of severe upper limb motor impairment remains a major challenge for rehabilitation medicine because effective treatment strategies are lacking. Commonly applied interventions in Germany, such as mirror therapy and impairment-oriented training, are limited in efficacy, demanding for new strategies to be found. By translating brain signals into control commands of external devices, brain-computer interfaces (BCIs) and brain-machine interfaces (BMIs) represent promising, neurotechnology-based alternatives for stroke patients with highly restricted arm and hand function. In this mini-review, we outline perspectives on how BCI-based therapy can be integrated into the different stages of neurorehabilitation in Germany to meet a long-term treatment approach: We found that it is most appropriate to start therapy with BCI-based neurofeedback immediately after early rehabilitation. BCI-driven functional electrical stimulation (FES) and BMI robotic therapy are well suited for subsequent post hospital curative treatment in the subacute stage. BCI-based hand exoskeleton training can be continued within outpatient occupational therapy to further improve hand function and address motivational issues in chronic stroke patients. Once the rehabilitation potential is exhausted, BCI technology can be used to drive assistive devices to compensate for impaired function. However, there are several challenges yet to overcome before such long-term treatment strategies can be implemented within broad clinical application: 1. developing reliable BCI systems with better usability; 2. conducting more research to improve BCI training paradigms and 3. establishing reliable methods to identify suitable patients.

Exploring Fatigue Effects on Performance Variation of Intensive Brain-Computer Interface Practice

Motor imagery (MI) is an endogenous mental process and is commonly used as an electroencephalogram (EEG)-based brain-computer interface (BCI) strategy. Previous studies of P300 and MI-based (without online feedback) BCI have shown that mental states like fatigue can negatively affect participants' EEG signatures. However, exogenous stimuli cause visual fatigue, which might have a different mechanism than endogenous tasks do. Furthermore, subjects could adjust themselves if online feedback is provided. In this sense, it is still unclear how fatigue affects online MI-based BCI performance. With this question, 12 healthy subjects are recruited to investigate this issue, and an MI-based online BCI experiment is performed for four sessions on different days. The first session is for training, and the other three sessions differ in rest condition and duration-no rest, 16-min eyes-open rest, and 16-min eyes-closed rest-arranged in a pseudo-random order. Multidimensional fatigue inventory (MFI) and short stress state questionnaire (SSSQ) reveal that general fatigue, mental fatigue, and distress have increased, while engagement has decreased significantly within certain sessions. However, the BCI performances, including percent valid correct (PVC) and information transfer rate (ITR), show no significant change across 400 trials. The results suggest that although the repetitive MI task has affected subjects' mental states, their BCI performances and feature separability within a session are not affected by the task significantly. Further electrophysiological analysis reveals that the alpha-band power in the sensorimotor area has an increasing tendency, while event-related desynchronization (ERD) modulation level has a decreasing trend. During the rest time, no physiological difference has been found in the eyes-open rest condition; on the contrary, the alpha-band power increase and subsequent decrease appear in the eyes-closed rest condition. In summary, this experiment shows evidence that mental states can change dramatically in the intensive MI-BCI practice, but BCI performances could be maintained.

Attention Detection in Virtual Environments Using EEG Signals: A Scoping Review

The competitive demand for attention is present in our daily lives, and the identification of neural processes in the EEG signals associated with the demand for specific attention can be useful to the individual's interactions in virtual environments. Since EEG-based devices can be portable, non-invasive, and present high temporal resolution technology for recording neural signal, the interpretations of virtual systems user's attention, fatigue and cognitive load based on parameters extracted from the EEG signal are relevant for several purposes, such as games, rehabilitation, and therapies. However, despite the large amount of studies on this subject, different methodological forms are highlighted and suggested in this work, relating virtual environments, demand of attention, workload and fatigue applications. In our summarization, we discuss controversies, current research gaps and future directions together with the background and final sections.

3D CNN to Estimate Reaction Time from Multi-Channel EEG

The study of human reaction time (RT) is invaluable not only to understand the sensory-motor functions but also to translate brain signals into machine comprehensible commands that can facilitate augmentative and alternative communication using brain-computer interfaces (BCI). Recent developments in sensor technologies, hardware computational capabilities, and neural network models have significantly helped advance biomedical signal processing research. This study is an attempt to utilize state-of-the-art resources to explore the relationship between human behavioral responses during perceptual decision-making and corresponding brain signals in the form of electroencephalograms (EEG). In this paper, a generalized 3D convolutional neural network (CNN) architecture is introduced to estimate RT for a simple visual task using single-trial multi-channel EEG. Earlier comparable studies have also employed a number of machine learning and deep learning-based models, but none of them considered inter-channel relationships while estimating RT. On the contrary, the use of 3D convolutional layers enabled us to consider the spatial relationship among adjacent channels while simultaneously utilizing spectral information from individual channels. Our model can predict RT with a root mean square error of 91.5 ms and a correlation coefficient of 0.83. These results surpass all the previous results attained from different studies.Clinical relevance Novel approaches to decode brain signals can facilitate research on brain-computer interfaces (BCIs), psychology, and neuroscience, enabling people to utilize assistive devices by root-causing psychological or neuromuscular disorders.

Decoding four hand gestures with a single bipolar pair of electrocorticography electrodes

Objective.Electrocorticography (ECoG) based brain-computer interfaces (BCIs) can be used to restore communication in individuals with locked-in syndrome. In motor-based BCIs, the number of degrees-of-freedom, and thus the speed of the BCI, directly depends on the number of classes that can be discriminated from the neural activity in the sensorimotor cortex. When considering minimally invasive BCI implants, the size of the subdural ECoG implant must be minimized without compromising the number of degrees-of-freedom.Approach.Here we investigated if four hand gestures could be decoded using a single ECoG strip of four consecutive electrodes spaced 1 cm apart and compared the performance between a unipolar and a bipolar montage. For that we collected data of seven individuals with intractable epilepsy implanted with ECoG grids, covering the hand region of the sensorimotor cortex. Based on the implanted grids, we generated virtual ECoG strips and compared the decoding accuracy between (a) a single unipolar electrode (Unipolar Electrode), (b) a combination of four unipolar electrodes (Unipolar Strip), (c) a single bipolar pair (Bipolar Pair) and (d) a combination of six bipolar pairs (Bipolar Strip).Main results.We show that four hand gestures can be equally well decoded using 'Unipolar Strips' (mean 67.4 ± 11.7%), 'Bipolar Strips' (mean 66.6 ± 12.1%) and 'Bipolar Pairs' (mean 67.6 ± 9.4%), while 'Unipolar Electrodes' (61.6 ± 5.9%) performed significantly worse compared to 'Unipolar Strips' and 'Bipolar Pairs'.Significance.We conclude that a single bipolar pair is a potential candidate for minimally invasive motor-based BCIs and encourage the use of ECoG as a robust and reliable BCI platform for multi-class movement decoding.

Heart rate variability predicts decline in sensorimotor rhythm control

Objective.Voluntary control of sensorimotor rhythms (SMRs, 8-12 Hz) can be used for brain-computer interface (BCI)-based operation of an assistive hand exoskeleton, e.g. in finger paralysis after stroke. To gain SMR control, stroke survivors are usually instructed to engage in motor imagery (MI) or to attempt moving the paralyzed fingers resulting in task- or event-related desynchronization (ERD) of SMR (SMR-ERD). However, as these tasks are cognitively demanding, especially for stroke survivors suffering from cognitive impairments, BCI control performance can deteriorate considerably over time. Therefore, it would be important to identify biomarkers that predict decline in BCI control performance within an ongoing session in order to optimize the man-machine interaction scheme.Approach.Here we determine the link between BCI control performance over time and heart rate variability (HRV). Specifically, we investigated whether HRV can be used as a biomarker to predict decline of SMR-ERD control across 17 healthy participants using Granger causality. SMR-ERD was visually displayed on a screen. Participants were instructed to engage in MI-based SMR-ERD control over two consecutive runs of 8.5 min each. During the 2nd run, task difficulty was gradually increased.Main results.While control performance (p= .18) and HRV (p= .16) remained unchanged across participants during the 1st run, during the 2nd run, both measures declined over time at high correlation (performance: -0.61%/10 s,p= 0; HRV: -0.007 ms/10 s,p< .001). We found that HRV exhibited predictive characteristics with regard to within-session BCI control performance on an individual participant level (p< .001).Significance.These results suggest that HRV can predict decline in BCI performance paving the way for adaptive BCI control paradigms, e.g. to individualize and optimize assistive BCI systems in stroke.

Can Children With Perinatal Stroke Use a Simple Brain Computer Interface?

[Figure: see text].

Using Brain Activity Patterns to Differentiate Real and Virtual Attended Targets during Augmented Reality Scenarios

Augmented reality is the fusion of virtual components and our real surroundings. The simultaneous visibility of generated and natural objects often requires users to direct their selective attention to a specific target that is either real or virtual. In this study, we investigated whether this target is real or virtual by using machine learning techniques to classify electroencephalographic (EEG) and eye tracking data collected in augmented reality scenarios. A shallow convolutional neural net classified 3 second EEG data windows from 20 participants in a person-dependent manner with an average accuracy above 70% if the testing data and training data came from different trials. This accuracy could be significantly increased to 77% using a multimodal late fusion approach that included the recorded eye tracking data. Person-independent EEG classification was possible above chance level for 6 out of 20 participants. Thus, the reliability of such a brain–computer interface is high enough for it to be treated as a useful input mechanism for augmented reality applications.

Research on fNIRS Recognition Method of Upper Limb Movement Intention

This paper aims at realizing upper limb rehabilitation training by using an fNIRS-BCI system. This article mainly focuses on the analysis and research of the cerebral blood oxygen signal in the system, and gradually extends the analysis and recognition method of the movement intention in the cerebral blood oxygen signal to the actual brain-computer interface system. Fifty subjects completed four upper limb movement paradigms: Lifting-up, putting down, pulling back, and pushing forward. Then, their near-infrared data and movement trigger signals were collected. In terms of the recognition algorithm for detecting the initial intention of upper limb movements, gradient boosting tree (GBDT) and random forest (RF) were selected for classification experiments. Finally, RF classifier with better comprehensive indicators was selected as the final classification algorithm. The best offline recognition rate was 94.4% (151/160). The ReliefF algorithm based on distance measurement and the genetic algorithm proposed in the genetic theory were used to select features. In terms of upper limb motion state recognition algorithms, logistic regression (LR), support vector machine (SVM), naive Bayes (NB), and linear discriminant analysis (LDA) were selected for experiments. Kappa coefficient was used as the classification index to evaluate the performance of the classifier. Finally, SVM classification got the best performance, and the four-class recognition accuracy rate was 84.4%. The results show that RF and SVM can achieve high recognition accuracy in motion intentions and the upper limb rehabilitation system designed in this paper has great application significance.

The influence of motivation and emotion on sensorimotor rhythm-based brain-computer interface performance

While decades of research have investigated and technically improved brain-computer interface (BCI)-controlled applications, relatively little is known about the psychological aspects of brain-computer interfacing. In 35 healthy students, we investigated whether extrinsic motivation manipulated via monetary reward and emotional state manipulated via video and music would influence behavioral and psychophysiological measures of performance with a sensorimotor rhythm (SMR)-based BCI. We found increased task-related brain activity in extrinsically motivated (rewarded) as compared with nonmotivated participants but no clear effect of emotional state manipulation. Our experiment investigated the short-term effect of motivation and emotion manipulation in a group of young healthy subjects, and thus, the significance for patients in the locked-in state, who may be in need of a BCI, remains to be investigated.

A BCI Based Alerting System for Attention Recovery of UAV Operators

As unmanned aerial vehicles have become popular, the number of accidents caused by an operator's inattention have increased. To prevent such accidents, the operator should maintain an attention status. However, limited research has been conducted on the brain-computer interface (BCI)-based system with an alerting module for the operator's attention recovery of unmanned aerial vehicles. Therefore, we introduce a detection and alerting system that prevents an unmanned aerial vehicle operator from falling into inattention status by using the operator's electroencephalogram signal. The proposed system consists of the following three components: a signal processing module, which collects and preprocesses an electroencephalogram signal of an operator, an inattention detection module, which determines whether an inattention status occurred based on the preprocessed signal, and, lastly, an alert providing module that presents stimulus to an operator when inattention is detected. As a result of evaluating the performance with a real-world dataset, it was shown that the proposed system successfully contributed to the recovery of operator attention in the evaluating dataset, although statistical significance could not be established due to the small number of subjects.

Evaluating person-centered factors associated with brain-computer interface access to a commercial augmentative and alternative communication paradigm

Current BCI-AAC systems largely utilize custom-made software and displays that may be unfamiliar to AAC stakeholders. Further, there is limited information available exploring the heterogenous profiles of individuals who may use BCI-AAC. Therefore, in this study, we aimed to evaluate how individuals with amyotrophic lateral sclerosis (ALS) learned to control a motor-based BCI switch in a row-column AAC scanning pattern, and person-centered factors associated with BCI-AAC performance. Four individuals with ALS completed 12 BCI-AAC training sessions, and three individuals without neurological impairment completed 3 BCI-AAC training sessions. To assess person-centered factors associated with BCI-AAC performance, participants completed both initial and recurring assessment measures including levels of cognition, motor ability, fatigue, and motivation. Three of four participants demonstrated either BCI-AAC performance in the range of neurotypical peers, or an improving BCI-AAC learning trajectory. However, BCI-AAC learning trajectories were variable. Assessment measures revealed that two participants presented with a suspicion for cognitive impairment yet achieved the highest levels of BCI-AAC accuracy with their increased levels of performance being possibly supported by largely unimpaired motor skills. Motor-based BCI switch access to a commercial AAC row-column scanning may be feasible for individuals with ALS and possibly supported by timely intervention.

Time-resolved estimation of strength of motor imagery representation by multivariate EEG decoding

Objective. Multivariate decoding enables access to information encoded in multiple brain activity features with high temporal resolution. However, whether the strength, of which this information is represented in the brain, can be extracted across time within single trials remains largely unexplored.Approach.In this study, we addressed this question by applying a support vector machine (SVM) to extract motor imagery (MI) representations, from electroencephalogram (EEG) data, and by performing time-resolved single-trial analyses of the multivariate decoding. EEG was recorded from a group of healthy participants during MI of opening and closing of the same hand.Main results.Cross-temporal decoding revealed both dynamic and stationary MI-relevant features during the task. Specifically, features representing MI evolved dynamically early in the trial and later stabilized into a stationary network of MI features. Using a hierarchical genetic algorithm for selection of MI-relevant features, we identified primarily contralateral alpha and beta frequency features over the sensorimotor and parieto-occipital cortices as stationary which extended into a bilateral pattern in the later part of the trial. During the stationary encoding of MI, by extracting the SVM prediction scores, we analyzed MI-relevant EEG activity patterns with respect to the temporal dynamics within single trials. We show that the SVM prediction score correlates to the amplitude of univariate MI-relevant features (as documented from an extensive repertoire of previous MI studies) within single trials, strongly suggesting that these are functional variations of MI strength hidden in trial averages.Significance.Our work demonstrates a powerful approach for estimating MI strength continually within single trials, having far-reaching impact for single-trial analyses. In terms of MI neurofeedback for motor rehabilitation, these results set the ground for more refined neurofeedback reflecting the strength of MI that can be provided to patients continually in time.

Toward brain-heart computer interfaces: a study on the classification of upper limb movements using multisystem directional estimates

OBJECTIVE

Brain-computer interfaces (BCI) exploit computational features from brain signals to perform a given task. Despite recent neurophysiology and clinical findings indicating the crucial role of functional interplay between brain and cardiovascular dynamics in locomotion, heartbeat information remains to be included in common BCI systems. In this study, we exploit the multidimensional features of directional and functional interplay between electroencephalographic and heartbeat spectra to classify upper limb movements into three classes.

APPROACH

We gathered data from 26 healthy volunteers that performed 90 movements; the data were processed using a recently proposed framework for brain-heart interplay (BHI) assessment based on synthetic physiological data generation. Extracted BHI features were employed to classify, through sequential forward selection scheme and k-nearest neighbors algorithm, among resting state and three classes of movements according to the kind of interaction with objects.

MAIN RESULTS

The results demonstrated that the proposed brain-heart computer interface (BHCI) system could distinguish between rest and movement classes automatically with an average 90% of accuracy.

SIGNIFICANCE

Further, this study provides neurophysiology insights indicating the crucial role of functional interplay originating at the cortical level onto the heart in the upper limb neural control. The inclusion of functional BHI insights might substantially improve the neuroscientific knowledge about motor control, and this may lead to advanced BHCI systems performances.

Nonspecific Visuospatial Imagery as a Novel Mental Task for Online EEG-Based BCI Control

Brain-computer interfaces (BCIs) can provide a means of communication to individuals with severe motor disorders, such as those presenting as locked-in. Many BCI paradigms rely on motor neural pathways, which are often impaired in these individuals. However, recent findings suggest that visuospatial function may remain intact. This study aimed to determine whether visuospatial imagery, a previously unexplored task, could be used to signify intent in an online electroencephalography (EEG)-based BCI. Eighteen typically developed participants imagined checkerboard arrow stimuli in four quadrants of the visual field in 5-s trials, while signals were collected using 16 dry electrodes over the visual cortex. In online blocks, participants received graded visual feedback based on their performance. An initial BCI pipeline (visuospatial imagery classifier I) attained a mean accuracy of [Formula: see text]% classifying rest against visuospatial imagery in online trials. This BCI pipeline was further improved using restriction to alpha band features (visuospatial imagery classifier II), resulting in a mean pseudo-online accuracy of [Formula: see text]%. Accuracies exceeded the threshold for practical BCIs in 12 participants. This study supports the use of visuospatial imagery as a real-time, binary EEG-BCI control paradigm.

Intralobular and Interlobular Parietal Functional Network Correlated to MI-BCI Performance

Brain-computer interface (BCI) brings hope to patients suffering from neuromuscular diseases, by allowing the control of external devices using neural signals from the central nervous system. However, a portion of individuals was unable to operate BCI with high efficacy. This research aimed to study the brain-wide functional connectivity differences that contributed to BCI performance, and investigate the relationship between task-related connectivity strength and BCI performance. Functional connectivity was estimated using pairwise Pearson's correlation from the EEG of 48 subjects performing left or right hand motor imagery (MI) tasks. The classification accuracy of linear support vector machine (SVM) to distinguish both tasks were used to represent MI-BCI performance. The significant differences in connectivity strengths were examined using Welch's T-test. The association between accuracy and connection strength was studied using correlation model. Three intralobular and fourteen interlobular connections from the parietal lobe showed a correlation of 0.31 and -0.34 respectively. Results indicate that alpha wave connectivity from 8 Hz to 13 Hz was more related to classification performance compared to high-frequency waves. Subject-independent trial-based analysis shows that MI trials executed with stronger intralobular and interlobular parietal connections performed significantly better than trials with weaker connections. Further investigation from an independent MI dataset reveals several similar connections that were correlated with MI-BCI performance. The functional connectivity of the parietal lobe could potentially allow prediction of MI-BCI performance and enable implementation of neurofeedback training for users to improve the usability of MI-BCI.

Deep Neural Network for Visual Stimulus-Based Reaction Time Estimation Using the Periodogram of Single-Trial EEG

Multiplexed deep neural networks (DNN) have engendered high-performance predictive models gaining popularity for decoding brain waves, extensively collected in the form of electroencephalogram (EEG) signals. In this paper, to the best of our knowledge, we introduce a first-ever DNN-based generalized approach to estimate reaction time (RT) using the periodogram representation of single-trial EEG in a visual stimulus-response experiment with 48 participants. We have designed a Fully Connected Neural Network (FCNN) and a Convolutional Neural Network (CNN) to predict and classify RTs for each trial. Though deep neural networks are widely known for classification applications, cascading FCNN/CNN with the Random Forest model, we designed a robust regression-based estimator to predict RT. With the FCNN model, the accuracies obtained for binary and 3-class classification were 93% and 76%, respectively, which further improved with the use of CNN (94% and 78%, respectively). The regression-based approach predicted RTs with correlation coefficients (CC) of 0.78 and 0.80 for FCNN and CNN, respectively. Investigating further, we found that the left central as well as parietal and occipital lobes were crucial for predicting RT, with significant activities in the theta and alpha frequency bands.