Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges

Optimizing spatial specificity and signal quality in fNIRS: an overview of potential challenges and possible options for improving the reliability of real-time applications

The optical brain imaging method functional near-infrared spectroscopy (fNIRS) is a promising tool for real-time applications such as neurofeedback and brain-computer interfaces. Its combination of spatial specificity and mobility makes it particularly attractive for clinical use, both at the bedside and in patients' homes. Despite these advantages, optimizing fNIRS for real-time use requires careful attention to two key aspects: ensuring good spatial specificity and maintaining high signal quality. While fNIRS detects superficial cortical brain regions, consistently and reliably targeting specific regions of interest can be challenging, particularly in studies that require repeated measurements. Variations in cap placement coupled with limited anatomical information may further reduce this accuracy. Furthermore, it is important to maintain good signal quality in real-time contexts to ensure that they reflect the true underlying brain activity. However, fNIRS signals are susceptible to contamination by cerebral and extracerebral systemic noise as well as motion artifacts. Insufficient real-time preprocessing can therefore cause the system to run on noise instead of brain activity. The aim of this review article is to help advance the progress of fNIRS-based real-time applications. It highlights the potential challenges in improving spatial specificity and signal quality, discusses possible options to overcome these challenges, and addresses further considerations relevant to real-time applications. By addressing these topics, the article aims to help improve the planning and execution of future real-time studies, thereby increasing their reliability and repeatability.

Neural Correlate and Movement Decoding of Simultaneous-and-Sequential Bimanual Movements Using EEG Signals

Bimanual coordination is important for developing a natural motor brain-computer interface (BCI) from electroencephalogram (EEG) signals, covering the aspects of bilateral arm training for rehabilitation, bimanual coordination for daily-life assistance, and also improving the multidimensional control of BCIs. For the same task targets of both hands, simultaneous and sequential bimanual movements are two different bimanual coordination manners. Planning and performing motor sequences are the fundamental abilities of humans, and it is more natural to execute sequential movements compared to simultaneous movements in many complex tasks. However, to date, for these two different manners in which two hands coordinated to reach the same task targets, the differences in the neural correlate and also the feasibility of movement discrimination have not been explored. In this study, we aimed to investigate these two issues based on a bimanual reaching task for the first time. Finally, neural correlates in the view of the movement-related cortical potentials, event-related oscillations, and source imaging showed unique neural encoding patterns of sequential movements. Besides, for the same task targets of both hands, the simultaneous and sequential bimanual movements were successfully discriminated in both pre-movement and movement execution periods. This study revealed the neural encoding patterns of sequential bimanual movements and presented its values in developing a more natural and good-performance motor BCI.

Revolutionizing Neurology: The Role of Artificial Intelligence in Advancing Diagnosis and Treatment

Artificial intelligence (AI) has emerged as a powerful tool in the field of neurology, significantly impacting the diagnosis and treatment of neurological disorders. Recent technological breakthroughs have given us access to a plethora of information relevant to many aspects of neurology. Neuroscience and AI share a long history of collaboration. Along with great potential, we encounter obstacles relating to data quality, ethics, and inherent difficulty in applying data science in healthcare. Neurological disorders pose intricate challenges due to their complex manifestations and variability. Automating image interpretation tasks, AI algorithms accurately identify brain structures and detect abnormalities. This accelerates diagnosis and reduces the workload on medical professionals. Treatment optimization benefits from AI simulations that model different scenarios and predict outcomes. These AI systems can currently perform many of the sophisticated perceptual and cognitive capacities of biological systems, such as object identification and decision making. Furthermore, AI is rapidly being used as a tool in neuroscience research, altering our understanding of brain functioning. It has the ability to revolutionize healthcare as we know it into a system in which humans and robots collaborate to deliver better care for our patients. Image analysis activities such as recognizing particular brain regions, calculating changes in brain volume over time, and detecting abnormalities in brain scans can be automated by AI systems. This lessens the strain on radiologists and neurologists while improving diagnostic accuracy and efficiency. It is now obvious that cutting-edge artificial intelligence models combined with high-quality clinical data will lead to enhanced prognostic and diagnostic models in neurological illness, permitting expert-level clinical decision aids across healthcare settings. In conclusion, AI's integration into neurology has revolutionized diagnosis, treatment, and research. As AI technologies advance, they promise to unravel the complexities of neurological disorders further, leading to improved patient care and quality of life. The symbiosis of AI and neurology offers a glimpse into a future where innovation and compassion converge to reshape neurological healthcare. This abstract provides a concise overview of the role of AI in neurology and its transformative potential.

Towards unlocking motor control in spinal cord injured by applying an online EEG-based framework to decode motor intention, trajectory and error processing

Brain-computer interfaces (BCIs) can translate brain signals directly into commands for external devices. Electroencephalography (EEG)-based BCIs mostly rely on the classification of discrete mental states, leading to unintuitive control. The ERC-funded project "Feel Your Reach" aimed to establish a novel framework based on continuous decoding of hand/arm movement intention, for a more natural and intuitive control. Over the years, we investigated various aspects of natural control, however, the individual components had not yet been integrated. Here, we present a first implementation of the framework in a comprehensive online study, combining (i) goal-directed movement intention, (ii) trajectory decoding, and (iii) error processing in a unique closed-loop control paradigm. Testing involved twelve able-bodied volunteers, performing attempted movements, and one spinal cord injured (SCI) participant. Similar movement-related cortical potentials and error potentials to previous studies were revealed, and the attempted movement trajectories were overall reconstructed. Source analysis confirmed the involvement of sensorimotor and posterior parietal areas for goal-directed movement intention and trajectory decoding. The increased experiment complexity and duration led to a decreased performance than each single BCI. Nevertheless, the study contributes to understanding natural motor control, providing insights for more intuitive strategies for individuals with motor impairments.

A time segment adaptive optimization method based on separability criterion and correlation analysis for motor imagery BCIs

Motor imagery (MI) plays a crucial role in brain-computer interface (BCI), and the classification of MI tasks using electroencephalogram (EEG) is currently under extensive investigation. During MI classification, individual differences among subjects in terms of response and time latency need to be considered. Optimizing the time segment for different subjects can enhance subsequent classification performance. In view of the individual differences of subjects in motor imagery tasks, this article proposes a Time Segment Adaptive Optimization method based on Separability criterion and Correlation analysis (TSAOSC). The fundamental principle of this method involves applying the separability criterion to various sizes of time windows within the training data, identifying the optimal raw reference signal, and adaptively adjusting the time segment position for each trial's data by analyzing its relationship with the optimal reference signal. We evaluated our method on three BCI competition datasets, respectively. The utilization of the TSAOSC method in the experiments resulted in an enhancement of 4.90% in average classification accuracy compared to its absence. Additionally, building upon the TSAOSC approach, this study proposes a Nonlinear-TSAOSC method (N-TSAOSC) for analyzing EEG signals with nonlinearity, which shows improvements in the classification accuracy of certain subjects. The results of the experiments demonstrate that the proposed method is an effective time segment optimization method, and it can be integrated into other algorithms to further improve their accuracy.

Bayesian learning from multi-way EEG feedback for robot navigation and target identification

Many brain-computer interfaces require a high mental workload. Recent research has shown that this could be greatly alleviated through machine learning, inferring user intentions via reactive brain responses. These signals are generated spontaneously while users merely observe assistive robots performing tasks. Using reactive brain signals, existing studies have addressed robot navigation tasks with a very limited number of potential target locations. Moreover, they use only binary, error-vs-correct classification of robot actions, leaving more detailed information unutilised. In this study a virtual robot had to navigate towards, and identify, target locations in both small and large grids, wherein any location could be the target. For the first time, we apply a system utilising detailed EEG information: 4-way classification of movements is performed, including specific information regarding when the target is reached. Additionally, we classify whether targets are correctly identified. Our proposed Bayesian strategy infers the most likely target location from the brain's responses. The experimental results show that our novel use of detailed information facilitates a more efficient and robust system than the state-of-the-art. Furthermore, unlike state-of-the-art approaches, we show scalability of our proposed approach: By tuning parameters appropriately, our strategy correctly identifies 98% of targets, even in large search spaces.

Decoding movement kinematics from EEG using an interpretable convolutional neural network

Continuous decoding of hand kinematics has been recently explored for the intuitive control of electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs). Deep neural networks (DNNs) are emerging as powerful decoders, for their ability to automatically learn features from lightly pre-processed signals. However, DNNs for kinematics decoding lack in the interpretability of the learned features and are only used to realize within-subject decoders without testing other training approaches potentially beneficial for reducing calibration time, such as transfer learning. Here, we aim to overcome these limitations by using an interpretable convolutional neural network (ICNN) to decode 2-D hand kinematics (position and velocity) from EEG in a pursuit tracking task performed by 13 participants. The ICNN is trained using both within-subject and cross-subject strategies, and also testing the feasibility of transferring the knowledge learned on other subjects on a new one. Moreover, the network eases the interpretation of learned spectral and spatial EEG features. Our ICNN outperformed most of the other state-of-the-art decoders, showing the best trade-off between performance, size, and training time. Furthermore, transfer learning improved kinematics prediction in the low data regime. The network attributed the highest relevance for decoding to the delta-band across all subjects, and to higher frequencies (alpha, beta, low-gamma) for a cluster of them; contralateral central and parieto-occipital sites were the most relevant, reflecting the involvement of sensorimotor, visual and visuo-motor processing. The approach improved the quality of kinematics prediction from the EEG, at the same time allowing interpretation of the most relevant spectral and spatial features.

Co-designing an interactive artificial intelligent system with post-stroke patients and caregivers to augment the lost abilities and improve their quality of life: a human-centric approach

Objectives

The motor disability due to stroke compromises the autonomy of patients and caregivers. To support autonomy and other personal and social needs, trustworthy, multifunctional, adaptive, and interactive assistive devices represent optimal solutions. To fulfill this aim, an artificial intelligence system named MAIA would aim to interpret users' intentions and translate them into actions performed by assistive devices. Analyzing their perspectives is essential to develop the MAIA system operating in harmony with patients' and caregivers' needs as much as possible.

Methods

Post-stroke patients and caregivers were interviewed to explore the impact of motor disability on their lives, previous experiences with assistive technologies, opinions, and attitudes about MAIA and their needs. Interview transcripts were analyzed using inductive thematic analysis.

Results

Sixteen interviews were conducted with 12 post-stroke patients and four caregivers. Three themes emerged: (1) Needs to be satisfied, (2) MAIA technology acceptance, and (3) Perceived trustfulness. Overall, patients are seeking rehabilitative technology, contrary to caregivers needing assistive technology to help them daily. An easy-to-use and ergonomic technology is preferable. However, a few participants trust a system based on artificial intelligence.

Conclusion

An interactive artificial intelligence technology could help post-stroke patients and their caregivers to restore motor autonomy. The insights from participants to develop the system depends on their motor ability and the role of patients or caregiver. Although technology grows exponentially, more efforts are needed to strengthen people's trust in advanced technology.

Challenges of neural interfaces for stroke motor rehabilitation

More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.

The effects of layer-wise relevance propagation-based feature selection for EEG classification: a comparative study on multiple datasets

Introduction

The brain-computer interface (BCI) allows individuals to control external devices using their neural signals. One popular BCI paradigm is motor imagery (MI), which involves imagining movements to induce neural signals that can be decoded to control devices according to the user's intention. Electroencephalography (EEG) is frequently used for acquiring neural signals from the brain in the fields of MI-BCI due to its non-invasiveness and high temporal resolution. However, EEG signals can be affected by noise and artifacts, and patterns of EEG signals vary across different subjects. Therefore, selecting the most informative features is one of the essential processes to enhance classification performance in MI-BCI.

Methods

In this study, we design a layer-wise relevance propagation (LRP)-based feature selection method which can be easily integrated into deep learning (DL)-based models. We assess its effectiveness for reliable class-discriminative EEG feature selection on two different publicly available EEG datasets with various DL-based backbone models in the subject-dependent scenario.

Results and discussion

The results show that LRP-based feature selection enhances the performance for MI classification on both datasets for all DL-based backbone models. Based on our analysis, we believe that it can broad its capability to different research domains.

A garment that measures brain activity: proof of concept of an EEG sensor layer fully implemented with smart textiles

This paper presents the first garment capable of measuring brain activity with accuracy comparable to that of state-of-the art dry electroencephalogram (EEG) systems. The main innovation is an EEG sensor layer (i.e., the electrodes, the signal transmission, and the cap support) made entirely of threads, fabrics, and smart textiles, eliminating the need for metal or plastic materials. The garment is connected to a mobile EEG amplifier to complete the measurement system. As a first proof of concept, the new EEG system (Garment-EEG) was characterized with respect to a state-of-the-art Ag/AgCl dry-EEG system (Dry-EEG) over the forehead area of healthy participants in terms of: (1) skin-electrode impedance; (2) EEG activity; (3) artifacts; and (4) user ergonomics and comfort. The results show that the Garment-EEG system provides comparable recordings to Dry-EEG, but it is more susceptible to artifacts under adverse recording conditions due to poorer contact impedances. The textile-based sensor layer offers superior ergonomics and comfort compared to its metal-based counterpart. We provide the datasets recorded with Garment-EEG and Dry-EEG systems, making available the first open-access dataset of an EEG sensor layer built exclusively with textile materials. Achieving user acceptance is an obstacle in the field of neurotechnology. The introduction of EEG systems encapsulated in wearables has the potential to democratize neurotechnology and non-invasive brain-computer interfaces, as they are naturally accepted by people in their daily lives. Furthermore, supporting the EEG implementation in the textile industry may result in lower cost and less-polluting manufacturing processes compared to metal and plastic industries.

Brain-Computer Interface Based on Steady-State Visual Evoked Potential Using Quick-Response Code Pattern for Wheelchair Control

Brain-computer interfaces (BCIs) are widely utilized in control applications for people with severe physical disabilities. Several researchers have aimed to develop practical brain-controlled wheelchairs. An existing electroencephalogram (EEG)-based BCI based on steady-state visually evoked potential (SSVEP) was developed for device control. This study utilized a quick-response (QR) code visual stimulus pattern for a robust existing system. Four commands were generated using the proposed visual stimulation pattern with four flickering frequencies. Moreover, we employed a relative power spectrum density (PSD) method for the SSVEP feature extraction and compared it with an absolute PSD method. We designed experiments to verify the efficiency of the proposed system. The results revealed that the proposed SSVEP method and algorithm yielded an average classification accuracy of approximately 92% in real-time processing. For the wheelchair simulated via independent-based control, the proposed BCI control required approximately five-fold more time than the keyboard control for real-time control. The proposed SSVEP method using a QR code pattern can be used for BCI-based wheelchair control. However, it suffers from visual fatigue owing to long-time continuous control. We will verify and enhance the proposed system for wheelchair control in people with severe physical disabilities.

Coherence based graph convolution network for motor imagery-induced EEG after spinal cord injury

Background

Spinal cord injury (SCI) may lead to impaired motor function, autonomic nervous system dysfunction, and other dysfunctions. Brain-computer Interface (BCI) system based on motor imagery (MI) can provide more scientific and effective treatment solutions for SCI patients.

Methods

According to the interaction between brain regions, a coherence-based graph convolutional network (C-GCN) method is proposed to extract the temporal-frequency-spatial features and functional connectivity information of EEG signals. The proposed algorithm constructs multi-channel EEG features based on coherence networks as graphical signals and then classifies MI tasks. Different from the traditional graphical convolutional neural network (GCN), the C-GCN method uses the coherence network of EEG signals to determine MI-related functional connections, which are used to represent the intrinsic connections between EEG channels in different rhythms and different MI tasks. EEG data of SCI patients and healthy subjects have been analyzed, where healthy subjects served as the control group.

Results

The experimental results show that the C-GCN method can achieve the best classification performance with certain reliability and stability, the highest classification accuracy is 96.85%.

Conclusion

The proposed framework can provide an effective theoretical basis for the rehabilitation treatment of SCI patients.

Brain computer interface to distinguish between self and other related errors in human agent collaboration

When a human and machine collaborate on a shared task, ambiguous events might occur that could be perceived as an error by the human partner. In such events, spontaneous error-related potentials (ErrPs) are evoked in the human brain. Knowing whom the human perceived as responsible for the error would help a machine in co-adaptation and shared control paradigms to better adapt to human preferences. Therefore, we ask whether self- and agent-related errors evoke different ErrPs. Eleven subjects participated in an electroencephalography human-agent collaboration experiment with a collaborative trajectory-following task on two collaboration levels, where movement errors occurred as trajectory deviations. Independently of the collaboration level, we observed a higher amplitude of the responses on the midline central Cz electrode for self-related errors compared to observed errors made by the agent. On average, Support Vector Machines classified self- and agent-related errors with 72.64% accuracy using subject-specific features. These results demonstrate that ErrPs can tell if a person relates an error to themselves or an external autonomous agent during collaboration. Thus, the collaborative machine will receive more informed feedback for the error attribution that allows appropriate error identification, a possibility for correction, and avoidance in future actions.

A human-in-the-loop approach for enhancing mobile robot navigation in presence of obstacles not detected by the sensory set

Human-in-the-loop approaches can greatly enhance the human-robot interaction by making the user an active part of the control loop, who can provide a feedback to the robot in order to augment its capabilities. Such feedback becomes even more important in all those situations where safety is of utmost concern, such as in assistive robotics. This study aims to realize a human-in-the-loop approach, where the human can provide a feedback to a specific robot, namely, a smart wheelchair, to augment its artificial sensory set, extending and improving its capabilities to detect and avoid obstacles. The feedback is provided by both a keyboard and a brain-computer interface: with this scope, the work has also included a protocol design phase to elicit and evoke human brain event-related potentials. The whole architecture has been validated within a simulated robotic environment, with electroencephalography signals acquired from different test subjects.

Communicative capital: a key resource for human-machine shared agency and collaborative capacity

In this work, we present a perspective on the role machine intelligence can play in supporting human abilities. In particular, we consider research in rehabilitation technologies such as prosthetic devices, as this domain requires tight coupling between human and machine. Taking an agent-based view of such devices, we propose that human-machine collaborations have a capacity to perform tasks which is a result of the combined agency of the human and the machine. We introduce communicative capital as a resource developed by a human and a machine working together in ongoing interactions. Development of this resource enables the partnership to eventually perform tasks at a capacity greater than either individual could achieve alone. We then examine the benefits and challenges of increasing the agency of prostheses by surveying literature which demonstrates that building communicative resources enables more complex, task-directed interactions. The viewpoint developed in this article extends current thinking on how best to support the functional use of increasingly complex prostheses, and establishes insight toward creating more fruitful interactions between humans and supportive, assistive, and augmentative technologies.

Cortical Reorganization of Early Somatosensory Processing in Hemiparetic Stroke

The cortical motor system can be reorganized following a stroke, with increased recruitment of the contralesional hemisphere. However, it is unknown whether a similar hemispheric shift occurs in the somatosensory system to adapt to this motor change, and whether this is related to movement impairments. This proof-of-concept study assessed somatosensory evoked potentials (SEPs), P50 and N100, in hemiparetic stroke participants and age-matched controls using high-density electroencephalograph (EEG) recordings during tactile finger stimulation. The laterality index was calculated to determine the hemispheric dominance of the SEP and re-confirmed with source localization. The study found that latencies of P50 and N100 were significantly delayed in stroke brains when stimulating the paretic hand. The amplitude of P50 in the contralateral (to stimulated hand) hemisphere was negatively correlated with the Fügl-Meyer upper extremity motor score in stroke. Bilateral cortical responses were detected in stroke, while only contralateral cortical responses were shown in controls, resulting in a significant difference in the laterality index. These results suggested that somatosensory reorganization after stroke involves increased recruitment of ipsilateral cortical regions, especially for the N100 SEP component. This reorganization delays the latency of somatosensory processing after a stroke. This research provided new insights related to the somatosensory reorganization after stroke, which could enrich future hypothesis-driven therapeutic rehabilitation strategies from a sensory or sensory-motor perspective.

Multi-Kernel Temporal and Spatial Convolution for EEG-Based Emotion Classification

Deep learning using an end-to-end convolutional neural network (ConvNet) has been applied to several electroencephalography (EEG)-based brain-computer interface tasks to extract feature maps and classify the target output. However, the EEG analysis remains challenging since it requires consideration of various architectural design components that influence the representational ability of extracted features. This study proposes an EEG-based emotion classification model called the multi-kernel temporal and spatial convolution network (MultiT-S ConvNet). The multi-scale kernel is used in the model to learn various time resolutions, and separable convolutions are applied to find related spatial patterns. In addition, we enhanced both the temporal and spatial filters with a lightweight gating mechanism. To validate the performance and classification accuracy of MultiT-S ConvNet, we conduct subject-dependent and subject-independent experiments on EEG-based emotion datasets: DEAP and SEED. Compared with existing methods, MultiT-S ConvNet outperforms with higher accuracy results and a few trainable parameters. Moreover, the proposed multi-scale module in temporal filtering enables extracting a wide range of EEG representations, covering short- to long-wavelength components. This module could be further implemented in any model of EEG-based convolution networks, and its ability potentially improves the model's learning capacity.

Determining optimal mobile neurofeedback methods for motor neurorehabilitation in children and adults with non-progressive neurological disorders: a scoping review

Background

Brain–computer interfaces (BCI), initially designed to bypass the peripheral motor system to externally control movement using brain signals, are additionally being utilized for motor rehabilitation in stroke and other neurological disorders. Also called neurofeedback training, multiple approaches have been developed to link motor-related cortical signals to assistive robotic or electrical stimulation devices during active motor training with variable, but mostly positive, functional outcomes reported. Our specific research question for this scoping review was: for persons with non-progressive neurological injuries who have the potential to improve voluntary motor control, which mobile BCI-based neurofeedback methods demonstrate or are associated with improved motor outcomes for Neurorehabilitation applications?

Methods

We searched PubMed, Web of Science, and Scopus databases with all steps from study selection to data extraction performed independently by at least 2 individuals. Search terms included: brain machine or computer interfaces, neurofeedback and motor; however, only studies requiring a motor attempt, versus motor imagery, were retained. Data extraction included participant characteristics, study design details and motor outcomes.

Results

From 5109 papers, 139 full texts were reviewed with 23 unique studies identified. All utilized EEG and, except for one, were on the stroke population. The most commonly reported functional outcomes were the Fugl-Meyer Assessment (FMA; n = 13) and the Action Research Arm Test (ARAT; n = 6) which were then utilized to assess effectiveness, evaluate design features, and correlate with training doses. Statistically and functionally significant pre-to post training changes were seen in FMA, but not ARAT. Results did not differ between robotic and electrical stimulation feedback paradigms. Notably, FMA outcomes were positively correlated with training dose.

Conclusion

This review on BCI-based neurofeedback training confirms previous findings of effectiveness in improving motor outcomes with some evidence of enhanced neuroplasticity in adults with stroke. Associative learning paradigms have emerged more recently which may be particularly feasible and effective methods for Neurorehabilitation. More clinical trials in pediatric and adult neurorehabilitation to refine methods and doses and to compare to other evidence-based training strategies are warranted.

Classification of motor imagery EEG using deep learning increases performance in inefficient BCI users

Motor Imagery Brain-Computer Interfaces (MI-BCIs) are AI-driven systems that capture brain activity patterns associated with mental imagination of movement and convert them into commands for external devices. Traditionally, MI-BCIs operate on Machine Learning (ML) algorithms, which require extensive signal processing and feature engineering to extract changes in sensorimotor rhythms (SMR). In recent years, Deep Learning (DL) models have gained popularity for EEG classification as they provide a solution for automatic extraction of spatio-temporal features in the signals. However, past BCI studies that employed DL models, only attempted them with a small group of participants, without investigating the effectiveness of this approach for different user groups such as inefficient users. BCI inefficiency is a known and unsolved problem within BCI literature, generally defined as the inability of the user to produce the desired SMR patterns for the BCI classifier. In this study, we evaluated the effectiveness of DL models in capturing MI features particularly in the inefficient users. EEG signals from 54 subjects who performed a MI task of left- or right-hand grasp were recorded to compare the performance of two classification approaches; a ML approach vs. a DL approach. In the ML approach, Common Spatial Patterns (CSP) was used for feature extraction and then Linear Discriminant Analysis (LDA) model was employed for binary classification of the MI task. In the DL approach, a Convolutional Neural Network (CNN) model was constructed on the raw EEG signals. Additionally, subjects were divided into high vs. low performers based on their online BCI accuracy and the difference between the two classifiers’ performance was compared between groups. Our results showed that the CNN model improved the classification accuracy for all subjects within the range of 2.37 to 28.28%, but more importantly, this improvement was significantly larger for low performers. Our findings show promise for employment of DL models on raw EEG signals in future MI-BCI systems, particularly for BCI inefficient users who are unable to produce desired sensorimotor patterns for conventional ML approaches.

Decoding of Turning Intention during Walking Based on EEG Biomarkers

In the EEG literature, there is a lack of asynchronous intention models that realistically propose interfaces for applications that must operate in real time. In this work, a novel BMI approach to detect in real time the intention to turn is proposed. For this purpose, an offline, pseudo-online and online analysis is presented to validate the EEG as a biomarker for the intention to turn. This article presents a methodology for the creation of a BMI that could differentiate two classes: monotonous walk and intention to turn. A comparison of some of the most popular algorithms in the literature is conducted. To filter the signal, two relevant algorithms are used: H∞ filter and ASR. For processing and classification, the mean of the covariance matrices in the Riemannian space was calculated and then, with various classifiers of different types, the distance of the test samples to each class in the Riemannian space was estimated. This dispenses with power-based models and the necessary baseline correction, which is a problem in realistic scenarios. In the cross-validation for a generic selection (valid for any subject) and a personalized one, the results were, on average, 66.2% and 69.6% with the best filter H∞. For the pseudo-online, the custom configuration for each subject was an average of 40.2% TP and 9.3 FP/min; the best subject obtained 43.9% TP and 2.9 FP/min. In the final validation test, this subject obtained 2.5 FP/min and an accuracy rate of 71.43%, and the turn anticipation was 0.21 s on average.

2020 International brain-computer interface competition: A review

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.

An Analysis of Intention Detection Strategies to Control Advanced Assistive Technologies at the CYBATHLON

With the increasing range of functionalities of advanced assistive technologies (AAT), reliable control and initiation of the desired actions become increasingly challenging for users. In this work, we present an analysis of current practices, user preferences, and usability of AAT intention detection strategies based on a survey among participants with disabilities at the CYBATHLON 2020 Global Edition. We collected data from 35 respondents, using devices in various disciplines and levels of technology maturity. We found that conventional, direct inputs such as buttons and joysticks are used by the majority of AAT (71.4%) due to their simplicity and learnability. However, 22 respondents (62.8%) reported a desire for more natural control using muscle or non-invasive brain signals, and 37.1% even reported an openness to invasive strategies for potentially improved control. The usability of the used strategies in terms of the explored attributes (reliability, mental effort, required learning) was mainly perceived positively, whereas no significant difference was observed across intention detection strategies and device types. It can be assumed that the strategies used during the CYBATHLON realistically represent options to control an AAT in a dynamic, physically and mentally demanding environment. Thus, this work underlines the need for carefully considering user needs and preferences for the selection of intention detection strategies in a context of use outside the laboratory.

Manual 3D Control of an Assistive Robotic Manipulator Using Alpha Rhythms and an Auditory Menu: A Proof-of-Concept

Brain–Computer Interfaces (BCIs) have been regarded as potential tools for individuals with severe motor disabilities, such as those with amyotrophic lateral sclerosis, that render interfaces that rely on movement unusable. This study aims to develop a dependent BCI system for manual end-point control of a robotic arm. A proof-of-concept system was devised using parieto-occipital alpha wave modulation and a cyclic menu with auditory cues. Users choose a movement to be executed and asynchronously stop said action when necessary. Tolerance intervals allowed users to cancel or confirm actions. Eight able-bodied subjects used the system to perform a pick-and-place task. To investigate the potential learning effects, the experiment was conducted twice over the course of two consecutive days. Subjects obtained satisfactory completion rates (84.0 ± 15.0% and 74.4 ± 34.5% for the first and second day, respectively) and high path efficiency (88.9 ± 11.7% and 92.2 ± 9.6%). Subjects took on average 439.7 ± 203.3 s to complete each task, but the robot was only in motion 10% of the time. There was no significant difference in performance between both days. The developed control scheme provided users with intuitive control, but a considerable amount of time is spent waiting for the right target (auditory cue). Implementing other brain signals may increase its speed.

Sliding-Mode Nonlinear Predictive Control of Brain-Controlled Mobile Robots

In this article, we develop a robust sliding-mode nonlinear predictive controller for brain-controlled robots with enhanced performance, safety, and robustness. First, the kinematics and dynamics of a mobile robot are built. After that, the proposed controller is developed by cascading a predictive controller and a smooth sliding-mode controller. The predictive controller integrates the human intention tracking with safety guarantee objectives into an optimization problem to minimize the invasion to human intention while maintaining robot safety. The smooth sliding-mode controller is designed to achieve robust desired velocity tracking. The results of human-in-the-loop simulation and robotic experiments both show the efficacy and robust performance of the proposed controller. This work provides an enabling design to enhance the future research and development of brain-controlled robots.

Usability of a Hybrid System Combining P300-Based Brain-Computer Interface and Commercial Assistive Technologies to Enhance Communication in People With Multiple Sclerosis

Brain-computer interface (BCI) can provide people with motor disabilities with an alternative channel to access assistive technology (AT) software for communication and environmental interaction. Multiple sclerosis (MS) is a chronic disease of the central nervous system that mostly starts in young adulthood and often leads to a long-term disability, possibly exacerbated by the presence of fatigue. Patients with MS have been rarely considered as potential BCI end-users. In this pilot study, we evaluated the usability of a hybrid BCI (h-BCI) system that enables both a P300-based BCI and conventional input devices (i.e., muscular dependent) to access mainstream applications through the widely used AT software for communication “Grid 3.” The evaluation was performed according to the principles of the user-centered design (UCD) with the aim of providing patients with MS with an alternative control channel (i.e., BCI), potentially less sensitive to fatigue. A total of 13 patients with MS were enrolled. In session I, participants were presented with a widely validated P300-based BCI (P3-speller); in session II, they had to operate Grid 3 to access three mainstream applications with (1) an AT conventional input device and (2) the h-BCI. Eight patients completed the protocol. Five out of eight patients with MS were successfully able to access the Grid 3 via the BCI, with a mean online accuracy of 83.3% (± 14.6). Effectiveness (online accuracy), satisfaction, and workload were comparable between the conventional AT inputs and the BCI channel in controlling the Grid 3. As expected, the efficiency (time for correct selection) resulted to be significantly lower for the BCI with respect to the AT conventional channels (Z = 0.2, p < 0.05). Although cautious due to the limited sample size, these preliminary findings indicated that the BCI control channel did not have a detrimental effect with respect to conventional AT channels on the ability to operate an AT software (Grid 3). Therefore, we inferred that the usability of the two access modalities was comparable. The integration of BCI with commercial AT input devices to access a widely used AT software represents an important step toward the introduction of BCIs into the AT centers’ daily practice.

Multimodal Human-Exoskeleton Interface for Lower Limb Movement Prediction Through a Dense Co-Attention Symmetric Mechanism

A challenging task for the biological neural signal-based human-exoskeleton interface is to achieve accurate lower limb movement prediction of patients with hemiplegia in rehabilitation training scenarios. The human-exoskeleton interface based on single-modal biological signals such as electroencephalogram (EEG) is currently not mature in predicting movements, due to its unreliability. The multimodal human-exoskeleton interface is a very novel solution to this problem. This kind of interface normally combines the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction is still limited—the connection between sEMG and EEG signals and the deep feature fusion between them are ignored. In this article, a Dense con-attention mechanism-based Multimodal Enhance Fusion Network (DMEFNet) is proposed for predicting lower limb movement of patients with hemiplegia. The DMEFNet introduces the con-attention structure to extract the common attention between sEMG and EEG signal features. To verify the effectiveness of DMEFNet, an sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96 and 88.44%, respectively.

Toward passive BCI: asynchronous decoding of neural responses to direction- and angle-specific perturbations during a simulated cockpit scenario

Neuroimaging studies have provided proof that loss of balance evokes specific neural transient wave complexes in electroencephalography (EEG), called perturbation evoked potentials (PEPs). Online decoding of balance perturbations from ongoing EEG signals can establish the possibility of implementing passive brain-computer interfaces (pBCIs) as a part of aviation/driving assistant systems. In this study, we investigated the feasibility of identifying the existence and expression of perturbations in four different conditions by using EEG signals. Fifteen healthy participants experienced four various postural changes while they sat in a glider cockpit. Sudden perturbations were exposed by a robot connected to a glider and moved to the right and left directions with tilting angles of 5 and 10 degrees. Perturbations occurred in an oddball paradigm in which participants were not aware of the time and expression of the perturbations. We employed a hierarchical approach to separate the perturbation and rest, and then discriminate the expression of perturbations. The performance of the BCI system was evaluated by using classification accuracy and F1 score. Asynchronously, we achieved average accuracies of 89.83 and 73.64% and average F1 scores of 0.93 and 0.60 for binary and multiclass classification, respectively. These results manifest the practicality of pBCI for the detection of balance disturbances in a realistic situation.

Past, Present, and Future of EEG-Based BCI Applications

An electroencephalography (EEG)-based brain-computer interface (BCI) is a system that provides a pathway between the brain and external devices by interpreting EEG. EEG-based BCI applications have initially been developed for medical purposes, with the aim of facilitating the return of patients to normal life. In addition to the initial aim, EEG-based BCI applications have also gained increasing significance in the non-medical domain, improving the life of healthy people, for instance, by making it more efficient, collaborative and helping develop themselves. The objective of this review is to give a systematic overview of the literature on EEG-based BCI applications from the period of 2009 until 2019. The systematic literature review has been prepared based on three databases PubMed, Web of Science and Scopus. This review was conducted following the PRISMA model. In this review, 202 publications were selected based on specific eligibility criteria. The distribution of the research between the medical and non-medical domain has been analyzed and further categorized into fields of research within the reviewed domains. In this review, the equipment used for gathering EEG data and signal processing methods have also been reviewed. Additionally, current challenges in the field and possibilities for the future have been analyzed.

Feel Your Reach: An EEG-Based Framework to Continuously Detect Goal-Directed Movements and Error Processing to Gate Kinesthetic Feedback Informed Artificial Arm Control

Establishing the basic knowledge, methodology, and technology for a framework for the continuous decoding of hand/arm movement intention was the aim of the ERC-funded project “Feel Your Reach”. In this work, we review the studies and methods we performed and implemented in the last 6 years, which build the basis for enabling severely paralyzed people to non-invasively control a robotic arm in real-time from electroencephalogram (EEG). In detail, we investigated goal-directed movement detection, decoding of executed and attempted movement trajectories, grasping correlates, error processing, and kinesthetic feedback. Although we have tested some of our approaches already with the target populations, we still need to transfer the “Feel Your Reach” framework to people with cervical spinal cord injury and evaluate the decoders’ performance while participants attempt to perform upper-limb movements. While on the one hand, we made major progress towards this ambitious goal, we also critically discuss current limitations.

Novel fNIRS study on homogeneous symmetric feature-based transfer learning for brain-computer interface

The brain-computer interface (BCI) provides an alternate means of communication between the brain and external devices by recognizing the brain activities and translating them into external commands. The functional Near-Infrared Spectroscopy (fNIRS) is becoming popular as a non-invasive modality for brain activity detection. The recent trends show that deep learning has significantly enhanced the performance of the BCI systems. But the inherent bottleneck for deep learning (in the domain of BCI) is the requirement of the vast amount of training data, lengthy recalibrating time, and expensive computational resources for training deep networks. Building a high-quality, large-scale annotated dataset for deep learning-based BCI systems is exceptionally tedious, complex, and expensive. This study investigates the novel application of transfer learning for fNIRS-based BCI to solve three objective functions (concerns), i.e., the problem of insufficient training data, reduced training time, and increased accuracy. We applied symmetric homogeneous feature-based transfer learning on convolutional neural network (CNN) designed explicitly for fNIRS data collected from twenty-six (26) participants performing the n-back task. The results suggested that the proposed method achieves the maximum saturated accuracy sooner and outperformed the traditional CNN model on averaged accuracy by 25.58% in the exact duration of training time, reducing the training time, recalibrating time, and computational resources.

Intention Detection Strategies for Robotic Upper-Limb Orthoses: A Scoping Review Considering Usability, Daily Life Application, and User Evaluation

Wearable robotic upper limb orthoses (ULO) are promising tools to assist or enhance the upper-limb function of their users. While the functionality of these devices has continuously increased, the robust and reliable detection of the user's intention to control the available degrees of freedom remains a major challenge and a barrier for acceptance. As the information interface between device and user, the intention detection strategy (IDS) has a crucial impact on the usability of the overall device. Yet, this aspect and the impact it has on the device usability is only rarely evaluated with respect to the context of use of ULO. A scoping literature review was conducted to identify non-invasive IDS applied to ULO that have been evaluated with human participants, with a specific focus on evaluation methods and findings related to functionality and usability and their appropriateness for specific contexts of use in daily life. A total of 93 studies were identified, describing 29 different IDS that are summarized and classified according to a four-level classification scheme. The predominant user input signal associated with the described IDS was electromyography (35.6%), followed by manual triggers such as buttons, touchscreens or joysticks (16.7%), as well as isometric force generated by residual movement in upper-limb segments (15.1%). We identify and discuss the strengths and weaknesses of IDS with respect to specific contexts of use and highlight a trade-off between performance and complexity in selecting an optimal IDS. Investigating evaluation practices to study the usability of IDS, the included studies revealed that, primarily, objective and quantitative usability attributes related to effectiveness or efficiency were assessed. Further, it underlined the lack of a systematic way to determine whether the usability of an IDS is sufficiently high to be appropriate for use in daily life applications. This work highlights the importance of a user- and application-specific selection and evaluation of non-invasive IDS for ULO. For technology developers in the field, it further provides recommendations on the selection process of IDS as well as to the design of corresponding evaluation protocols.

Single-Trial Classification of Error-Related Potentials in People with Motor Disabilities: A Study in Cerebral Palsy, Stroke, and Amputees

Brain-computer interface performance may be reduced over time, but adapting the classifier could reduce this problem. Error-related potentials (ErrPs) could label data for continuous adaptation. However, this has scarcely been investigated in populations with severe motor impairments. The aim of this study was to detect ErrPs from single-trial EEG in offline analysis in participants with cerebral palsy, an amputation, or stroke, and determine how much discriminative information different brain regions hold. Ten participants with cerebral palsy, eight with an amputation, and 25 with a stroke attempted to perform 300-400 wrist and ankle movements while a sham BCI provided feedback on their performance for eliciting ErrPs. Pre-processed EEG epochs were inputted in a multi-layer perceptron artificial neural network. Each brain region was used as input individually (Frontal, Central, Temporal Right, Temporal Left, Parietal, and Occipital), the combination of the Central region with each of the adjacent regions, and all regions combined. The Frontal and Central regions were most important, and adding additional regions only improved performance slightly. The average classification accuracies were 84 ± 4%, 87± 4%, and 85 ± 3% for cerebral palsy, amputation, and stroke participants. In conclusion, ErrPs can be detected in participants with motor impairments; this may have implications for developing adaptive BCIs or automatic error correction.

Review of tDCS Configurations for Stimulation of the Lower-Limb Area of Motor Cortex and Cerebellum

This article presents an exhaustive analysis of the works present in the literature pertaining to transcranial direct current stimulation(tDCS) applications. The aim of this work is to analyze the specific characteristics of lower-limb stimulation, identifying the strengths and weaknesses of these works and framing them with the current knowledge of tDCS. The ultimate goal of this work is to propose areas of improvement to create more effective stimulation therapies with less variability.

Multi-Parameter Physiological State Monitoring in Target Detection Under Real-World Settings

Mental state changes induced by stimuli under experimental settings or by daily events in real life affect task performance and are entwined with physical and mental health. In this study, we developed a physiological state indicator with five parameters that reflect the subject's real-time physiological states based on online EEG signal processing. These five parameters are attention, fatigue, stress, and the brain activity shifts of the left and right hemispheres. We designed a target detection experiment modified by a cognitive attention network test for validating the effectiveness of the proposed indicator, as such conditions would better approximate a real chaotic environment. Results demonstrated that attention levels while performing the target detection task were significantly higher than during rest periods, but also exhibited a decay over time. In contrast, the fatigue level increased gradually and plateaued by the third rest period. Similar to attention levels, the stress level decreased as the experiment proceeded. These parameters are therefore shown to be highly correlated to different stages of the experiment, suggesting their usage as primary factors in passive brain-computer interfaces (BCI). In addition, the left and right brain activity indexes reveal the EEG neural modulations of the corresponding hemispheres, which set a feasible reference of activation for an active BCI control system, such as one executing motor imagery tasks. The proposed indicator is applicable to potential passive and active BCI applications for monitoring the subject's physiological state change in real-time, along with providing a means of evaluating the associated signal quality to enhance the BCI performance.

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user’s error expectation of the robot’s current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user’s preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user’s preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.

Teaching an assistive robotic manipulator to move objects in a cluttered table requires demonstrations from expert operators, but what if the experts are individuals with motor disabilities? Batzianoulis et al. propose a learning approach which combines robot autonomy and a brain-computer interfacing that decodes whether the generated trajectories meet the user’s criteria, and show how their system enables the robot to learn individual user’s preferred behaviors using less than five demonstrations that are not necessarily optimal.

3D CNN to Estimate Reaction Time from Multi-Channel EEG

The study of human reaction time (RT) is invaluable not only to understand the sensory-motor functions but also to translate brain signals into machine comprehensible commands that can facilitate augmentative and alternative communication using brain-computer interfaces (BCI). Recent developments in sensor technologies, hardware computational capabilities, and neural network models have significantly helped advance biomedical signal processing research. This study is an attempt to utilize state-of-the-art resources to explore the relationship between human behavioral responses during perceptual decision-making and corresponding brain signals in the form of electroencephalograms (EEG). In this paper, a generalized 3D convolutional neural network (CNN) architecture is introduced to estimate RT for a simple visual task using single-trial multi-channel EEG. Earlier comparable studies have also employed a number of machine learning and deep learning-based models, but none of them considered inter-channel relationships while estimating RT. On the contrary, the use of 3D convolutional layers enabled us to consider the spatial relationship among adjacent channels while simultaneously utilizing spectral information from individual channels. Our model can predict RT with a root mean square error of 91.5 ms and a correlation coefficient of 0.83. These results surpass all the previous results attained from different studies.Clinical relevance Novel approaches to decode brain signals can facilitate research on brain-computer interfaces (BCIs), psychology, and neuroscience, enabling people to utilize assistive devices by root-causing psychological or neuromuscular disorders.

A Review of the Role of Machine Learning Techniques towards Brain–Computer Interface Applications

This review article provides a deep insight into the Brain–Computer Interface (BCI) and the application of Machine Learning (ML) technology in BCIs. It investigates the various types of research undertaken in this realm and discusses the role played by ML in performing different BCI tasks. It also reviews the ML methods used for mental state detection, mental task categorization, emotion classification, electroencephalogram (EEG) signal classification, event-related potential (ERP) signal classification, motor imagery categorization, and limb movement classification. This work explores the various methods employed in BCI mechanisms for feature extraction, selection, and classification and provides a comparative study of reviewed methods. This paper assists the readers to gain information regarding the developments made in BCI and ML domains and future improvements needed for improving and designing better BCI applications.

Electroencephalographic Recording of the Movement-Related Cortical Potential in Ecologically Valid Movements: A Scoping Review

The movement-related cortical potential (MRCP) is a brain signal that can be recorded using surface electroencephalography (EEG) and represents the cortical processes involved in movement preparation. The MRCP has been widely researched in simple, single-joint movements, however, these movements often lack ecological validity. Ecological validity refers to the generalizability of the findings to real-world situations, such as neurological rehabilitation. This scoping review aimed to synthesize the research evidence investigating the MRCP in ecologically valid movement tasks. A search of six electronic databases identified 102 studies that investigated the MRCP during multi-joint movements; 59 of these studies investigated ecologically valid movement tasks and were included in the review. The included studies investigated 15 different movement tasks that were applicable to everyday situations, but these were largely carried out in healthy populations. The synthesized findings suggest that the recording and analysis of MRCP signals is possible in ecologically valid movements, however the characteristics of the signal appear to vary across different movement tasks (i.e., those with greater complexity, increased cognitive load, or a secondary motor task) and different populations (i.e., expert performers, people with Parkinson’s Disease, and older adults). The scarcity of research in clinical populations highlights the need for further research in people with neurological and age-related conditions to progress our understanding of the MRCPs characteristics and to determine its potential as a measure of neurological recovery and intervention efficacy. MRCP-based neuromodulatory interventions applied during ecologically valid movements were only represented in one study in this review as these have been largely delivered during simple joint movements. No studies were identified that used ecologically valid movements to control BCI-driven external devices; this may reflect the technical challenges associated with accurately classifying functional movements from MRCPs. Future research investigating MRCP-based interventions should use movement tasks that are functionally relevant to everyday situations. This will facilitate the application of this knowledge into the rehabilitation setting.

Detection of Error-Related Potentials in Stroke Patients from EEG Using an Artificial Neural Network

Error-related potentials (ErrPs) have been proposed as a means for improving brain-computer interface (BCI) performance by either correcting an incorrect action performed by the BCI or label data for continuous adaptation of the BCI to improve the performance. The latter approach could be relevant within stroke rehabilitation where BCI calibration time could be minimized by using a generalized classifier that is continuously being individualized throughout the rehabilitation session. This may be achieved if data are correctly labelled. Therefore, the aims of this study were: (1) classify single-trial ErrPs produced by individuals with stroke, (2) investigate test-retest reliability, and (3) compare different classifier calibration schemes with different classification methods (artificial neural network, ANN, and linear discriminant analysis, LDA) with waveform features as input for meaningful physiological interpretability. Twenty-five individuals with stroke operated a sham BCI on two separate days where they attempted to perform a movement after which they received feedback (error/correct) while continuous EEG was recorded. The EEG was divided into epochs: ErrPs and NonErrPs. The epochs were classified with a multi-layer perceptron ANN based on temporal features or the entire epoch. Additionally, the features were classified with shrinkage LDA. The features were waveforms of the ErrPs and NonErrPs from the sensorimotor cortex to improve the explainability and interpretation of the output of the classifiers. Three calibration schemes were tested: within-day, between-day, and across-participant. Using within-day calibration, 90% of the data were correctly classified with the entire epoch as input to the ANN; it decreased to 86% and 69% when using temporal features as input to ANN and LDA, respectively. There was poor test-retest reliability between the two days, and the other calibration schemes led to accuracies in the range of 63-72% with LDA performing the best. There was no association between the individuals' impairment level and classification accuracies. The results show that ErrPs can be classified in individuals with stroke, but that user- and session-specific calibration is needed for optimal ErrP decoding with this approach. The use of ErrP/NonErrP waveform features makes it possible to have a physiological meaningful interpretation of the output of the classifiers. The results may have implications for labelling data continuously in BCIs for stroke rehabilitation and thus potentially improve the BCI performance.

BioPyC, an Open-Source Python Toolbox for Offline Electroencephalographic and Physiological Signals Classification

Research on brain–computer interfaces (BCIs) has become more democratic in recent decades, and experiments using electroencephalography (EEG)-based BCIs has dramatically increased. The variety of protocol designs and the growing interest in physiological computing require parallel improvements in processing and classification of both EEG signals and bio signals, such as electrodermal activity (EDA), heart rate (HR) or breathing. If some EEG-based analysis tools are already available for online BCIs with a number of online BCI platforms (e.g., BCI2000 or OpenViBE), it remains crucial to perform offline analyses in order to design, select, tune, validate and test algorithms before using them online. Moreover, studying and comparing those algorithms usually requires expertise in programming, signal processing and machine learning, whereas numerous BCI researchers come from other backgrounds with limited or no training in such skills. Finally, existing BCI toolboxes are focused on EEG and other brain signals but usually do not include processing tools for other bio signals. Therefore, in this paper, we describe BioPyC, a free, open-source and easy-to-use Python platform for offline EEG and biosignal processing and classification. Based on an intuitive and well-guided graphical interface, four main modules allow the user to follow the standard steps of the BCI process without any programming skills: (1) reading different neurophysiological signal data formats, (2) filtering and representing EEG and bio signals, (3) classifying them, and (4) visualizing and performing statistical tests on the results. We illustrate BioPyC use on four studies, namely classifying mental tasks, the cognitive workload, emotions and attention states from EEG signals.

Electrophysiological Decoding of Spatial and Color Processing in Human Prefrontal Cortex

The prefrontal cortex (PFC) plays a pivotal role in goal-directed cognition, yet its representational code remains an open problem with decoding techniques ineffective in disentangling task-relevant variables from PFC. Here we applied regularized linear discriminant analysis to human scalp EEG data and were able to distinguish a mental-rotation task versus a color-perception task with 87% decoding accuracy. Dorsal and ventral areas in lateral PFC provided the dominant features dissociating the two tasks. Our findings show that EEG can reliably decode two independent task states from PFC and emphasize the PFC dorsal/ventral functional specificity in processing the where rotation task versus the what color task.

Yet another artefact rejection study: an exploration of cleaning methods for biological and neuromodulatory noise

Objective. Electroencephalography (EEG) cleaning has been a longstanding issue in the research community. In recent times, huge leaps have been made in the field, resulting in very promising techniques to address the issue. The most widespread ones rely on a family of mathematical methods known as blind source separation (BSS), ideally capable of separating artefactual signals from the brain originated ones. However, corruption of EEG data still remains a problem, especially in real life scenario where a mixture of artefact components affects the signal and thus correctly choosing the correct cleaning procedure can be non trivial. Our aim is here to evaluate and score the plethora of available BSS-based cleaning methods, providing an overview of their advantages and downsides and of their best field of application.Approach. To address this, we here first characterized and modeled different types of artefact, i.e. arising from muscular or blinking activity as well as from transcranial alternate current stimulation. We then tested and scored several BSS-based cleaning procedures on semi-synthetic datasets corrupted by the previously modeled noise sources. Finally, we built a lifelike dataset affected by many artefactual components. We tested an iterative multistep approach combining different BSS steps, aimed at sequentially removing each specific artefactual component.Main results. We did not find an overall best method, as different scenarios require different approaches. We therefore provided an overview of the performance in terms of both reconstruction accuracy and computational burden of each method in different use cases.Significance. Our work provides insightful guidelines for signal cleaning procedures in the EEG related field.

Improving motor imagery classification during induced motor perturbations

Objective.Motor imagery is the mental simulation of movements. It is a common paradigm to design brain-computer interfaces (BCIs) that elicits the modulation of brain oscillatory activity similar to real, passive and induced movements. In this study, we used peripheral stimulation to provoke movements of one limb during the performance of motor imagery tasks. Unlike other works, in which induced movements are used to support the BCI operation, our goal was to test and improve the robustness of motor imagery based BCI systems to perturbations caused by artificially generated movements.Approach.We performed a BCI session with ten participants who carried out motor imagery of three limbs. In some of the trials, one of the arms was moved by neuromuscular stimulation. We analysed 2-class motor imagery classifications with and without movement perturbations. We investigated the performance decrease produced by these disturbances and designed different computational strategies to attenuate the observed classification accuracy drop.Main results.When the movement was induced in a limb not coincident with the motor imagery classes, extracting oscillatory sources of the movement imagination tasks resulted in BCI performance being similar to the control (undisturbed) condition; when the movement was induced in a limb also involved in the motor imagery tasks, the performance drop was significantly alleviated by spatially filtering out the neural noise caused by the stimulation. We also show that the loss of BCI accuracy was accompanied by weaker power of the sensorimotor rhythm. Importantly, this residual power could be used to predict whether a BCI user will perform with sufficient accuracy under the movement disturbances.Significance.We provide methods to ameliorate and even eliminate motor related afferent disturbances during the performance of motor imagery tasks. This can help improving the reliability of current motor imagery based BCI systems.

A Bibliometric Analysis of Human-Machine Interaction Methodology for Electric-Powered Wheelchairs Driving from 1998 to 2020

Electric power wheelchairs (EPWs) enhance the mobility capability of the elderly and the disabled, while the human-machine interaction (HMI) determines how well the human intention will be precisely delivered and how human-machine system cooperation will be efficiently conducted. A bibliometric quantitative analysis of 1154 publications related to this research field, published between 1998 and 2020, was conducted. We identified the development status, contributors, hot topics, and potential future research directions of this field. We believe that the combination of intelligence and humanization of an EPW HMI system based on human-machine collaboration is an emerging trend in EPW HMI methodology research. Particular attention should be paid to evaluating the applicability and benefits of the EPW HMI methodology for the users, as well as how much it contributes to society. This study offers researchers a comprehensive understanding of EPW HMI studies in the past 22 years and latest trends from the evolutionary footprints and forward-thinking insights regarding future research.

Online detection of movement during natural and self-initiated reach-and-grasp actions from EEG signals

Movement intention detection using electroencephalography (EEG) is a challenging but essential component of brain-computer interfaces (BCIs) for people with motor disabilities.Objective.The goal of this study is to develop a new experimental paradigm to perform asynchronous online detection of movement based on low-frequency time-domain EEG features, concretely on movement-related cortical potentials. The paradigm must be easily transferable to people without any residual upper-limb movement function and the BCI must be independent of upper-limb movement onset measurements and external cues.Approach. In a study with non-disabled participants, we evaluated a novel BCI paradigm to detect self-initiated reach-and-grasp movements. Two experimental conditions were involved. In one condition, participants performed reach-and-grasp movements to a target and simultaneously shifted their gaze towards it. In a control condition, participants solely shifted their gaze towards the target (oculomotor task). The participants freely decided when to initiate the tasks. After eye artefact correction, the EEG signals were time-locked to the saccade onset and the resulting amplitude features were exploited on a hierarchical classification approach to detect movement asynchronously.Main results. With regards to BCI performance, 54.1% (14.4% SD) of the movements were correctly identified, and all participants achieved a performance above chance-level (around 12%). An average of 21.5% (14.1% SD) of the oculomotor tasks were falsely detected as upper-limb movement. In an additional rest condition, 1.7 (1.6 SD) false positives per minute were measured. Through source imaging, movement information was mapped to sensorimotor, posterior parietal and occipital areas.Significance. We present a novel approach for movement detection using EEG signals which does not rely on upper-limb movement onset measurements or on the presentation of external cues. The participants' behaviour closely matches the natural behaviour during goal-directed reach-and-grasp movements, which also constitutes an advantage with respect to current BCI protocols.

Online control of an assistive active glove by slow cortical signals in patients with amyotrophic lateral sclerosis

Objective.A brain-computer interface (BCI) allows users to control external devices using brain signals that can be recorded non-invasively via electroencephalography (EEG). Movement related cortical potentials (MRCPs) are an attractive option for BCI control since they arise naturally during movement execution and imagination, and therefore, do not require an extensive training. This study tested the feasibility of online detection of reaching and grasping using MRCPs for the application in patients suffering from amyotrophic lateral sclerosis (ALS).Approach.A BCI system was developed to trigger closing of a soft assistive glove by detecting a reaching movement. The custom-made software application included data collection, a novel method for collecting the input data for classifier training from the offline recordings based on a sliding window approach, and online control of the glove. Eight healthy subjects and two ALS patients were recruited to test the developed BCI system. They performed assessment blocks without the glove active (NG), in which the movement detection was indicated by a sound feedback, and blocks (G) in which the glove was controlled by the BCI system. The true positive rate (TPR) and the positive predictive value (PPV) were adopted as the outcome measures. Correlation analysis between forehead EEG detecting ocular artifacts and sensorimotor area EEG was conducted to confirm the validity of the results.Main results.The overall median TPR and PPV were >0.75 for online BCI control, in both healthy individuals and patients, with no significant difference across the blocks (NG versus G).Significance.The results demonstrate that cortical activity during reaching can be detected and used to control an external system with a limited amount of training data (30 trials). The developed BCI system can be used to provide grasping assistance to ALS patients.

Brain-Computer Interface (BCI) Control of a Virtual Assistant in a Smartphone to Manage Messaging Applications

Brain-computer interfaces (BCI) are a type of assistive technology that uses the brain signals of users to establish a communication and control channel between them and an external device. BCI systems may be a suitable tool to restore communication skills in severely motor-disabled patients, as BCI do not rely on muscular control. The loss of communication is one of the most negative consequences reported by such patients. This paper presents a BCI system focused on the control of four mainstream messaging applications running in a smartphone: WhatsApp, Telegram, e-mail and short message service (SMS). The control of the BCI is achieved through the well-known visual P300 row-column paradigm (RCP), allowing the user to select control commands as well as spelling characters. For the control of the smartphone, the system sends synthesized voice commands that are interpreted by a virtual assistant running in the smartphone. Four tasks related to the four mentioned messaging services were tested with 15 healthy volunteers, most of whom were able to accomplish the tasks, which included sending free text e-mails to an address proposed by the subjects themselves. The online performance results obtained, as well as the results of subjective questionnaires, support the viability of the proposed system.