Review of real brain-controlled wheelchairs

Considerations and discussions on the clear definition and definite scope of brain-computer interfaces

Brain-computer interface (BCI) is a revolutionizing human-computer interaction with potential applications in both medical and non-medical fields, emerging as a cutting-edge and trending research direction. Increasing numbers of groups are engaging in BCI research and development. However, in recent years, there has been some confusion regarding BCI, including misleading and hyped propaganda about BCI, and even non-BCI technologies being labeled as BCI. Therefore, a clear definition and a definite scope for BCI are thoroughly considered and discussed in the paper, based on the existing definitions of BCI, including the six key or essential components of BCI. In the review, different from previous definitions of BCI, BCI paradigms and neural coding are explicitly included in the clear definition of BCI provided, and the BCI user (the brain) is clearly identified as a key component of the BCI system. Different people may have different viewpoints on the definition and scope of BCI, as well as some related issues, which are discussed in the article. This review argues that a clear definition and definite scope of BCI will benefit future research and commercial applications. It is hoped that this review will reduce some of the confusion surrounding BCI and promote sustainable development in this field.

Enhancing SSVEP-BCI Performance Under Fatigue State Using Dynamic Stopping Strategy

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have emerged as a prominent technology due to their high information transfer rate, rapid calibration time, and robust signal-to-noise ratio. However, a critical challenge for practical applications is performance degradation caused by user fatigue during prolonged use. This work proposes novel methods to address this challenge by dynamically adjusting data acquisition length and updating detection models based on a fatigue-aware stopping strategy. Two 16-target SSVEP-BCIs were employed, one using low-frequency and the other using high-frequency stimulation. A self-recorded fatigue dataset from 24 subjects was utilized for extensive evaluation. A simulated online experiment demonstrated that the proposed methods outperform the conventional fixed stopping strategy in terms of classification accuracy, information transfer rate, and selection time, irrespective of stimulation frequency. These findings suggest that the proposed approach can significantly improve SSVEP-BCI performance under fatigue conditions, leading to superior performance during extended use.

Front-End Replication Dynamic Window (FRDW) for Online Motor Imagery Classification

Motor imagery (MI) is a classical paradigm in electroencephalogram (EEG) based brain-computer interfaces (BCIs). Online accurate and fast decoding is very important to its successful applications. This paper proposes a simple yet effective front-end replication dynamic window (FRDW) algorithm for this purpose. Dynamic windows enable the classification based on a test EEG trial shorter than those used in training, improving the decision speed; front-end replication fills a short test EEG trial to the length used in training, improving the classification accuracy. Within-subject and cross-subject online MI classification experiments on three public datasets, with three different classifiers and three different data augmentation approaches, demonstrated that FRDW can significantly increase the information transfer rate in MI decoding. Additionally, FR can also be used in training data augmentation. FRDW helped win national champion of the China BCI Competition in 2022.

A Calibration-Free Hybrid Approach Combining SSVEP and EOG for Continuous Control

While SSVEP-BCI has been widely developed to control external devices, most of them rely on the discrete control strategy. The continuous SSVEP-BCI enables users to continuously deliver commands and receive real-time feedback from the devices, but it suffers from the transition state problem, a period the erroneous recognition, when users shift their gazes between targets. To resolve this issue, we proposed a novel calibration-free Bayesian approach by hybridizing SSVEP and electrooculography (EOG). First, canonical correlation analysis (CCA) was applied to detect the evoked SSVEPs, and saccade during the gaze shift was detected by EOG data using an adaptive threshold method. Then, the new target after the gaze shift was recognized based on a Bayesian optimization approach, which combined the detection of SSVEP and saccade together and calculated the optimized probability distribution of the targets. Eighteen healthy subjects participated in the offline and online experiments. The offline experiments showed that the proposed hybrid BCI had significantly higher overall continuous accuracy and shorter gaze-shifting time compared to FBCCA, CCA, MEC, and PSDA. In online experiments, the proposed hybrid BCI significantly outperformed CCA-based SSVEP-BCI in terms of continuous accuracy (77.61 ± 1.36%vs. 68.86 ± 1.08% and gaze-shifting time (0.93 ± 0.06s vs. 1.94 ± 0.08s). Additionally, participants also perceived a significant improvement over the CCA-based SSVEP-BCI when the newly proposed decoding approach was used. These results validated the efficacy of the proposed hybrid Bayesian approach for the BCI continuous control without any calibration. This study provides an effective framework for combining SSVEP and EOG, and promotes the potential applications of plug-and-play BCIs in continuous control.

Evaluation of Single-Trial Classification to Control a Visual ERP-BCI under a Situation Awareness Scenario

An event-related potential (ERP)-based brain-computer interface (BCI) can be used to monitor a user's cognitive state during a surveillance task in a situational awareness context. The present study explores the use of an ERP-BCI for detecting new planes in an air traffic controller (ATC). Two experiments were conducted to evaluate the impact of different visual factors on target detection. Experiment 1 validated the type of stimulus used and the effect of not knowing its appearance location in an ERP-BCI scenario. Experiment 2 evaluated the effect of the size of the target stimulus appearance area and the stimulus salience in an ATC scenario. The main results demonstrate that the size of the plane appearance area had a negative impact on the detection performance and on the amplitude of the P300 component. Future studies should address this issue to improve the performance of an ATC in stimulus detection using an ERP-BCI.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Assembling global and local spatial-temporal filters to extract discriminant information of EEG in RSVP task

Objective.Brain-computer interface (BCI) system has developed rapidly in the past decade. And rapid serial visual presentation (RSVP) is an important BCI paradigm to detect the targets in high-speed image streams. For decoding electroencephalography (EEG) in RSVP task, the ensemble-model methods have better performance than the single-model ones.Approach.This study proposed a method based on ensemble learning to extract discriminant information of EEG. An extreme gradient boosting framework was utilized to sequentially generate the sub models, including one global spatial-temporal filter and a group of local ones. EEG was reshaped into a three-dimensional form by remapping the electrode dimension into a 2D array to learn the spatial-temporal features from real local space.Main results.A benchmark RSVP EEG dataset was utilized to evaluate the performance of the proposed method, where EEG data of 63 subjects were analyzed. Compared with several state-of-the-art methods, the spatial-temporal patterns of proposed method were more consistent with P300, and the proposed method can provide significantly better classification performance.Significance.The ensemble model in this study was end-to-end optimized, which can avoid error accumulation. The sub models optimized by gradient boosting theory can extract discriminant information complementarily and non-redundantly.

Trends in EEG signal feature extraction applications

This paper will focus on electroencephalogram (EEG) signal analysis with an emphasis on common feature extraction techniques mentioned in the research literature, as well as a variety of applications that this can be applied to. In this review, we cover single and multi-dimensional EEG signal processing and feature extraction techniques in the time domain, frequency domain, decomposition domain, time-frequency domain, and spatial domain. We also provide pseudocode for the methods discussed so that they can be replicated by practitioners and researchers in their specific areas of biomedical work. Furthermore, we discuss artificial intelligence applications such as assistive technology, neurological disease classification, brain-computer interface systems, as well as their machine learning integration counterparts, to complete the overall pipeline design for EEG signal analysis. Finally, we discuss future work that can be innovated in the feature extraction domain for EEG signal analysis.

Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain-computer interfaces

Objective.Covariance matrices of noisy multichannel electroencephalogram (EEG) time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain-computer interfaces (BCI) based on event-related potentials (ERP) and a linear discriminant analysis (LDA) classifier, the state of the art covariance estimation uses shrinkage regularization. As this is a general covariance regularization approach, we aim at improving LDA further by better exploiting the domain-specific characteristics of the EEG to regularize the covariance estimates.Approach.We propose to enforce a block-Toeplitz structure for the covariance matrix of the LDA, which implements an assumption of signal stationarity in short time windows for each channel.Main results.An offline re-analysis of data collected from 213 subjects under 13 different event-related potential BCI protocols showed a significantly increased binary classification performance of this 'ToeplitzLDA' compared to shrinkage regularized LDA (up to 6 AUC points,p < 0.001) and Riemannian classification approaches (up to 2 AUC points,p < 0.001). In an unsupervised visual speller application, this improvement would translate to a relative reduction of spelling errors by 81% on average for 25 subjects. Additionally, aside from lower memory and reduced time complexity for LDA training, ToeplitzLDA proves to be robust against drastic increases of the number of temporal features.Significance.The proposed covariance estimation allows BCI researchers to improve classification rates and reduce calibration times of BCI protocols using event-related potentials and thus support the usability of corresponding applications. Its lower computational and memory needs could make it a valuable algorithm especially for mobile BCIs.

On the feasibility of simple brain-computer interface systems for enabling children with severe physical disabilities to explore independent movement

Introduction

Children with severe physical disabilities are denied their fundamental right to move, restricting their development, independence, and participation in life. Brain-computer interfaces (BCIs) could enable children with complex physical needs to access power mobility (PM) devices, which could help them move safely and independently. BCIs have been studied for PM control for adults but remain unexamined in children. In this study, we explored the feasibility of BCI-enabled PM control for children with severe physical disabilities, assessing BCI performance, standard PM skills and tolerability of BCI.

Materials and methods

Patient-oriented pilot trial. Eight children with quadriplegic cerebral palsy attended two sessions where they used a simple, commercial-grade BCI system to activate a PM trainer device. Performance was assessed through controlled activation trials (holding the PM device still or activating it upon verbal and visual cueing), and basic PM skills (driving time, number of activations, stopping) were assessed through distance trials. Setup and calibration times, headset tolerability, workload, and patient/caregiver experience were also evaluated.

Results

All participants completed the study with favorable tolerability and no serious adverse events or technological challenges. Average control accuracy was 78.3 ± 12.1%, participants were more reliably able to activate (95.7 ± 11.3%) the device than hold still (62.1 ± 23.7%). Positive trends were observed between performance and prior BCI experience and age. Participants were able to drive the PM device continuously an average of 1.5 meters for 3.0 s. They were able to stop at a target 53.1 ± 23.3% of the time, with significant variability. Participants tolerated the headset well, experienced mild-to-moderate workload and setup/calibration times were found to be practical. Participants were proud of their performance and both participants and families were eager to participate in future power mobility sessions.

Discussion

BCI-enabled PM access appears feasible in disabled children based on evaluations of performance, tolerability, workload, and setup/calibration. Performance was comparable to existing pediatric BCI literature and surpasses established cut-off thresholds (70%) of “effective” BCI use. Participants exhibited PM skills that would categorize them as “emerging operational learners.” Continued exploration of BCI-enabled PM for children with severe physical disabilities is justified.

Sliding-Mode Nonlinear Predictive Control of Brain-Controlled Mobile Robots

In this article, we develop a robust sliding-mode nonlinear predictive controller for brain-controlled robots with enhanced performance, safety, and robustness. First, the kinematics and dynamics of a mobile robot are built. After that, the proposed controller is developed by cascading a predictive controller and a smooth sliding-mode controller. The predictive controller integrates the human intention tracking with safety guarantee objectives into an optimization problem to minimize the invasion to human intention while maintaining robot safety. The smooth sliding-mode controller is designed to achieve robust desired velocity tracking. The results of human-in-the-loop simulation and robotic experiments both show the efficacy and robust performance of the proposed controller. This work provides an enabling design to enhance the future research and development of brain-controlled robots.

An autonomous hybrid brain-computer interface system combined with eye-tracking in virtual environment

BACKGROUND

Brain-computer interface (BCI) has become an effective human-machine interactive way. However, the performance of the traditional BCI system needs to be further improved, such as flexibility, robustness, and accuracy. We aim to develop an autonomous hybrid BCI system combined with eye-tracking for the control tasks in the virtual environment.

NEW METHOD

This work developed an autonomous control strategy and proposed an effective fusion method for electroencephalogram (EEG) and eye tracking. For the autonomous control, the sliding window method was adopted to analyze the user's eye-gaze data. When the variance of eye-gaze data was less than the threshold, target recognition was triggered. EEG and eye-gaze data were synchronously collected and fused for classification. In addition, a fusion method based on particle swarm optimization (PSO) was proposed, which can find the best fusion weights to adapt to the differences of single modalities.

RESULTS

EEG data and eye-gaze data of 15 subjects in steady-state visual evoked potentials (SSVEP) tasks were collected to evaluate the effectiveness of the hybrid BCI system. The results showed that the PSO fusion method performed best in all fusion methods. And the proposed hybrid BCI system obtained higher accuracy and information transfer rate (ITR) than the single-modality.

COMPARISON WITH EXISTING METHODS

The PSO fusion method was compared with average weighting fusion, prior weighting fusion, support vector machine, decision tree, random forest, and extreme random tree.

CONCLUSION

The proposed methods of autonomous control and dual-modal fusion can improve the flexibility, robustness and classification performance of the hybrid BCI system.

Unlocking Independence: Exploring Movement with Brain-Computer Interface for Children with Severe Physical Disabilities

Children with severe physical disabilities are often unable to independently explore their environments, further contributing to complex developmental delays. Brain-computer interfaces (BCIs) could be a novel access method to power mobility for children who struggle to use existing alternate access technologies, allowing them to reap the developmental, social, and psychological benefits of independent mobility. In this pilot study we demonstrated that children with quadriplegic cerebral palsy can use a simple BCI system to explore movement with a power mobility device. Four children were able to use the BCI to drive forward at least 7m, although more practice is needed to achieve more efficient driving skills through sustained BCI activations.

Motor-Imagery EEG-Based BCIs in Wheelchair Movement and Control: A Systematic Literature Review

The pandemic emergency of the coronavirus disease 2019 (COVID-19) shed light on the need for innovative aids, devices, and assistive technologies to enable people with severe disabilities to live their daily lives. EEG-based Brain-Computer Interfaces (BCIs) can lead individuals with significant health challenges to improve their independence, facilitate participation in activities, thus enhancing overall well-being and preventing impairments. This systematic review provides state-of-the-art applications of EEG-based BCIs, particularly those using motor-imagery (MI) data, to wheelchair control and movement. It presents a thorough examination of the different studies conducted since 2010, focusing on the algorithm analysis, features extraction, features selection, and classification techniques used as well as on wheelchair components and performance evaluation. The results provided in this paper could highlight the limitations of current biomedical instrumentations applied to people with severe disabilities and bring focus to innovative research topics.

SSVEP-Based Brain-Computer Interface With a Limited Number of Frequencies Based on Dual-Frequency Biased Coding

How to encode as many targets as possible with a limited-frequency resource is a difficult problem in the practical use of a steady-state visual evoked potential (SSVEP) based brain-computer interface (BCI) speller. To solve this problem, this study developed a novel method called dual-frequency biased coding (DFBC) to tag targets in a SSVEP-based 48-character virtual speller, in which each target is encoded with a permutation sequence consisting of two permuted flickering periods that flash at different frequencies. The proposed paradigm was validated by 11 participants in an offline experiment and 7 participants in an online experiment. Three occipital channels (O1, Oz, and O2) were used to obtain the SSVEP signals for identifying the targets. Based on the coding characteristics of the DFBC method, the proposed approach has the ability of self-correction and thus achieves an accuracy of 76.6% and 79.3% for offline and online experiments, respectively, which outperforms the traditional multiple frequencies sequential coding (MFSC) method. This study demonstrates that DFBC is an efficient method for coding a high number of SSVEP targets with a small number of available frequencies.

A Roadmap Towards Standards for Neurally Controlled End Effectors

The control and manipulation of various types of end effectors such as powered exoskeletons, prostheses, and ‘neural’ cursors by brain-machine interface (BMI) systems has been the target of many research projects. A seamless “plug and play” interface between any BMI and end effector is desired, wherein similar user's intent cause similar end effectors to behave identically. This report is based on the outcomes of an IEEE Standards Association Industry Connections working group on End Effectors for Brain-Machine Interfacing that convened to identify and address gaps in the existing standards for BMI-based solutions with a focus on the end-effector component. A roadmap towards standardization of end effectors for BMI systems is discussed by identifying current device standards that are applicable for end effectors. While current standards address basic electrical and mechanical safety, and to some extent, performance requirements, several gaps exist pertaining to unified terminologies, data communication protocols, patient safety and risk mitigation.

Effect of Distracting Background Speech in an Auditory Brain-Computer Interface

Studies so far have analyzed the effect of distractor stimuli in different types of brain-computer interface (BCI). However, the effect of a background speech has not been studied using an auditory event-related potential (ERP-BCI), a convenient option when the visual path cannot be adopted by users. Thus, the aim of the present work is to examine the impact of a background speech on selection performance and user workload in auditory BCI systems. Eleven participants tested three conditions: (i) auditory BCI control condition, (ii) auditory BCI with a background speech to ignore (non-attentional condition), and (iii) auditory BCI while the user has to pay attention to the background speech (attentional condition). The results demonstrated that, despite no significant differences in performance, shared attention to auditory BCI and background speech required a higher cognitive workload. In addition, the P300 target stimuli in the non-attentional condition were significantly higher than those in the attentional condition for several channels. The non-attentional condition was the only condition that showed significant differences in the amplitude of the P300 between target and non-target stimuli. The present study indicates that background speech, especially when it is attended to, is an important interference that should be avoided while using an auditory BCI.

Implementation of an SSVEP-based intelligent home service robot system

BACKGROUND

People with severe neuromuscular disorders caused by an accident or congenital disease cannot normally interact with the physical environment. The intelligent robot technology offers the possibility to solve this problem. However, the robot can hardly carry out the task without understanding the subject's intention as it relays on speech or gestures. Brain-computer interface (BCI), a communication system that operates external devices by directly converting brain activity into digital signals, provides a solution for this.

OBJECTIVE

In this study, a noninvasive BCI-based humanoid robotic system was designed and implemented for home service.

METHODS

A humanoid robot that is equipped with multi-sensors navigates to the object placement area under the guidance of a specific symbol "Naomark", which has a unique ID, and then sends the information of the scanned object back to the user interface. Based on this information, the subject gives commands to the robot to grab the wanted object and give it to the subject. To identify the subject's intention, the channel projection-based canonical correlation analysis (CP-CCA) method was utilized for the steady state visual evoked potential-based BCI system.

RESULTS

The offline results showed that the average classification accuracy of all subjects reached 90%, and the online task completion rate was over 95%.

CONCLUSION

Users can complete the grab task with minimum commands, avoiding the control burden caused by complex commands. This would provide a useful assistance means for people with severe motor impairment in their daily life.

Modeling of Human Operator Behavior for Brain-Actuated Mobile Robots Steering

Human operator control of brain-actuated robot steering based on electroencephalograph (EEG)-signals is a complex behavior consisting of surroundings perceiving, decision making, and commands issuing and differs among individual operators. However, no existing models allow decoupling the user from the loop to improve the system design and testing process, which can capture such behavior of a brain-actuated robot. To address this problem, in this paper, we propose an operator brain-controlled steering model consisting of an operator decision model based on the queuing network (QN) cognitive architecture and a brain-machine interface (BMI) performance model. The QN-based operator decision model can mimic the human decision process with the individual operator differences considered. The new BMI performance model is built to represent the varied accuracy of BMI during brain-controlled direction operations. Furthermore, the model is simulated and validated against the results of human operator-in-the-loop experiments. The results show that the proposed model can reproduce the behavior of human operators thanks to its similar direction control performance.

Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review

Brain-Computer Interface (BCI), in essence, aims at controlling different assistive devices through the utilization of brain waves. It is worth noting that the application of BCI is not limited to medical applications, and hence, the research in this field has gained due attention. Moreover, the significant number of related publications over the past two decades further indicates the consistent improvements and breakthroughs that have been made in this particular field. Nonetheless, it is also worth mentioning that with these improvements, new challenges are constantly discovered. This article provides a comprehensive review of the state-of-the-art of a complete BCI system. First, a brief overview of electroencephalogram (EEG)-based BCI systems is given. Secondly, a considerable number of popular BCI applications are reviewed in terms of electrophysiological control signals, feature extraction, classification algorithms, and performance evaluation metrics. Finally, the challenges to the recent BCI systems are discussed, and possible solutions to mitigate the issues are recommended.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Indoor Simulated Training Environment for Brain-Controlled Wheelchair Based on Steady-State Visual Evoked Potentials

Brain-controlled wheelchair (BCW) has the potential to improve the quality of life for people with motor disabilities. A lot of training is necessary for users to learn and improve BCW control ability and the performances of BCW control are crucial for patients in daily use. In consideration of safety and efficiency, an indoor simulated training environment is built up in this paper to improve the performance of BCW control. The indoor simulated environment mainly realizes BCW implementation, simulated training scenario setup, path planning and recommendation, simulated operation, and scoring. And the BCW is based on steady-state visual evoked potentials (SSVEP) and the filter bank canonical correlation analysis (FBCCA) is used to analyze the electroencephalography (EEG). Five tasks include individual accuracy, simple linear path, obstacles avoidance, comprehensive steering scenarios, and evaluation task are designed, 10 healthy subjects were recruited and carried out the 7-days training experiment to assess the performance of the training environment. Scoring and command-consuming are conducted to evaluate the improvement before and after training. The results indicate that the average accuracy is 93.55% and improves from 91.05% in the first stage to 96.05% in the second stage (p = 0.001). Meanwhile, the average score increases from 79.88 in the first session to 96.66 in the last session and tend to be stable (p < 0.001). The average number of commands and collisions to complete the tasks decreases significantly with or without the approximate shortest path (p < 0.001). These results show that the performance of subjects in BCW control achieves improvement and verify the feasibility and effectiveness of the proposed simulated training environment.

Brain-machine interfaces for controlling lower-limb powered robotic systems

OBJECTIVE

Lower-limb, powered robotics systems such as exoskeletons and orthoses have emerged as novel robotic interventions to assist or rehabilitate people with walking disabilities. These devices are generally controlled by certain physical maneuvers, for example pressing buttons or shifting body weight. Although effective, these control schemes are not what humans naturally use. The usability and clinical relevance of these robotics systems could be further enhanced by brain-machine interfaces (BMIs). A number of preliminary studies have been published on this topic, but a systematic understanding of the experimental design, tasks, and performance of BMI-exoskeleton systems for restoration of gait is lacking.

APPROACH

To address this gap, we applied standard systematic review methodology for a literature search in PubMed and EMBASE databases and identified 11 studies involving BMI-robotics systems. The devices, user population, input and output of the BMIs and robot systems respectively, neural features, decoders, denoising techniques, and system performance were reviewed and compared.

MAIN RESULTS

Results showed BMIs classifying walk versus stand tasks are the most common. The results also indicate that electroencephalography (EEG) is the only recording method for humans. Performance was not clearly presented in most of the studies. Several challenges were summarized, including EEG denoising, safety, responsiveness and others.

SIGNIFICANCE

We conclude that lower-body powered exoskeletons with automated gait intention detection based on BMIs open new possibilities in the assistance and rehabilitation fields, although the current performance, clinical benefits and several key challenging issues indicate that additional research and development is required to deploy these systems in the clinic and at home. Moreover, rigorous EEG denoising techniques, suitable performance metrics, consistent trial reporting, and more clinical trials are needed to advance the field.

A review and experimental study on the application of classifiers and evolutionary algorithms in EEG-based brain-machine interface systems

OBJECTIVE

Considering the importance and the near-future development of noninvasive brain-machine interface (BMI) systems, this paper presents a comprehensive theoretical-experimental survey on the classification and evolutionary methods for BMI-based systems in which EEG signals are used.

APPROACH

The paper is divided into two main parts. In the first part, a wide range of different types of the base and combinatorial classifiers including boosting and bagging classifiers and evolutionary algorithms are reviewed and investigated. In the second part, these classifiers and evolutionary algorithms are assessed and compared based on two types of relatively widely used BMI systems, sensory motor rhythm-BMI and event-related potentials-BMI. Moreover, in the second part, some of the improved evolutionary algorithms as well as bi-objective algorithms are experimentally assessed and compared.

MAIN RESULTS

In this study two databases are used, and cross-validation accuracy (CVA) and stability to data volume (SDV) are considered as the evaluation criteria for the classifiers. According to the experimental results on both databases, regarding the base classifiers, linear discriminant analysis and support vector machines with respect to CVA evaluation metric, and naive Bayes with respect to SDV demonstrated the best performances. Among the combinatorial classifiers, four classifiers, Bagg-DT (bagging decision tree), LogitBoost, and GentleBoost with respect to CVA, and Bagging-LR (bagging logistic regression) and AdaBoost (adaptive boosting) with respect to SDV had the best performances. Finally, regarding the evolutionary algorithms, single-objective invasive weed optimization (IWO) and bi-objective nondominated sorting IWO algorithms demonstrated the best performances.

SIGNIFICANCE

We present a general survey on the base and the combinatorial classification methods for EEG signals (sensory motor rhythm and event-related potentials) as well as their optimization methods through the evolutionary algorithms. In addition, experimental and statistical significance tests are carried out to study the applicability and effectiveness of the reviewed methods.

Early Detection of Hemodynamic Responses Using EEG: A Hybrid EEG-fNIRS Study

Enhanced classification accuracy and a sufficient number of commands are highly demanding in brain computer interfaces (BCIs). For a successful BCI, early detection of brain commands in time is essential. In this paper, we propose a novel classifier using a modified vector phase diagram and the power of electroencephalography (EEG) signal for early prediction of hemodynamic responses. EEG and functional near-infrared spectroscopy (fNIRS) signals for a motor task (thumb tapping) were obtained concurrently. Upon the resting state threshold circle in the vector phase diagram that uses the maximum values of oxy- and deoxy-hemoglobin (ΔHbO and ΔHbR) during the resting state, we introduce a secondary (inner) threshold circle using the ΔHbO and ΔHbR magnitudes during the time window of 1 s where an EEG activity is noticeable. If the trajectory of ΔHbO and ΔHbR touches the resting state threshold circle after passing through the inner circle, this indicates that ΔHbO was increasing and ΔHbR was decreasing (i.e., the start of a hemodynamic response). It takes about 0.5 s for an fNIRS signal to cross the resting state threshold circle after crossing the EEG-based circle. Thus, an fNIRS-based BCI command can be generated in 1.5 s. We achieved an improved accuracy of 86.0% using the proposed method in comparison with the 63.8% accuracy obtained using linear discriminant analysis in a window of 0~1.5 s. Moreover, the active brain locations (identified using the proposed scheme) were spatially specific when a t-map was made after 10 s of stimulation. These results demonstrate the possibility of enhancing the classification accuracy for a brain-computer interface with a time window of 1.5 s using the proposed method.

A review of disability EEG based wheelchair control system: Coherent taxonomy, open challenges and recommendations

CONTEXT

Intelligent wheelchair technology has recently been utilised to address several mobility problems. Techniques based on brain-computer interface (BCI) are currently used to develop electric wheelchairs. Using human brain control in wheelchairs for people with disability has elicited widespread attention due to its flexibility.

OBJECTIVE

This study aims to determine the background of recent studies on wheelchair control based on BCI for disability and map the literature survey into a coherent taxonomy. The study intends to identify the most important aspects in this emerging field as an impetus for using BCI for disability in electric-powered wheelchair (EPW) control, which remains a challenge. The study also attempts to provide recommendations for solving other existing limitations and challenges.

METHODS

We systematically searched all articles about EPW control based on BCI for disability in three popular databases: ScienceDirect, IEEE and Web of Science. These databases contain numerous articles that considerably influenced this field and cover most of the relevant theoretical and technical issues.

RESULTS

We selected 100 articles on the basis of our inclusion and exclusion criteria. A large set of articles (55) discussed on developing real-time wheelchair control systems based on BCI for disability signals. Another set of articles (25) focused on analysing BCI for disability signals for wheelchair control. The third set of articles (14) considered the simulation of wheelchair control based on BCI for disability signals. Four articles designed a framework for wheelchair control based on BCI for disability signals. Finally, one article reviewed concerns regarding wheelchair control based on BCI for disability signals.

DISCUSSION

Since 2007, researchers have pursued the possibility of using BCI for disability in EPW control through different approaches. Regardless of type, articles have focused on addressing limitations that impede the full efficiency of BCI for disability and recommended solutions for these limitations.

CONCLUSIONS

Studies on wheelchair control based on BCI for disability considerably influence society due to the large number of people with disability. Therefore, we aim to provide researchers and developers with a clear understanding of this platform and highlight the challenges and gaps in the current and future studies.

Evaluation of Switch and Continuous Navigation Paradigms to Command a Brain-Controlled Wheelchair

A brain-computer interface (BCI) is a technology allowing patients with severe motor dysfunctions to use their electroencephalographic signals to create a communication channel to control devices. The objective of this paper is to study the feasibility of continuous and switch control modes for a brain-controlled wheelchair (BCW) using sensorimotor rhythms (SMR) modulated through a right-hand motor imagery task. Previous studies, which used a continuous navigation control with SMR, have reported the difficulty of maintaining the motor imagery task for a long time, especially for the forward command. The switch control has been presented as a proposal that may help to solve this issue since this task is only used temporary for either disabling or enabling the movement. Regarding the methodology, 10 of 15 able-bodied users, who had overcome the criterion of 30% error rate in the calibration phase, controlled the BCW using both paradigms. The navigation tasks consisted of a straight path divided in five sections: in three of them the users had to move forward, and in the other two the users had to maintain their position. To assess user performance in the device management, a usability approach was adopted, measuring the factors of effectiveness, efficiency, and satisfaction. Then, variables related to the time employed and commands selected by the user or parameters related to the confusion matrix were applied. In addition, the scores in NASA-TLX and two ad hoc questionnaires were considered to discuss the user experience controlling the wheelchair. Despite the results showed that the best system for a specific user relies on his/her abilities and preferences, the switch control mode obtained better accuracy (0.59 ± 0.17 for continuous and 0.72 ± 0.05 for switch). Furthermore, the switch paradigm can be recommended for the advance sections as with it users could complete the advance sections in less time (42.2 ± 28.7 s for continuous and 15.47 ± 3.43 s for switch), while the continuous mode seems to be better at keeping the wheelchair stopped (42.45 ± 16.01 s for continuous and 24.35 ± 10.94 s for switch).

Feature Extraction and Classification Methods for Hybrid fNIRS-EEG Brain-Computer Interfaces

In this study, a brain-computer interface (BCI) framework for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) for locked-in syndrome (LIS) patients is investigated. Brain tasks, channel selection methods, and feature extraction and classification algorithms available in the literature are reviewed. First, we categorize various types of patients with cognitive and motor impairments to assess the suitability of BCI for each of them. The prefrontal cortex is identified as a suitable brain region for imaging. Second, the brain activity that contributes to the generation of hemodynamic signals is reviewed. Mental arithmetic and word formation tasks are found to be suitable for use with LIS patients. Third, since a specific targeted brain region is needed for BCI, methods for determining the region of interest are reviewed. The combination of a bundled-optode configuration and threshold-integrated vector phase analysis turns out to be a promising solution. Fourth, the usable fNIRS features and EEG features are reviewed. For hybrid BCI, a combination of the signal peak and mean fNIRS signals and the highest band powers of EEG signals is promising. For classification, linear discriminant analysis has been most widely used. However, further research on vector phase analysis as a classifier for multiple commands is desirable. Overall, proper brain region identification and proper selection of features will improve classification accuracy. In conclusion, five future research issues are identified, and a new BCI scheme, including brain therapy for LIS patients and using the framework of hybrid fNIRS-EEG BCI, is provided.

Control of a 7-DOF Robotic Arm System With an SSVEP-Based BCI

Although robot technology has been successfully used to empower people who suffer from motor disabilities to increase their interaction with their physical environment, it remains a challenge for individuals with severe motor impairment, who do not have the motor control ability to move robots or prosthetic devices by manual control. In this study, to mitigate this issue, a noninvasive brain-computer interface (BCI)-based robotic arm control system using gaze based steady-state visual evoked potential (SSVEP) was designed and implemented using a portable wireless electroencephalogram (EEG) system. A 15-target SSVEP-based BCI using a filter bank canonical correlation analysis (FBCCA) method allowed users to directly control the robotic arm without system calibration. The online results from 12 healthy subjects indicated that a command for the proposed brain-controlled robot system could be selected from 15 possible choices in 4[Formula: see text]s (i.e. 2[Formula: see text]s for visual stimulation and 2[Formula: see text]s for gaze shifting) with an average accuracy of 92.78%, resulting in a 15 commands/min transfer rate. Furthermore, all subjects (even naive users) were able to successfully complete the entire move-grasp-lift task without user training. These results demonstrated an SSVEP-based BCI could provide accurate and efficient high-level control of a robotic arm, showing the feasibility of a BCI-based robotic arm control system for hand-assistance.

Brain-Computer Interface application: auditory serial interface to control a two-class motor-imagery-based wheelchair

BACKGROUND

Certain diseases affect brain areas that control the movements of the patients' body, thereby limiting their autonomy and communication capacity. Research in the field of Brain-Computer Interfaces aims to provide patients with an alternative communication channel not based on muscular activity, but on the processing of brain signals. Through these systems, subjects can control external devices such as spellers to communicate, robotic prostheses to restore limb movements, or domotic systems. The present work focus on the non-muscular control of a robotic wheelchair.

METHOD

A proposal to control a wheelchair through a Brain-Computer Interface based on the discrimination of only two mental tasks is presented in this study. The wheelchair displacement is performed with discrete movements. The control signals used are sensorimotor rhythms modulated through a right-hand motor imagery task or mental idle state. The peculiarity of the control system is that it is based on a serial auditory interface that provides the user with four navigation commands. The use of two mental tasks to select commands may facilitate control and reduce error rates compared to other endogenous control systems for wheelchairs.

RESULTS

Seventeen subjects initially participated in the study; nine of them completed the three sessions of the proposed protocol. After the first calibration session, seven subjects were discarded due to a low control of their electroencephalographic signals; nine out of ten subjects controlled a virtual wheelchair during the second session; these same nine subjects achieved a medium accuracy level above 0.83 on the real wheelchair control session.

CONCLUSION

The results suggest that more extensive training with the proposed control system can be an effective and safe option that will allow the displacement of a wheelchair in a controlled environment for potential users suffering from some types of motor neuron diseases.