Vision-aided brain-machine interface training system for robotic arm control and clinical application on two patients with cervical spinal cord injury

IMH-Net: a convolutional neural network for end-to-end EEG motor imagery classification

As the main component of Brain-computer interface (BCI) technology, the classification algorithm based on EEG has developed rapidly. The previous algorithms were often based on subject-dependent settings, resulting in BCI needing to be calibrated for new users. In this work, we propose IMH-Net, an end-to-end subject-independent model. The model first uses Inception blocks extracts the frequency domain features of the data, then further compresses the feature vectors to extract the spatial domain features, and finally learns the global information and classification through Multi-Head Attention mechanism. On the OpenBMI dataset, IMH-Net obtained 73.90 ± 13.10% accuracy and 73.09 ± 14.99% F1-score in subject-independent manner, which improved the accuracy by 1.96% compared with the comparison model. On the BCI competition IV dataset 2a, this model also achieved the highest accuracy and F1-score in subject-dependent manner. The IMH-Net model we proposed can improve the accuracy of subject-independent Motor Imagery (MI), and the robustness of the algorithm is high, which has strong practical value in the field of BCI.

BCI-controlled wheelchairs: end-users' perceptions, needs, and expectations, an interview-based study

PURPOSE

Brain-computer interface (BCI)-controlled wheelchairs have the potential to improve the independence of people with mobility impairments. The low uptake of BCI devices has been linked to a lack of knowledge among researchers of the needs of end-users that should influence BCI development.

MATERIALS AND METHODS

This study used semi-structured interviews to learn about the perceptions, needs, and expectations of spinal cord injury (SCI) patients with regards to a BCI-controlled wheelchair. Topics discussed in the interview include: paradigms, shared control, safety, robustness, channel selection, hardware, and experimental design. The interviews were recorded and then transcribed. Analysis was carried out using coding based on grounded theory principles.

RESULTS

The majority of participants had a positive view of BCI-controlled wheelchair technology and were willing to use the technology. Core issues were raised regarding safety, cost and aesthetics. Interview discussions were linked to state-of-the-art BCI technology. The results challenge the current reliance of researchers on the motor-imagery paradigm by suggesting end-users expect highly intuitive paradigms. There also needs to be a stronger focus on obstacle avoidance and safety features in BCI wheelchairs. Finally, the development of control approaches that can be personalized for individual users may be instrumental for widespread adoption of these devices.

CONCLUSIONS

This study, based on interviews with SCI patients, indicates that BCI-controlled wheelchairs are a promising assistive technology that would be well received by end-users. Recommendations for a more person-centered design of BCI controlled wheelchairs are made and clear avenues for future research are identified.

A Comprehensive Review of Endogenous EEG-Based BCIs for Dynamic Device Control

Electroencephalogram (EEG)-based brain–computer interfaces (BCIs) provide a novel approach for controlling external devices. BCI technologies can be important enabling technologies for people with severe mobility impairment. Endogenous paradigms, which depend on user-generated commands and do not need external stimuli, can provide intuitive control of external devices. This paper discusses BCIs to control various physical devices such as exoskeletons, wheelchairs, mobile robots, and robotic arms. These technologies must be able to navigate complex environments or execute fine motor movements. Brain control of these devices presents an intricate research problem that merges signal processing and classification techniques with control theory. In particular, obtaining strong classification performance for endogenous BCIs is challenging, and EEG decoder output signals can be unstable. These issues present myriad research questions that are discussed in this review paper. This review covers papers published until the end of 2021 that presented BCI-controlled dynamic devices. It discusses the devices controlled, EEG paradigms, shared control, stabilization of the EEG signal, traditional machine learning and deep learning techniques, and user experience. The paper concludes with a discussion of open questions and avenues for future work.

A brain-actuated robotic arm system using non-invasive hybrid brain-computer interface and shared control strategy

Objective.The electroencephalography (EEG)-based brain-computer interfaces (BCIs) have been used in the control of robotic arms. The performance of non-invasive BCIs may not be satisfactory due to the poor quality of EEG signals, so the shared control strategies were tried as an alternative solution. However, most of the existing shared control methods set the arbitration rules manually, which highly depended on the specific tasks and developer's experience. In this study, we proposed a novel shared control model that automatically optimized the control commands in a dynamical way based on the context in real-time control. Besides, we employed the hybrid BCI to better allocate commands with multiple functions. The system allowed non-invasive BCI users to manipulate a robotic arm moving in a three-dimensional (3D) space and complete a pick-place task of multiple objects.Approach.Taking the scene information obtained by computer vision as a knowledge base, a machine agent was designed to infer the user's intention and generate automatic commands. Based on the inference confidence and user's characteristic, the proposed shared control model fused the machine autonomy and human intention dynamically for robotic arm motion optimization during the online control. In addition, we introduced a hybrid BCI scheme that applied steady-state visual evoked potentials and motor imagery to the divided primary and secondary BCI interfaces to better allocate the BCI resources (e.g. decoding computing power, screen occupation) and realize the multi-dimensional control of the robotic arm.Main results.Eleven subjects participated in the online experiments of picking and placing five objects that scattered at different positions in a 3D workspace. The results showed that most of the subjects could control the robotic arm to complete accurate and robust picking task with an average success rate of approximately 85% under the shared control strategy, while the average success rate of placing task controlled by pure BCI was 50% approximately.Significance.In this paper, we proposed a novel shared controller for motion automatic optimization, together with a hybrid BCI control scheme that allocated paradigms according to the importance of commands to realize multi-dimensional and effective control of a robotic arm. Our study indicated that the shared control strategy with hybrid BCI could greatly improve the performance of the brain-actuated robotic arm system.

A Roadmap Towards Standards for Neurally Controlled End Effectors

The control and manipulation of various types of end effectors such as powered exoskeletons, prostheses, and ‘neural’ cursors by brain-machine interface (BMI) systems has been the target of many research projects. A seamless “plug and play” interface between any BMI and end effector is desired, wherein similar user's intent cause similar end effectors to behave identically. This report is based on the outcomes of an IEEE Standards Association Industry Connections working group on End Effectors for Brain-Machine Interfacing that convened to identify and address gaps in the existing standards for BMI-based solutions with a focus on the end-effector component. A roadmap towards standardization of end effectors for BMI systems is discussed by identifying current device standards that are applicable for end effectors. While current standards address basic electrical and mechanical safety, and to some extent, performance requirements, several gaps exist pertaining to unified terminologies, data communication protocols, patient safety and risk mitigation.