A Human-Humanoid Interaction Through the Use of BCI for Locked-In ALS Patients Using Neuro-Biological Feedback Fusion

Biosignal-integrated robotic systems with emerging trends in visual interfaces: A systematic review

Human-machine interfaces (HMI) are currently a trendy and rapidly expanding area of research. Interestingly, the human user does not readily observe the interface between humans and machines. Instead, interactions between the machine and electrical signals from the user's body are obscured by complex control algorithms. The result is effectively a one-way street, wherein data is only transmitted from human to machine. Thus, a gap remains in the literature: how can information be effectively conveyed to the user to enable mutual understanding between humans and machines? Here, this paper reviews recent advancements in biosignal-integrated wearable robotics, with a particular emphasis on "visualization"-the presentation of relevant data, statistics, and visual feedback to the user. This review article covers various signals of interest, such as electroencephalograms and electromyograms, and explores novel sensor architectures and key materials. Recent developments in wearable robotics are examined from control and mechanical design perspectives. Additionally, we discuss current visualization methods and outline the field's future direction. While much of the HMI field focuses on biomedical and healthcare applications, such as rehabilitation of spinal cord injury and stroke patients, this paper also covers less common applications in manufacturing, defense, and other domains.

Multilayer network-based channel selection for motor imagery brain-computer interface

Objective. The number of electrode channels in a motor imagery-based brain-computer interface (MI-BCI) system influences not only its decoding performance, but also its convenience for use in applications. Although many channel selection methods have been proposed in the literature, they are usually based on the univariate features of a single channel. This leads to a loss of the interaction between channels and the exchange of information between networks operating at different frequency bands.Approach. We integrate brain networks containing four frequency bands into a multilayer network framework and propose a multilayer network-based channel selection (MNCS) method for MI-BCI systems. A graph learning-based method is used to estimate the multilayer network from electroencephalogram (EEG) data that are filtered by multiple frequency bands. The multilayer participation coefficient of the multilayer network is then computed to select EEG channels that do not contain redundant information. Furthermore, the common spatial pattern (CSP) method is used to extract effective features. Finally, a support vector machine classifier with a linear kernel is trained to accurately identify MI tasks.Main results. We used three publicly available datasets from the BCI Competition containing data on 12 healthy subjects and one dataset containing data on 15 stroke patients to validate the effectiveness of our proposed method. The results showed that the proposed MNCS method outperforms all channels (85.8% vs. 93.1%, 84.4% vs. 89.0%, 71.7% vs. 79.4%, and 72.7% vs. 84.0%). Moreover, it achieved significantly higher decoding accuracies on MI-BCI systems than state-of-the-art methods (pairedt-tests,p< 0.05).Significance. The experimental results showed that the proposed MNCS method can select appropriate channels to improve the decoding performance as well as the convenience of the application of MI-BCI systems.

A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration

Coupling brain-computer interfaces (BCIs) and robotic systems in the future can enable seamless personal assistant systems in everyday life, with the requests that can be performed in a discrete manner, using one's brain activity only. These types of systems might be of a particular interest for people with locked-in syndrome (LIS) or amyotrophic lateral sclerosis (ALS) because they can benefit from communicating with robotic assistants using brain sensing interfaces. In this proof-of-concept work, we explored how a wireless and wearable BCI device can control a quadruped robot-Boston Dynamics' Spot. The device measures the user's electroencephalography (EEG) and electrooculography (EOG) activity of the user from the electrodes embedded in the glasses' frame. The user responds to a series of questions with YES/NO answers by performing a brain-teaser activity of mental calculus. Each question-answer pair has a pre-configured set of actions for Spot. For instance, Spot was prompted to walk across a room, pick up an object, and retrieve it for the user (i.e., bring a bottle of water) when a sequence resolved to a YES response. Our system achieved at a success rate of 83.4%. To the best of our knowledge, this is the first integration of wireless, non-visual-based BCI systems with Spot in the context of personal assistant use cases. While this BCI quadruped robot system is an early prototype, future iterations may embody friendly and intuitive cues similar to regular service dogs. As such, this project aims to pave a path towards future developments in modern day personal assistant robots powered by wireless and wearable BCI systems in everyday living conditions.

Digital Alternative Communication for Individuals with Amyotrophic Lateral Sclerosis: What We Have

Amyotrophic Lateral Sclerosis is a disease that compromises the motor system and the functional abilities of the person in an irreversible way, causing the progressive loss of the ability to communicate. Tools based on Augmentative and Alternative Communication are essential for promoting autonomy and improving communication, life quality, and survival. This Systematic Literature Review aimed to provide evidence on eye-image-based Human-Computer Interaction approaches for the Augmentative and Alternative Communication of people with Amyotrophic Lateral Sclerosis. The Systematic Literature Review was conducted and guided following a protocol consisting of search questions, inclusion and exclusion criteria, and quality assessment, to select primary studies published between 2010 and 2021 in six repositories: Science Direct, Web of Science, Springer, IEEE Xplore, ACM Digital Library, and PubMed. After the screening, 25 primary studies were evaluated. These studies showcased four low-cost, non-invasive Human-Computer Interaction strategies employed for Augmentative and Alternative Communication in people with Amyotrophic Lateral Sclerosis. The strategies included Eye-Gaze, which featured in 36% of the studies; Eye-Blink and Eye-Tracking, each accounting for 28% of the approaches; and the Hybrid strategy, employed in 8% of the studies. For these approaches, several computational techniques were identified. For a better understanding, a workflow containing the development phases and the respective methods used by each strategy was generated. The results indicate the possibility and feasibility of developing Human-Computer Interaction resources based on eye images for Augmentative and Alternative Communication in a control group. The absence of experimental testing in people with Amyotrophic Lateral Sclerosis reiterates the challenges related to the scalability, efficiency, and usability of these technologies for people with the disease. Although challenges still exist, the findings represent important advances in the fields of health sciences and technology, promoting a promising future with possibilities for better life quality.

Driving Mode Selection through SSVEP-Based BCI and Energy Consumption Analysis

Background: The brain–computer interface (BCI) is a highly cross-discipline technology and its successful application in various domains has received increasing attention. However, the BCI-enabled automobile industry is has been comparatively less investigated. In particular, there are currently no studies focusing on brain-controlled driving mode selection. Specifically, different driving modes indicate different driving styles which can be selected according to the road condition or the preference of individual drivers. Methods: In this paper, a steady-state visual-evoked potential (SSVEP)-based driving mode selection system is proposed. Upon this system, drivers can select the intended driving modes by only gazing at the corresponding SSVEP stimuli. A novel EEG processing algorithm named inter-trial distance minimization analysis (ITDMA) is proposed to enhance SSVEP detection. Both offline and real-time experiments were carried out to validate the effectiveness of the proposed system. Conclusion: The results show that a high selection accuracy of up to 92.3% can be realized, although this depends on the specific choice of flickering duration, the number of EEG channels, and the number of training signals. Additionally, energy consumption is investigated in terms of which the proposed brain-controlled system considerably differs from a traditional driving mode selection system, and the main reason is shown to be the existence of a detection error.

A novel brain-controlled wheelchair combined with computer vision and augmented reality

BACKGROUND

Brain-controlled wheelchairs (BCWs) are important applications of brain-computer interfaces (BCIs). Currently, most BCWs are semiautomatic. When users want to reach a target of interest in their immediate environment, this semiautomatic interaction strategy is slow.

METHODS

To this end, we combined computer vision (CV) and augmented reality (AR) with a BCW and proposed the CVAR-BCW: a BCW with a novel automatic interaction strategy. The proposed CVAR-BCW uses a translucent head-mounted display (HMD) as the user interface, uses CV to automatically detect environments, and shows the detected targets through AR technology. Once a user has chosen a target, the CVAR-BCW can automatically navigate to it. For a few scenarios, the semiautomatic strategy might be useful. We integrated a semiautomatic interaction framework into the CVAR-BCW. The user can switch between the automatic and semiautomatic strategies.

RESULTS

We recruited 20 non-disabled subjects for this study and used the accuracy, information transfer rate (ITR), and average time required for the CVAR-BCW to reach each designated target as performance metrics. The experimental results showed that our CVAR-BCW performed well in indoor environments: the average accuracies across all subjects were 83.6% (automatic) and 84.1% (semiautomatic), the average ITRs were 8.2 bits/min (automatic) and 8.3 bits/min (semiautomatic), the average times required to reach a target were 42.4 s (automatic) and 93.4 s (semiautomatic), and the average workloads and degrees of fatigue for the two strategies were both approximately 20.

CONCLUSIONS

Our CVAR-BCW provides a user-centric interaction approach and a good framework for integrating more advanced artificial intelligence technologies, which may be useful in the field of disability assistance.

A Regional Smoothing Block Sparse Bayesian Learning Method With Temporal Correlation for Channel Selection in P300 Speller

The P300-based brain-computer interfaces (BCIs) enable participants to communicate by decoding the electroencephalography (EEG) signal. Different regions of the brain correspond to various mental activities. Therefore, removing weak task-relevant and noisy channels through channel selection is necessary when decoding a specific type of activity from EEG. It can improve the recognition accuracy and reduce the training time of the subsequent models. This study proposes a novel block sparse Bayesian-based channel selection method for the P300 speller. In this method, we introduce block sparse Bayesian learning (BSBL) into the channel selection of P300 BCI for the first time and propose a regional smoothing BSBL (RSBSBL) by combining the spatial distribution properties of EEG. The RSBSBL can determine the number of channels adaptively. To ensure practicality, we design an automatic selection iteration strategy model to reduce the time cost caused by the inverse operation of the large-size matrix. We verified the proposed method on two public P300 datasets and on our collected datasets. The experimental results show that the proposed method can remove the inferior channels and work with the classifier to obtain high-classification accuracy. Hence, RSBSBL has tremendous potential for channel selection in P300 tasks.

A Human-Computer Control System Based on Intelligent Recognition of Eye Movements and Its Application in Wheelchair Driving

This paper presents a practical human-computer interaction system for wheelchair motion through eye tracking and eye blink detection. In this system, the pupil in the eye image has been extracted after binarization, and the center of the pupil was localized to capture the trajectory of eye movement and determine the direction of eye gaze. Meanwhile, convolutional neural networks for feature extraction and classification of open-eye and closed-eye images have been built, and machine learning was performed by extracting features from multiple individual images of open-eye and closed-eye states for input to the system. As an application of this human-computer interaction control system, experimental validation was carried out on a modified wheelchair and the proposed method proved to be effective and reliable based on the experimental results.

Biomedical signals and machine learning in amyotrophic lateral sclerosis: a systematic review

INTRODUCTION

The use of machine learning (ML) techniques in healthcare encompasses an emerging concept that envisages vast contributions to the tackling of rare diseases. In this scenario, amyotrophic lateral sclerosis (ALS) involves complexities that are yet not demystified. In ALS, the biomedical signals present themselves as potential biomarkers that, when used in tandem with smart algorithms, can be useful to applications within the context of the disease.

METHODS

This Systematic Literature Review (SLR) consists of searching for and investigating primary studies that use ML techniques and biomedical signals related to ALS. Following the definition and execution of the SLR protocol, 18 articles met the inclusion, exclusion, and quality assessment criteria, and answered the SLR research questions.

DISCUSSIONS

Based on the results, we identified three classes of ML applications combined with biomedical signals in the context of ALS: diagnosis (72.22%), communication (22.22%), and survival prediction (5.56%).

CONCLUSIONS

Distinct algorithmic models and biomedical signals have been reported and present promising approaches, regardless of their classes. In summary, this SLR provides an overview of the primary studies analyzed as well as directions for the construction and evolution of technology-based research within the scope of ALS.

Brain-Computer Interface-Based Humanoid Control: A Review

A Brain-Computer Interface (BCI) acts as a communication mechanism using brain signals to control external devices. The generation of such signals is sometimes independent of the nervous system, such as in Passive BCI. This is majorly beneficial for those who have severe motor disabilities. Traditional BCI systems have been dependent only on brain signals recorded using Electroencephalography (EEG) and have used a rule-based translation algorithm to generate control commands. However, the recent use of multi-sensor data fusion and machine learning-based translation algorithms has improved the accuracy of such systems. This paper discusses various BCI applications such as tele-presence, grasping of objects, navigation, etc. that use multi-sensor fusion and machine learning to control a humanoid robot to perform a desired task. The paper also includes a review of the methods and system design used in the discussed applications.

Acceptability Study of A3-K3 Robotic Architecture for a Neurorobotics Painting

In this paper, authors present a novel architecture for controlling an industrial robot via Brain Computer Interface. The robot used is a Series 2000 KR 210-2. The robotic arm was fitted with DI drawing devices that clamp, hold and manipulate various artistic media like brushes, pencils, pens. User selected a high-level task, for instance a shape or movement, using a human machine interface and the translation in robot movement was entirely demanded to the Robot Control Architecture defining a plan to accomplish user's task. The architecture was composed by a Human Machine Interface based on P300 Brain Computer Interface and a robotic architecture composed by a deliberative layer and a reactive layer to translate user's high-level command in a stream of movement for robots joints. To create a real-case scenario, the architecture was presented at Ars Electronica Festival, where the A3-K3 architecture has been used for painting. Visitors completed a survey to address 4 self-assessed different dimensions related to human-robot interaction: the technology knowledge, the personal attitude, the innovativeness and the satisfaction. The obtained results have led to further exploring the border of human-robot interaction, highlighting the possibilities of human expression in the interaction process with a machine to create art.

Neurophysiological predictors and spectro-spatial discriminative features for enhancing SMR-BCI

Neural engineering research is actively engaged in optimizing the robustness of sensorimotor rhythms (SMR)-brain-computer interface (BCI) to boost its potential real-world use.

OBJECTIVE

This paper investigates two vital factors in efficient and robust SMR-BCI design-algorithms that address subject-variability of optimal features and neurophysiological factors that correlate with BCI performance. Existing SMR-BCI research using electroencephalogram (EEG) to classify bilateral motor imagery (MI) focus on identifying subject-specific frequency bands with most discriminative motor patterns localized to sensorimotor region.

APPROACH

A novel strategy to further optimize BCI performance by taking into account the variability of discriminative spectral regions across various EEG channels is proposed in this paper.

MAIN RESULTS

The proposed technique results in a significant ([Formula: see text]) increase in average ([Formula: see text]) classification accuracy by [Formula: see text] accompanied by a considerable reduction in number of channels and bands. The session-to-session transfer variation in spectro-spatial patterns using proposed algorithm is investigated offline and classification performance of the optimized BCI model is successfully evaluated in an online SMR-BCI. Further, the effective prediction of SMR-BCI performance with physiological indicators derived from multi-channel resting-state EEG is demonstrated. The results indicate that the resting state activation patterns such as entropy and gamma power from pre-motor (fronto-central) and posterior (parietal and centro-parietal) areas, and beta power from posterior (centro-parietal) areas estimate BCI performance with minimum error. These patterns, strongly related to BCI performance, may represent certain cognitive states during rest.

SIGNIFICANCE

Findings reported in this paper imply the need for subject-specific modelling of BCI and the prediction of BCI performance using multi-channel rest-state parameters, to ensure enhanced BCI performance.