Application of BCI systems in neurorehabilitation: a scoping review

Functional near-infrared spectroscopy in non-invasive neuromodulation

ABSTRACT

Non-invasive cerebral neuromodulation technologies are essential for the reorganization of cerebral neural networks, which have been widely applied in the field of central neurological diseases, such as stroke, Parkinson's disease, and mental disorders. Although significant advances have been made in neuromodulation technologies, the identification of optimal neurostimulation parameters including the cortical target, duration, and inhibition or excitation pattern is still limited due to the lack of guidance for neural circuits. Moreover, the neural mechanism underlying neuromodulation for improved behavioral performance remains poorly understood. Recently, advancements in neuroimaging have provided insight into neuromodulation techniques. Functional near-infrared spectroscopy, as a novel non-invasive optical brain imaging method, can detect brain activity by measuring cerebral hemodynamics with the advantages of portability, high motion tolerance, and anti-electromagnetic interference. Coupling functional near-infrared spectroscopy with neuromodulation technologies offers an opportunity to monitor the cortical response, provide real-time feedback, and establish a closed-loop strategy integrating evaluation, feedback, and intervention for neurostimulation, which provides a theoretical basis for development of individualized precise neurorehabilitation. We aimed to summarize the advantages of functional near-infrared spectroscopy and provide an overview of the current research on functional near-infrared spectroscopy in transcranial magnetic stimulation, transcranial electrical stimulation, neurofeedback, and brain-computer interfaces. Furthermore, the future perspectives and directions for the application of functional near-infrared spectroscopy in neuromodulation are summarized. In conclusion, functional near-infrared spectroscopy combined with neuromodulation may promote the optimization of central neural reorganization to achieve better functional recovery from central nervous system diseases.

A comprehensive guide to BCI-based stroke neurorehabilitation interventions

Brain-Computer Interfaces (BCIs) offer the potential to facilitate neurorehabilitation in stroke patients by decoding user intentions from the central nervous system, thereby enabling control over external devices. Despite their promise, the diverse range of intervention parameters and technical challenges in clinical settings have hindered the accumulation of substantial evidence supporting the efficacy and effectiveness of BCIs in stroke rehabilitation. This article introduces a practical guide designed to navigate through these challenges in conducting BCI interventions for stroke rehabilitation. Applicable regardless of infrastructure and study design limitations, this guide acts as a comprehensive reference for executing BCI-based stroke interventions. Furthermore, it encapsulates insights gleaned from administering hundreds of BCI rehabilitation sessions to stroke patients.•Presents a comprehensive methodology for implementing BCI-based upper extremity therapy in stroke patients.•Provides detailed guidance on the number of sessions, trials, as well as the necessary hardware and software for effective intervention.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Investigating the impact of visual perspective in a motor imagery-based brain-robot interaction: A pilot study with healthy participants

Introduction

Motor Imagery (MI)-based Brain Computer Interfaces (BCI) have raised gained attention for their use in rehabilitation therapies since they allow controlling an external device by using brain activity, in this way promoting brain plasticity mechanisms that could lead to motor recovery. Specifically, rehabilitation robotics can provide precision and consistency for movement exercises, while embodied robotics could provide sensory feedback that can help patients improve their motor skills and coordination. However, it is still not clear whether different types of visual feedback may affect the elicited brain response and hence the effectiveness of MI-BCI for rehabilitation.

Methods

In this paper, we compare two visual feedback strategies based on controlling the movement of robotic arms through a MI-BCI system: 1) first-person perspective, with visual information that the user receives when they view the robot arms from their own perspective; and 2) third-person perspective, whereby the subjects observe the robot from an external perspective. We studied 10 healthy subjects over three consecutive sessions. The electroencephalographic (EEG) signals were recorded and evaluated in terms of the power of the sensorimotor rhythms, as well as their lateralization, and spatial distribution.

Results

Our results show that both feedback perspectives can elicit motor-related brain responses, but without any significant differences between them. Moreover, the evoked responses remained consistent across all sessions, showing no significant differences between the first and the last session.

Discussion

Overall, these results suggest that the type of perspective may not influence the brain responses during a MI- BCI task based on a robotic feedback, although, due to the limited sample size, more evidence is required. Finally, this study resulted into the production of 180 labeled MI EEG datasets, publicly available for research purposes.

Brain-machine interface based on transfer-learning for detecting the appearance of obstacles during exoskeleton-assisted walking

Introduction

Brain-machine interfaces (BMIs) attempt to establish communication between the user and the device to be controlled. BMIs have great challenges to face in order to design a robust control in the real field of application. The artifacts, high volume of training data, and non-stationarity of the signal of EEG-based interfaces are challenges that classical processing techniques do not solve, showing certain shortcomings in the real-time domain. Recent advances in deep-learning techniques open a window of opportunity to solve some of these problems. In this work, an interface able to detect the evoked potential that occurs when a person intends to stop due to the appearance of an unexpected obstacle has been developed.

Material and methods

First, the interface was tested on a treadmill with five subjects, in which the user stopped when an obstacle appeared (simulated by a laser). The analysis is based on two consecutive convolutional networks: the first one to discern the intention to stop against normal walking and the second one to correct false detections of the previous one.

Results and discussion

The results were superior when using the methodology of the two consecutive networks vs. only the first one in a cross-validation pseudo-online analysis. The false positives per min (FP/min) decreased from 31.8 to 3.9 FP/min and the number of repetitions in which there were no false positives and true positives (TP) improved from 34.9% to 60.3% NOFP/TP. This methodology was tested in a closed-loop experiment with an exoskeleton, in which the brain-machine interface (BMI) detected an obstacle and sent the command to the exoskeleton to stop. This methodology was tested with three healthy subjects, and the online results were 3.8 FP/min and 49.3% NOFP/TP. To make this model feasible for non-able bodied patients with a reduced and manageable time frame, transfer-learning techniques were applied and validated in the previous tests, and were then applied to patients. The results for two incomplete Spinal Cord Injury (iSCI) patients were 37.9% NOFP/TP and 7.7 FP/min.

Teleintervention for users of augmentative and alternative communication devices: A systematic review

AIM

To synthesize existing evidence on the effectiveness of speech-language teleinterventions delivered via videoconferencing to users of augmentative and alternative communication (AAC) devices.

METHOD

A systematic literature search was conducted in 10 electronic databases, from inception until August 2021. Included were speech-language teleinterventions delivered by researchers and/or clinicians via videoconferencing to users of AAC devices, without restrictions on chronological age and clinical diagnosis. The quality of the studies included in the review was appraised using the Downs and Black checklist and the Single-Case Experimental Design Scale; risk of bias was assessed using the Risk Of Bias In Non-Randomized Studies - of Interventions and the single-case design risk of bias tools.

RESULTS

Six teleinterventions including 25 participants with a variety of conditions, such as Down syndrome, autism, Rett syndrome, and amyotrophic lateral sclerosis met the inclusion criteria. Five studies used a single-case experimental design and one was a cohort study. Teleinterventions included active consultation (n = 2), functional communication training (n = 2), brain-computer interface (n = 1), and both teleintervention and in-person intervention (n = 1). All teleinterventions reported an increase in participants' independent use of AAC devices during the training sessions compared to baseline, as well as an overall high satisfaction and treatment acceptability.

INTERPRETATION

Speech-language teleinterventions for users of AAC devices show great potential for a successful method of service delivery. Future telehealth studies with larger sample sizes and more robust methodology are strongly encouraged to allow the generalization of results across different populations.

WHAT THIS PAPER ADDS

Individuals can learn to use augmentative and alternative communication (AAC) devices independently during tele-AAC interventions. Service providers and recipients reported an overall high satisfaction and acceptability for AAC services delivered via teleinterventions. Speech-language teleinterventions may be an effective method of providing AAC intervention services.

On the Influence of Aging on Classification Performance in the Visual EEG Oddball Paradigm Using Statistical and Temporal Features

The utilization of a non-invasive electroencephalogram (EEG) as an input sensor is a common approach in the field of the brain-computer interfaces (BCI). However, the collected EEG data pose many challenges, one of which may be the age-related variability of event-related potentials (ERPs), which are often used as primary EEG BCI signal features. To assess the potential effects of aging, a sample of 27 young and 43 older healthy individuals participated in a visual oddball study, in which they passively viewed frequent stimuli among randomly occurring rare stimuli while being recorded with a 32-channel EEG set. Two types of EEG datasets were created to train the classifiers, one consisting of amplitude and spectral features in time and another with extracted time-independent statistical ERP features. Among the nine classifiers tested, linear classifiers performed best. Furthermore, we show that classification performance differs between dataset types. When temporal features were used, maximum individuals' performance scores were higher, had lower variance, and were less affected overall by within-class differences such as age. Finally, we found that the effect of aging on classification performance depends on the classifier and its internal feature ranking. Accordingly, performance will differ if the model favors features with large within-class differences. With this in mind, care must be taken in feature extraction and selection to find the correct features and consequently avoid potential age-related performance degradation in practice.

JUNO Project: Deployment and Validation of a Low-Cost Cloud-Based Robotic Platform for Reliable Smart Navigation and Natural Interaction with Humans in an Elderly Institution

This paper describes the main results of the JUNO project, a proof of concept developed in the Region of Murcia in Spain, where a smart assistant robot with capabilities for smart navigation and natural human interaction has been developed and deployed, and it is being validated in an elderly institution with real elderly users. The robot is focused on helping people carry out cognitive stimulation exercises and other entertainment activities since it can detect and recognize people, safely navigate through the residence, and acquire information about attention while users are doing the mentioned exercises. All the information could be shared through the Cloud, if needed, and health professionals, caregivers and relatives could access such information by considering the highest standards of privacy required in these environments. Several tests have been performed to validate the system, which combines classic techniques and new Deep Learning-based methods to carry out the requested tasks, including semantic navigation, face detection and recognition, speech to text and text to speech translation, and natural language processing, working both in a local and Cloud-based environment, obtaining an economically affordable system. The paper also discusses the limitations of the platform and proposes several solutions to the detected drawbacks in this kind of complex environment, where the fragility of users should be also considered.

Beyond passive observation: feedback anticipation and observation activate the mirror system in virtual finger movement control via P300-BCI

Action observation (AO) is widely used as a post-stroke therapy to activate sensorimotor circuits through the mirror neuron system. However, passive observation is often considered to be less effective and less interactive than goal-directed movement observation, leading to the suggestion that observation of goal-directed actions may have stronger therapeutic potential, as goal-directed AO has been shown to activate mechanisms for monitoring action errors. Some studies have also suggested the use of AO as a form of Brain-computer interface (BCI) feedback. In this study, we investigated the potential for observation of virtual hand movements within a P300-based BCI as a feedback system to activate the mirror neuron system. We also explored the role of feedback anticipation and estimation mechanisms during movement observation. Twenty healthy subjects participated in the study. We analyzed event-related desynchronization and synchronization (ERD/S) of sensorimotor EEG rhythms and Error-related potentials (ErrPs) during observation of virtual hand finger flexion presented as feedback in the P300-BCI loop and compared the dynamics of ERD/S and ErrPs during observation of correct feedback and errors. We also analyzed these EEG markers during passive AO under two conditions: when subjects anticipated the action demonstration and when the action was unexpected. A pre-action mu-ERD was found both before passive AO and during action anticipation within the BCI loop. Furthermore, a significant increase in beta-ERS was found during AO within incorrect BCI feedback trials. We suggest that the BCI feedback may exaggerate the passive-AO effect, as it engages feedback anticipation and estimation mechanisms as well as movement error monitoring simultaneously. The results of this study provide insights into the potential of P300-BCI with AO-feedback as a tool for neurorehabilitation.

Personalized Brain-Computer Interface and Its Applications

Brain-computer interfaces (BCIs) are a new technology that subverts traditional human-computer interaction, where the control signal source comes directly from the user's brain. When a general BCI is used for practical applications, it is difficult for it to meet the needs of different individuals because of the differences among individual users in physiological and mental states, sensations, perceptions, imageries, cognitive thinking activities, and brain structures and functions. For this reason, it is necessary to customize personalized BCIs for specific users. So far, few studies have elaborated on the key scientific and technical issues involved in personalized BCIs. In this study, we will focus on personalized BCIs, give the definition of personalized BCIs, and detail their design, development, evaluation methods and applications. Finally, the challenges and future directions of personalized BCIs are discussed. It is expected that this study will provide some useful ideas for innovative studies and practical applications of personalized BCIs.

A Novel Hybrid Brain-Computer Interface Combining Motor Imagery and Intermodulation Steady-State Visual Evoked Potential

The hybrid brain-computer interface (hBCI) combining motor imagery (MI) and steady-state visual evoked potential (SSVEP) has been proven to have better performance than a pure MI- or SSVEP-based brain-computer interface (BCI). In most studies on hBCIs, subjects have been required to focus their attention on flickering light-emitting diodes (LEDs) or blocks while imagining body movements. However, these two classical tasks performed concurrently have a poor correlation. Therefore, it is necessary to reduce the task complexity of such a system and improve its user-friendliness. Aiming to achieve this goal, this study proposes a novel hybrid BCI that combines MI and intermodulation SSVEPs. In the proposed system, images of both hands flicker at the same frequency (i.e., 30 Hz) but at different grasp frequencies (i.e., 1 Hz for the left hand, and 1.5 Hz for the right hand), resulting in different intermodulation frequencies for encoding targets. Additionally, movement observation for subjects can help to perform the MI task better. In this study, two types of brain signals are classified independently and then fused by a scoring mechanism based on the probability distribution of relevant parameters. The online verification results showed that the average accuracies of 12 healthy subjects and 11 stroke patients were 92.40 ± 7.45% and 73.07 ± 9.07%, respectively. The average accuracies of 10 healthy subjects in the MI, SSVEP, and hybrid tasks were 84.00 ± 12.81%, 80.75 ± 8.08%, and 89.00 ± 9.94%, respectively. The high recognition accuracy verifies the feasibility and robustness of the proposed system. This study provides a novel and natural paradigm for a hybrid BCI based on MI and SSVEP.

Ten-Hour Stable Noninvasive Brain-Computer Interface Realized by Semidry Hydrogel-Based Electrodes

Noninvasive brain-computer interface (BCI) has been extensively studied from many aspects in the past decade. In order to broaden the practical applications of BCI technique, it is essential to develop electrodes for electroencephalogram (EEG) collection with advanced characteristics such as high conductivity, long-term effectiveness, and biocompatibility. In this study, we developed a silver-nanowire/PVA hydrogel/melamine sponge (AgPHMS) semidry EEG electrode for long-lasting monitoring of EEG signal. Benefiting from the water storage capacity of PVA hydrogel, the electrolyte solution can be continuously released to the scalp-electrode interface during used. The electrolyte solution can infiltrate the stratum corneum and reduce the scalp-electrode impedance to 10 kΩ-15 kΩ. The flexible structure enables the electrode with mechanical stability, increases the wearing comfort, and reduces the scalp-electrode gap to reduce contact impedance. As a result, a long-term BCI application based on measurements of motion-onset visual evoked potentials (mVEPs) shows that the 3-hour BCI accuracy of the new electrode (77% to 100%) is approximately the same as that of conventional electrodes supported by a conductive gel during the first hour. Furthermore, the BCI system based on the new electrode can retain low contact impedance for 10 hours on scalp, which greatly improved the ability of BCI technique.

Review of tDCS Configurations for Stimulation of the Lower-Limb Area of Motor Cortex and Cerebellum

This article presents an exhaustive analysis of the works present in the literature pertaining to transcranial direct current stimulation(tDCS) applications. The aim of this work is to analyze the specific characteristics of lower-limb stimulation, identifying the strengths and weaknesses of these works and framing them with the current knowledge of tDCS. The ultimate goal of this work is to propose areas of improvement to create more effective stimulation therapies with less variability.

4D attention-based neural network for EEG emotion recognition

Electroencephalograph (EEG) emotion recognition is a significant task in the brain-computer interface field. Although many deep learning methods are proposed recently, it is still challenging to make full use of the information contained in different domains of EEG signals. In this paper, we present a novel method, called four-dimensional attention-based neural network (4D-aNN) for EEG emotion recognition. First, raw EEG signals are transformed into 4D spatial-spectral-temporal representations. Then, the proposed 4D-aNN adopts spectral and spatial attention mechanisms to adaptively assign the weights of different brain regions and frequency bands, and a convolutional neural network (CNN) is utilized to deal with the spectral and spatial information of the 4D representations. Moreover, a temporal attention mechanism is integrated into a bidirectional Long Short-Term Memory (LSTM) to explore temporal dependencies of the 4D representations. Our model achieves state-of-the-art performances on both DEAP, SEED and SEED-IV datasets under intra-subject splitting. The experimental results have shown the effectiveness of the attention mechanisms in different domains for EEG emotion recognition.

Brain-Computer Interfaces and Creative Expression: Interface Considerations for Rehabilitative and Therapeutic Interactions

By translating brain signals into new kinds of outputs, Brain-Computer Interface (BCI) systems hold tremendous potential as both transformative rehabilitation and communication tools. BCIs can be considered a unique technology, in that they are able to provide a direct link between the brain and the external environment. By affording users with opportunities for communication and self-expression, BCI systems serve as a bridge between abled-bodied and disabled users, in turn reducing existing barriers between these groups. This perspective piece explores the complex shifting relationship between neuroadaptive systems and humans by foregrounding personal experience and embodied interaction as concepts through which to evaluate digital environments cultivated through the design of BCI interfaces. To underscore the importance of fostering human-centered experiences through technologically mediated interactions, this work offers a conceptual framework through which the rehabilitative and therapeutic possibilities of BCI user-system engagement could be furthered. By inviting somatic analysis towards the design of BCI interfaces and incorporating tenets of creative arts therapies practices into hybrid navigation paradigms for self-expressive applications, this work highlights the need for examining individual technological interactions as sites with meaning-making potentiality, as well as those conceived through unique exchanges based on user-specific needs for communication. Designing BCI interfaces in ways that afford users with increased options for navigation, as well as with the ability to share subjective and collective experiences, helps to redefine existing boundaries of digital and physical user-system interactions and encourages the reimagining of these systems as novel digital health tools for recovery.

Brain-Computer Interfaces for Children With Complex Communication Needs and Limited Mobility: A Systematic Review

Brain-computer interfaces (BCIs) represent a new frontier in the effort to maximize the ability of individuals with profound motor impairments to interact and communicate. While much literature points to BCIs' promise as an alternative access pathway, there have historically been few applications involving children and young adults with severe physical disabilities. As research is emerging in this sphere, this article aims to evaluate the current state of translating BCIs to the pediatric population. A systematic review was conducted using the Scopus, PubMed, and Ovid Medline databases. Studies of children and adolescents that reported BCI performance published in English in peer-reviewed journals between 2008 and May 2020 were included. Twelve publications were identified, providing strong evidence for continued research in pediatric BCIs. Research evidence was generally at multiple case study or exploratory study level, with modest sample sizes. Seven studies focused on BCIs for communication and five on mobility. Articles were categorized and grouped based on type of measurement (i.e., non-invasive and invasive), and the type of brain signal (i.e., sensory evoked potentials or movement-related potentials). Strengths and limitations of studies were identified and used to provide requirements for clinical translation of pediatric BCIs. This systematic review presents the state-of-the-art of pediatric BCIs focused on developing advanced technology to support children and youth with communication disabilities or limited manual ability. Despite a few research studies addressing the application of BCIs for communication and mobility in children, results are encouraging and future works should focus on customizable pediatric access technologies based on brain activity.

Inter-stimulus phase coherence in steady-state somatosensory evoked potentials and its application in improving the performance of single-channel MI-BCI

Objective. With the development of clinical applications of motor imagery-based brain-computer interfaces (MI-BCIs), a single-channel MI-BCI system that can be easily assembled is an attractive goal. However, due to the low quality of the spectral power features in the traditional MI-BCI paradigm, the recognition performance of current single-channel systems is far lower than that of multi-channel systems, impeding their use in clinical applications.Approach.In this study, the subjects' right and left hands were stimulated simultaneously at different frequencies to induce steady-state somatosensory evoked potentials (SSSEP). Subjects then performed motor imagery (MI) tasks. A new electroencephalography (EEG) index, inter-stimulus phase coherence (ISPC), was built to measure phase desynchronization of SSSEP caused by MI. Then, ISPC is introduced as a feature into left-hand and right-hand MI recognition.Main results.ISPC analysis found that left-handed MI can cause a significant decrease in phase synchronization in contralateral sensorimotor SSSEP, while right-handed MI has little effect on it, and vice versa. Combining ISPC features with traditional spectral power features, the single-channel left-hand versus right-hand MI recognition accuracy reaches 81.0%, which is much higher than that observed with traditional MI paradigms (about 60%).Significance.This work shows that the hybrid MI-SSSEP paradigm can provide more sensitive EEG features to decode motor intentions, demonstrating its potential for clinical applications.

Upper Limb Home-Based Robotic Rehabilitation During COVID-19 Outbreak

The coronavirus disease (COVID-19) outbreak requires rapid reshaping of rehabilitation services to include patients recovering from severe COVID-19 with post-intensive care syndromes, which results in physical deconditioning and cognitive impairments, patients with comorbid conditions, and other patients requiring physical therapy during the outbreak with no or limited access to hospital and rehabilitation centers. Considering the access barriers to quality rehabilitation settings and services imposed by social distancing and stay-at-home orders, these patients can be benefited from providing access to affordable and good quality care through home-based rehabilitation. The success of such treatment will depend highly on the intensity of the therapy and effort invested by the patient. Monitoring patients' compliance and designing a home-based rehabilitation that can mentally engage them are the critical elements in home-based therapy's success. Hence, we study the state-of-the-art telerehabilitation frameworks and robotic devices, and comment about a hybrid model that can use existing telerehabilitation framework and home-based robotic devices for treatment and simultaneously assess patient's progress remotely. Second, we comment on the patients' social support and engagement, which is critical for the success of telerehabilitation service. As the therapists are not physically present to guide the patients, we also discuss the adaptability requirement of home-based telerehabilitation. Finally, we suggest that the reformed rehabilitation services should consider both home-based solutions for enhancing the activities of daily living and an on-demand ambulatory rehabilitation unit for extensive training where we can monitor both cognitive and motor performance of the patients remotely.

Abdul Majeed A. The classification of motor imagery response: an accuracy enhancement through the ensemble of random subspace k-NN

Brain-computer interface (BCI) is a viable alternative communication strategy for patients of neurological disorders as it facilitates the translation of human intent into device commands. The performance of BCIs primarily depends on the efficacy of the feature extraction and feature selection techniques, as well as the classification algorithms employed. More often than not, high dimensional feature set contains redundant features that may degrade a given classifier's performance. In the present investigation, an ensemble learning-based classification algorithm, namely random subspace k-nearest neighbour (k-NN) has been proposed to classify the motor imagery (MI) data. The common spatial pattern (CSP) has been applied to extract the features from the MI response, and the effectiveness of random forest (RF)-based feature selection algorithm has also been investigated. In order to evaluate the efficacy of the proposed method, an experimental study has been implemented using four publicly available MI dataset (BCI Competition III dataset 1 (data-1), dataset IIIA (data-2), dataset IVA (data-3) and BCI Competition IV dataset II (data-4)). It was shown that the ensemble-based random subspace k-NN approach achieved the superior classification accuracy (CA) of 99.21%, 93.19%, 93.57% and 90.32% for data-1, data-2, data-3 and data-4, respectively against other models evaluated, namely linear discriminant analysis, support vector machine, random forest, Naïve Bayes and the conventional k-NN. In comparison with other classification approaches reported in the recent studies, the proposed method enhanced the accuracy by 2.09% for data-1, 1.29% for data-2, 4.95% for data-3 and 5.71% for data-4, respectively. Moreover, it is worth highlighting that the RF feature selection technique employed in the present study was able to significantly reduce the feature dimension without compromising the overall CA. The outcome from the present study implies that the proposed method may significantly enhance the accuracy of MI data classification.

An Introductory Tutorial on Brain–Computer Interfaces and Their Applications

The prospect and potentiality of interfacing minds with machines has long captured human imagination. Recent advances in biomedical engineering, computer science, and neuroscience are making brain–computer interfaces a reality, paving the way to restoring and potentially augmenting human physical and mental capabilities. Applications of brain–computer interfaces are being explored in applications as diverse as security, lie detection, alertness monitoring, gaming, education, art, and human cognition augmentation. The present tutorial aims to survey the principal features and challenges of brain–computer interfaces (such as reliable acquisition of brain signals, filtering and processing of the acquired brainwaves, ethical and legal issues related to brain–computer interface (BCI), data privacy, and performance assessment) with special emphasis to biomedical engineering and automation engineering applications. The content of this paper is aimed at students, researchers, and practitioners to glimpse the multifaceted world of brain–computer interfacing.

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain-computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go.

Summary of over Fifty Years with Brain-Computer Interfaces-A Review

Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.

EEG-based emotion recognition using 4D convolutional recurrent neural network

In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition.

Evaluation of mental workload during automobile driving using one-class support vector machine with eye movement data

The aim of this study is to investigate the usefulness of the anomaly detection method by one-class support vector machine (OCSVM) for the evaluation of mental workload (MWL) during automobile driving. Twelve students (six males and six females) participated. The participants performed driving tasks with a driving simulator (DS) and the N-back task that was used to control their MWL. The N-back task had five difficulty levels from "none" to "3-back." Eye and head movements were measured during the DS driving. Results showed that the standard deviation (SD) of the gaze angle, SD of eyeball rotation angle, share rate of head movement, and blink frequency had significant correlations with the task difficulty. The decision boundary of OCSVM could detect 95% of high MWL state (i.e., "3-back" state). In addition, the absolute value of the distance from the decision boundary increased with the task difficulty from "0-back" to "3-back."

The effects of explosion sound on the brain based on electroencephalogram signals

In this paper, the brain reactions to explosion sound are investigated. Electroencephalogram (EEG) signals of 17 people were recorded. Subjects were selected from three groups: staff who did not face explosion before, blasting employees of surface, and underground mining workers. Routine EEG signals, also called explosion sounds, were recorded. Explosion sound was broadcasted without any previous alarm. Then it was repeated with their pre-awareness. Gradient and time duration of Delta band of EEG signals were extracted as features. Results showed that for blasting employees, especially underground ones, an increase of mean amplitude of delta band power of EEG signals of motor, speech, auditory and visual sensations were occurred, while in the case of staff it was decreased. This shows consciousness arising of blasting employees with hearing explosion sound. The reaction of somatosensory sense was dropped for all three groups. In general, reaction time for blasting employees has been longer than staff.

BCIAUT-P300: A Multi-Session and Multi-Subject Benchmark Dataset on Autism for P300-Based Brain-Computer-Interfaces

There is a lack of multi-session P300 datasets for Brain-Computer Interfaces (BCI). Publicly available datasets are usually limited by small number of participants with few BCI sessions. In this sense, the lack of large, comprehensive datasets with various individuals and multiple sessions has limited advances in the development of more effective data processing and analysis methods for BCI systems. This is particularly evident to explore the feasibility of deep learning methods that require large datasets. Here we present the BCIAUT-P300 dataset, containing 15 autism spectrum disorder individuals undergoing 7 sessions of P300-based BCI joint-attention training, for a total of 105 sessions. The dataset was used for the 2019 IFMBE Scientific Challenge organized during MEDICON 2019 where, in two phases, teams from all over the world tried to achieve the best possible object-detection accuracy based on the P300 signals. This paper presents the characteristics of the dataset and the approaches followed by the 9 finalist teams during the competition. The winner obtained an average accuracy of 92.3% with a convolutional neural network based on EEGNet. The dataset is now publicly released and stands as a benchmark for future P300-based BCI algorithms based on multiple session data.

The combination of brain-computer interfaces and artificial intelligence: applications and challenges

Brain-computer interfaces (BCIs) have shown great prospects as real-time bidirectional links between living brains and actuators. Artificial intelligence (AI), which can advance the analysis and decoding of neural activity, has turbocharged the field of BCIs. Over the past decade, a wide range of BCI applications with AI assistance have emerged. These "smart" BCIs including motor and sensory BCIs have shown notable clinical success, improved the quality of paralyzed patients' lives, expanded the athletic ability of common people and accelerated the evolution of robots and neurophysiological discoveries. However, despite technological improvements, challenges remain with regard to the long training periods, real-time feedback, and monitoring of BCIs. In this article, the authors review the current state of AI as applied to BCIs and describe advances in BCI applications, their challenges and where they could be headed in the future.

Brain Functional Networks Study of Subacute Stroke Patients With Upper Limb Dysfunction After Comprehensive Rehabilitation Including BCI Training

Brain computer interface (BCI)-based training is promising for the treatment of stroke patients with upper limb (UL) paralysis. However, most stroke patients receive comprehensive treatment that not only includes BCI, but also routine training. The purpose of this study was to investigate the topological alterations in brain functional networks following comprehensive treatment, including BCI training, in the subacute stage of stroke. Twenty-five hospitalized subacute stroke patients with moderate to severe UL paralysis were assigned to one of two groups: 4-week comprehensive treatment, including routine and BCI training (BCI group, BG, n = 14) and 4-week routine training without BCI support (control group, CG, n = 11). Functional UL assessments were performed before and after training, including, Fugl-Meyer Assessment-UL (FMA-UL), Action Research Arm Test (ARAT), and Wolf Motor Function Test (WMFT). Neuroimaging assessment of functional connectivity (FC) in the BG was performed by resting state functional magnetic resonance imaging. After training, as compared with baseline, all clinical assessments (FMA-UL, ARAT, and WMFT) improved significantly (p < 0.05) in both groups. Meanwhile, better functional improvements were observed in FMA-UL (p < 0.05), ARAT (p < 0.05), and WMFT (p < 0.05) in the BG. Meanwhile, FC of the BG increased across the whole brain, including the temporal, parietal, and occipital lobes and subcortical regions. More importantly, increased inter-hemispheric FC between the somatosensory association cortex and putamen was strongly positively associated with UL motor function after training. Our findings demonstrate that comprehensive rehabilitation, including BCI training, can enhance UL motor function better than routine training for subacute stroke patients. The reorganization of brain functional networks topology in subacute stroke patients allows for increased coordination between the multi-sensory and motor-related cortex and the extrapyramidal system. Future long-term, longitudinal, controlled neuroimaging studies are needed to assess the effectiveness of BCI training as an approach to promote brain plasticity during the subacute stage of stroke.

Brain-computer interfaces and virtual reality for neurorehabilitation

Brain-computer interfaces (BCIs) and virtual reality (VR) are two technologic advances that are changing our way of interacting with the world. BCIs can be used to influence and can serve as a control mechanism in navigation tasks, communication, or other assistive functions. VR can create ad hoc interactive scenarios that involve all our senses, stimulate the brain in a multisensory fashion, and increase the motivation and fun with game-like environments. VR and motion tracking enable natural human-computer interaction at cognitive and physical levels. This includes both brain and body in the design of meaningful VR experiences; these cases in which participants feel naturally present could help augment the benefits of BCIs for assistive and neurorehabilitation applications for the relearning of motor and cognitive skills. VR technology is now available at the consumer level thanks to the proliferation of affordable head-mounted displays (HMDs). Merging both technologies into simplified, practical devices may help democratize these technologies.

Neuromusculoskeletal Modeling-Based Prostheses for Recovery After Spinal Cord Injury

Concurrent stimulation and reinforcement of motor and sensory pathways has been proposed as an effective approach to restoring function after developmental or acquired neurotrauma. This can be achieved by applying multimodal rehabilitation regimens, such as thought-controlled exoskeletons or epidural electrical stimulation to recover motor pattern generation in individuals with spinal cord injury (SCI). However, the human neuromusculoskeletal (NMS) system has often been oversimplified in designing rehabilitative and assistive devices. As a result, the neuromechanics of the muscles is seldom considered when modeling the relationship between electrical stimulation, mechanical assistance from exoskeletons, and final joint movement. A powerful way to enhance current neurorehabilitation is to develop the next generation prostheses incorporating personalized NMS models of patients. This strategy will enable an individual voluntary interfacing with multiple electromechanical rehabilitation devices targeting key afferent and efferent systems for functional improvement. This narrative review discusses how real-time NMS models can be integrated with finite element (FE) of musculoskeletal tissues and interface multiple assistive and robotic devices with individuals with SCI to promote neural restoration. In particular, the utility of NMS models for optimizing muscle stimulation patterns, tracking functional improvement, monitoring safety, and providing augmented feedback during exercise-based rehabilitation are discussed.

A Novel Modular Headmount Design for non-invasive Scalp EEG Recordings in Awake Animal Models

We have designed and developed a novel, noninvasive modular headmount to be used for awake animal scalp electroencephalography (EEG). The design is based on a developing rat that will accommodate rapid head growth. Desired characteristics include non-invasiveness, adjustable quantity and positioning, light weight, and tolerability by the animal. Axial Dependent Modular Electrode Mount (ADMEM), as designed here, addresses the aforementioned constraints by using light-weight and adjustable materials. The initial prototype of ADMEM has been tested in vivo with rat pups, using the open field test to assess for stress and anxiety at two post-installation time-points: one day after ADMEM installation (acute time-point) and four days after ADMEM installation (sub-acute time-point). There was no significant difference in normal developmental weight gain between Control and ADMEM rat groups. Although no significant difference was found in the level of anxiety between groups at the acute time-point, the ADMEM group spent significantly less time in the center of the open field test, suggesting higher anxiety. The test also showed no difference in the measured traveled distances between Control and ADMEM groups on either time-points.

Multipurpose Virtual Reality Environment for Biomedical and Health Applications

Virtual reality is a trending, widely accessible, and contemporary technology of increasing utility to biomedical and health applications. However, most implementations of virtual reality environments are tailored to specific applications. We describe the complete development of a novel, open-source virtual reality environment that is suitable for multipurpose biomedical and healthcare applications. This environment can be interfaced with different hardware and data sources, ranging from gyroscopes to fMRI scanners. The developed environment simulates an immersive (first-person perspective) run in the countryside, in a virtual landscape with various salient features. The utility of the developed VR environment has been validated via two test applications: an application in the context of motor rehabilitation following injury of the lower limbs and an application in the context of real-time functional magnetic resonance imaging neurofeedback, to regulate brain function in specific brain regions of interest. Both applications were tested by pilot subjects that unanimously provided very positive feedback, suggesting that appropriately designed VR environments can indeed be robustly and efficiently used for multiple biomedical purposes. We attribute the versatility of our approach on three principles implicit in the design: selectivity, immersiveness, and adaptability. The software, including both applications, is publicly available free of charge, via a GitHub repository, in support of the Open Science Initiative. Although using this software requires specialized hardware and engineering know-how, we anticipate our contribution to catalyze further progress, interdisciplinary collaborations and replicability, with regards to the usage of virtual reality in biomedical and health applications.

Robotics in Health Care: Perspectives of Robot-Aided Interventions in Clinical Practice for Rehabilitation of Upper Limbs

Robot-aided systems to support the physical rehabilitation of individuals with neurological impairment is one of the fields that has been widely developed in the last few decades. However, the adoption of these systems in clinical practice remains limited. In order to better understanding the causes of this limitation, a systematic review of robot-based systems focused on upper extremity rehabilitation is presented in this paper. A systematic search and review of related articles in the literature were conducted. The chosen works were analyzed according to the type of device, the data analysis capability, the therapy method, the human–robot interaction, the safety strategies, and the focus of treatment. As a conclusion, self-adaptation for personalizing the treatments, safeguarding and enhancing of patient–robot interaction towards training essential factors of movement generation into the same paradigm, or the use of lifelike environments in fully-immersive virtual reality for increasing the assimilation of motor gains could be relevant factors to develop more accepted robot-aided systems in clinical practice.

Assistive technology for communication of older adults: a systematic review

OBJECTIVE

Describe the use of assistive technology to enhance communication opportunities for older adults.

METHODS

A systematic review was conducted in two databases, PubMed and Web of Science, by using two different searches in each. The search was limited to original articles, in English language, including people aged 60 years and older that used any type of assistive technology for communication. The articles found in the initial search were filtered by title, abstracts and the remaining articles were fully read.

RESULTS

Eighteen studies were included in this review after the reading of full-texts. Most of the studies included apparently healthy participants with communication limitations due to aging related changes and the others included people with some pathology that prevent them from normal communication.

CONCLUSION

Four categories of assistive technology were identified: assistive technology for people with speech problems; robot or videoconferencing systems; Information and Communication Technologies and, other types of assistive technology for communication, such as hearing aids and scrapbooks. Assistive technology for communication of older adults is not only used by people with disabilities that prevent them from usual communication. They are mostly for older adults without a pathological communication problem.

Network Brain-Computer Interface (nBCI): An Alternative Approach for Cognitive Prosthetics

Brain computer interfaces (BCIs) have been applied to sensorimotor systems for many years. However, BCI technology has broad potential beyond sensorimotor systems. The emerging field of cognitive prosthetics, for example, promises to improve learning and memory for patients with cognitive impairment. Unfortunately, our understanding of the neural mechanisms underlying these cognitive processes remains limited in part due to the extensive individual variability in neural coding and circuit function. As a consequence, the development of methods to ascertain optimal control signals for cognitive decoding and restoration remains an active area of inquiry. To advance the field, robust tools are required to quantify time-varying and task-dependent brain states predictive of cognitive performance. Here, we suggest that network science is a natural language in which to formulate and apply such tools. In support of our argument, we offer a simple demonstration of the feasibility of a network approach to BCI control signals, which we refer to as network BCI (nBCI). Finally, in a single subject example, we show that nBCI can reliably predict online cognitive performance and is superior to certain common spectral approaches currently used in BCIs. Our review of the literature and preliminary findings support the notion that nBCI could provide a powerful approach for future applications in cognitive prosthetics.

Machine Learning Classification to Identify the Stage of Brain-Computer Interface Therapy for Stroke Rehabilitation Using Functional Connectivity

Interventional therapy using brain-computer interface (BCI) technology has shown promise in facilitating motor recovery in stroke survivors; however, the impact of this form of intervention on functional networks outside of the motor network specifically is not well-understood. Here, we investigated resting-state functional connectivity (rs-FC) in stroke participants undergoing BCI therapy across stages, namely pre- and post-intervention, to identify discriminative functional changes using a machine learning classifier with the goal of categorizing participants into one of the two therapy stages. Twenty chronic stroke participants with persistent upper-extremity motor impairment received neuromodulatory training using a closed-loop neurofeedback BCI device, and rs-functional MRI (rs-fMRI) scans were collected at four time points: pre-, mid-, post-, and 1 month post-therapy. To evaluate the peak effects of this intervention, rs-FC was analyzed from two specific stages, namely pre- and post-therapy. In total, 236 seeds spanning both motor and non-motor regions of the brain were computed at each stage. A univariate feature selection was applied to reduce the number of features followed by a principal component-based data transformation used by a linear binary support vector machine (SVM) classifier to classify each participant into a therapy stage. The SVM classifier achieved a cross-validation accuracy of 92.5% using a leave-one-out method. Outside of the motor network, seeds from the fronto-parietal task control, default mode, subcortical, and visual networks emerged as important contributors to the classification. Furthermore, a higher number of functional changes were observed to be strengthening from the pre- to post-therapy stage than the ones weakening, both of which involved motor and non-motor regions of the brain. These findings may provide new evidence to support the potential clinical utility of BCI therapy as a form of stroke rehabilitation that not only benefits motor recovery but also facilitates recovery in other brain networks. Moreover, delineation of stronger and weaker changes may inform more optimal designs of BCI interventional therapy so as to facilitate strengthened and suppress weakened changes in the recovery process.

"Enheduanna-A Manifesto of Falling" Live Brain-Computer Cinema Performance: Performer and Audience Participation, Cognition and Emotional Engagement Using Multi-Brain BCI Interaction

The fields of neural prosthetic technologies and Brain-Computer Interfaces (BCIs) have witnessed in the past 15 years an unprecedented development, bringing together theories and methods from different scientific fields, digital media, and the arts. More in particular, artists have been amongst the pioneers of the design of relevant applications since their emergence in the 1960s, pushing the boundaries of applications in real-life contexts. With the new research, advancements, and since 2007, the new low-cost commercial-grade wireless devices, there is a new increasing number of computer games, interactive installations, and performances that involve the use of these interfaces, combining scientific, and creative methodologies. The vast majority of these works use the brain-activity of a single participant. However, earlier, as well as recent examples, involve the simultaneous interaction of more than one participants or performers with the use of Electroencephalography (EEG)-based multi-brain BCIs. In this frame, we discuss and evaluate “Enheduanna—A Manifesto of Falling,” a live brain-computer cinema performance that enables for the first time the simultaneous real-time multi-brain interaction of more than two participants, including a performer and members of the audience, using a passive EEG-based BCI system in the context of a mixed-media performance. The performance was realised as a neuroscientific study conducted in a real-life setting. The raw EEG data of seven participants, one performer and two different members of the audience for each performance, were simultaneously recorded during three live events. The results reveal that the majority of the participants were able to successfully identify whether their brain-activity was interacting with the live video projections or not. A correlation has been found between their answers to the questionnaires, the elements of the performance that they identified as most special, and the audience's indicators of attention and emotional engagement. Also, the results obtained from the performer's data analysis are consistent with the recall of working memory representations and the increase of cognitive load. Thus, these results prove the efficiency of the interaction design, as well as the importance of the directing strategy, dramaturgy and narrative structure on the audience's perception, cognitive state, and engagement.

A qualitative study adopting a user-centered approach to design and validate a brain computer interface for cognitive rehabilitation for people with brain injury

Cognitive rehabilitation is established as a core intervention within rehabilitation programs following a traumatic brain injury (TBI). Digitally enabled assistive technologies offer opportunities for clinicians to increase remote access to rehabilitation supporting transition into home. Brain Computer Interface (BCI) systems can harness the residual abilities of individuals with limited function to gain control over computers through their brain waves. This paper presents an online cognitive rehabilitation application developed with therapists, to work remotely with people who have TBI, who will use BCI at home to engage in the therapy. A qualitative research study was completed with people who are community dwellers post brain injury (end users), and a cohort of therapists involved in cognitive rehabilitation. A user-centered approach over three phases in the development, design and feasibility testing of this cognitive rehabilitation application included two tasks (Find-a-Category and a Memory Card task). The therapist could remotely prescribe activity with different levels of difficulty. The service user had a home interface which would present the therapy activities. This novel work was achieved by an international consortium of academics, business partners and service users.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

Testing a novel method for improving wayfinding by means of a P3b Virtual Reality Visual Paradigm in normal aging

BACKGROUND

We propose a virtual reality (VR) model, reproducing a house environment, where color modification of target places, obtainable by home automation in a real ambient, was tested by means of a P3b paradigm. The target place (bathroom door) was designed to be recognized during a virtual wayfinding in a realistic reproduction of a house environment. Different color and luminous conditions, easily obtained in the real ambient from a remote home automation control, were applied to the target and standard places, all the doors being illuminated in white (W), and only target doors colored with a green (G) or red (R) spotlight. Three different Virtual Environments (VE) were depicted, as the bathroom was designed in the aisle (A), living room (L) and bedroom (B). EEG was recorded from 57 scalp electrodes in 10 healthy subjects in the 60-80 year age range (O-old group) and 12 normal cases in the 20-30 year age range (Y-young group).

RESULTS

In Young group, all the target stimuli determined a significant increase in P3b amplitude on the parietal, occipital and central electrodes compared to frequent stimuli condition, whatever was the color of the target door, while in elderly group the P3b obtained by the green and red colors was significantly different from the frequent stimulus, on the parietal, occipital, and central derivations, while the White stimulus did not evoke a significantly larger P3b with respect to frequent stimulus.

DISCUSSION

The modulation of P3b amplitude, obtained by color and luminance change of target place, suggests that cortical resources, able to compensate the age-related progressive loss of cognitive performance, need to be facilitated even in normal elderly. The event-related responses obtained by virtual reality may be a reliable method to test the environmental feasibility to age-related cognitive changes.