Heading for new shores! Overcoming pitfalls in BCI design

Considerations and discussions on the clear definition and definite scope of brain-computer interfaces

Brain-computer interface (BCI) is a revolutionizing human-computer interaction with potential applications in both medical and non-medical fields, emerging as a cutting-edge and trending research direction. Increasing numbers of groups are engaging in BCI research and development. However, in recent years, there has been some confusion regarding BCI, including misleading and hyped propaganda about BCI, and even non-BCI technologies being labeled as BCI. Therefore, a clear definition and a definite scope for BCI are thoroughly considered and discussed in the paper, based on the existing definitions of BCI, including the six key or essential components of BCI. In the review, different from previous definitions of BCI, BCI paradigms and neural coding are explicitly included in the clear definition of BCI provided, and the BCI user (the brain) is clearly identified as a key component of the BCI system. Different people may have different viewpoints on the definition and scope of BCI, as well as some related issues, which are discussed in the article. This review argues that a clear definition and definite scope of BCI will benefit future research and commercial applications. It is hoped that this review will reduce some of the confusion surrounding BCI and promote sustainable development in this field.

Brain-computer interface for augmentative and alternative communication access: The initial training needs and learning preferences of speech-language pathologists

PURPOSE

To enable the codesign of a training framework for brain-computer interfaces for augmentative and alternative communications access (BCI-AAC), the aim of this study is to evaluate the initial BCI-AAC training needs and preferred learning strategies of speech-language pathologists (SLPs) with AAC experience.

METHOD

Eleven SLPs employed across a broad range of settings completed a semi-structured interview. A grounded theory approach alongside peer debriefing and review, member checking, and triangulation procedures were utilised for thematic analysis to help ensure data reliability and credibility.

RESULT

Regarding critical training needs, SLPs identified the subthemes of (a) personalisation of intervention, (b) technical aspects, (c) BCI-AAC system types and access, and (d) how to support stakeholders in BCI-AAC implementation. Regarding learning strategy preferences, participants discussed (a) expert guidance and demonstrations, (b) hands-on experience, alongside (c) media and presentations.

CONCLUSION

Findings present a continuum of critical training needs ranging from more foundational information to more personalised assessment and intervention consideration. These thematic results present a first step in developing a basic framework for SLP training in BCI-AAC to utilise and build from as technology development continues, and provides an important initial starting point for the codesign of clinically focused BCI-AAC trainings.

Transfer learning promotes acquisition of individual BCI skills

Subject training is crucial for acquiring brain-computer interface (BCI) control. Typically, this requires collecting user-specific calibration data due to high inter-subject neural variability that limits the usability of generic decoders. However, calibration is cumbersome and may produce inadequate data for building decoders, especially with naïve subjects. Here, we show that a decoder trained on the data of a single expert is readily transferrable to inexperienced users via domain adaptation techniques allowing calibration-free BCI training. We introduce two real-time frameworks, (i) Generic Recentering (GR) through unsupervised adaptation and (ii) Personally Assisted Recentering (PAR) that extends GR by employing supervised recalibration of the decoder parameters. We evaluated our frameworks on 18 healthy naïve subjects over five online sessions, who operated a customary synchronous bar task with continuous feedback and a more challenging car racing game with asynchronous control and discrete feedback. We show that along with improved task-oriented BCI performance in both tasks, our frameworks promoted subjects' ability to acquire individual BCI skills, as the initial neurophysiological control features of an expert subject evolved and became subject specific. Furthermore, those features were task-specific and were learned in parallel as participants practiced the two tasks in every session. Contrary to previous findings implying that supervised methods lead to improved online BCI control, we observed that longitudinal training coupled with unsupervised domain matching (GR) achieved similar performance to supervised recalibration (PAR). Therefore, our presented frameworks facilitate calibration-free BCIs and have immediate implications for broader populations-such as patients with neurological pathologies-who might struggle to provide suitable initial calibration data.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.

Multimodal pre-screening can predict BCI performance variability: A novel subject-specific experimental scheme

BACKGROUND

Brain-computer interface (BCI) systems currently lack the required robustness for long-term daily use due to inter- and intra-subject performance variability. In this study, we propose a novel personalized scheme for a multimodal BCI system, primarily using functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), to identify, predict, and compensate for factors affecting competence-related and interfering factors associated with performance.

METHOD

11 (out of 13 recruited) participants, including five participants with motor deficits, completed four sessions on average. During the training sessions, the subjects performed a short pre-screening phase, followed by three variations of a novel visou-mental (VM) protocol. Features extracted from the pre-screening phase were used to construct predictive platforms using stepwise multivariate linear regression (MLR) models. In the test sessions, we employed a task-correction phase where our predictive models were used to predict the ideal task variation to maximize performance, followed by an interference-correction phase. We then investigated the associations between predicted and actual performances and evaluated the outcome of correction strategies.

RESULT

The predictive models resulted in respective adjusted R-squared values of 0.942, 0.724, and 0.939 for the first, second, and third variation of the task, respectively. The statistical analyses showed significant associations between the performances predicted by predictive models and the actual performances for the first two task variations, with rhos of 0.7289 (p-value = 0.011) and 0.6970 (p-value = 0.017), respectively. For 81.82 % of the subjects, the task/workload correction stage correctly determined which task variation provided the highest accuracy, with an average performance gain of 5.18 % when applying the correction strategies.

CONCLUSION

Our proposed method can lead to an integrated multimodal predictive framework to compensate for BCI performance variability, particularly, for people with severe motor deficits.

What stakeholders with neurodegenerative conditions value about speech and accuracy in development of BCI systems for communication

This research examined values of individuals with neurodegenerative conditions about features of speed and accuracy as they consider potential use of augmentative and alternative communication brain-computer interface systems (AAC-BCI). Sixty-six individuals with neurodegenerative disease responded to prompts about six hypothetical ethical vignettes. Data were analyzed with qualitative content analysis. The following themes emerged. (1) Disease progression may contribute to the trade-off between speed and accuracy with AAC-BCI systems. (2) Individual experiences with technology use inform their views about the speed-accuracy trade-off. (3) There is a range of views about how slow or inaccurate communication may impact relationships, the integrity of a message, and quality of life. (4) Design solutions are proposed to address trade-offs in AAC-BCI systems. With the rapid development of AAC-BCI systems, user-centered design must integrate values of potential end-users illustrating that context, partner, message, and environment impact the prioritization of speed or accuracy in any communication exchange.

Impact of dataset size and long-term ECoG-based BCI usage on deep learning decoders performance

Introduction

In brain-computer interfaces (BCI) research, recording data is time-consuming and expensive, which limits access to big datasets. This may influence the BCI system performance as machine learning methods depend strongly on the training dataset size. Important questions arise: taking into account neuronal signal characteristics (e.g., non-stationarity), can we achieve higher decoding performance with more data to train decoders? What is the perspective for further improvement with time in the case of long-term BCI studies? In this study, we investigated the impact of long-term recordings on motor imagery decoding from two main perspectives: model requirements regarding dataset size and potential for patient adaptation.

Methods

We evaluated the multilinear model and two deep learning (DL) models on a long-term BCI & Tetraplegia (ClinicalTrials.gov identifier: NCT02550522) clinical trial dataset containing 43 sessions of ECoG recordings performed with a tetraplegic patient. In the experiment, a participant executed 3D virtual hand translation using motor imagery patterns. We designed multiple computational experiments in which training datasets were increased or translated to investigate the relationship between models' performance and different factors influencing recordings.

Results

Our results showed that DL decoders showed similar requirements regarding the dataset size compared to the multilinear model while demonstrating higher decoding performance. Moreover, high decoding performance was obtained with relatively small datasets recorded later in the experiment, suggesting motor imagery patterns improvement and patient adaptation during the long-term experiment. Finally, we proposed UMAP embeddings and local intrinsic dimensionality as a way to visualize the data and potentially evaluate data quality.

Discussion

DL-based decoding is a prospective approach in BCI which may be efficiently applied with real-life dataset size. Patient-decoder co-adaptation is an important factor to consider in long-term clinical BCI.

Human factors engineering of BCI: an evaluation for satisfaction of BCI based on motor imagery

Existing brain-computer interface (BCI) research has made great progress in improving the accuracy and information transfer rate (ITR) of BCI systems. However, the practicability of BCI is still difficult to achieve. One of the important reasons for this difficulty is that human factors are not fully considered in the research and development of BCI. As a result, BCI systems have not yet reached users' expectations. In this study, we investigate a BCI system of motor imagery for lower limb synchronous rehabilitation as an example. From the perspective of human factors engineering of BCI, a comprehensive evaluation method of BCI system development is proposed based on the concept of human-centered design and evaluation. Subjects' satisfaction ratings for BCI sensors, visual analog scale (VAS), subjects' satisfaction rating of the BCI system, and the mental workload rating for subjects manipulating the BCI system, as well as interview/follow-up comprehensive evaluation of motor imagery of BCI (MI-BCI) system satisfaction were used. The methods and concepts proposed in this study provide useful insights for the design of personalized MI-BCI. We expect that the human factors engineering of BCI could be applied to the design and satisfaction evaluation of MI-BCI, so as to promote the practical application of this kind of BCI.

Evaluating the perspectives of those with severe physical impairments while learning BCI control of a commercial augmentative and alternative communication paradigm

Augmentative and alternative communication (AAC) techniques can provide access to communication for individuals with severe physical impairments. Brain-computer interface (BCI) access techniques may serve alongside existing AAC access methods to provide communication device control. However, there is limited information available about how individual perspectives change with motor-based BCI-AAC learning. Four individuals with ALS completed 12 BCI-AAC training sessions in which they made letter selections during an automatic row-column scanning pattern via a motor-based BCI-AAC. Recurring measures were taken before and after each BCI-AAC training session to evaluate changes associated with BCI-AAC performance, and included measures of fatigue, frustration, mental effort, physical effort, device satisfaction, and overall ease of device control. Levels of pre- to post-fatigue were low for use of the BCI-AAC system. However, participants indicated different perceptions of the term fatigue, with three participants discussing fatigue to be generally synonymous with physical effort, and one mental effort. Satisfaction with the BCI-AAC system was related to BCI-AAC performance for two participants, and levels of frustration for two participants. Considering a range of person-centered measures in future clinical BCI-AAC applications is important for optimizing and standardizing BCI-AAC assessment procedures.

Learning to control a BMI-driven wheelchair for people with severe tetraplegia

Mind-controlled wheelchairs are an intriguing assistive mobility solution applicable in complete paralysis. Despite progress in brain-machine interface (BMI) technology, its translation remains elusive. The primary objective of this study is to probe the hypothesis that BMI skill acquisition by end-users is fundamental to control a non-invasive brain-actuated intelligent wheelchair in real-world settings. We demonstrate that three tetraplegic spinal-cord injury users could be trained to operate a non-invasive, self-paced thought-controlled wheelchair and execute complex navigation tasks. However, only the two users exhibiting increasing decoding performance and feature discriminancy, significant neuroplasticity changes and improved BMI command latency, achieved high navigation performance. In addition, we show that dexterous, continuous control of robots is possible through low-degree of freedom, discrete and uncertain control channels like a motor imagery BMI, by blending human and artificial intelligence through shared-control methodologies. We posit that subject learning and shared-control are the key components paving the way for translational non-invasive BMI.

Closed-loop motor imagery EEG simulation for brain-computer interfaces

In a brain-computer interface (BCI) system, the testing of decoding algorithms, tasks, and their parameters is critical for optimizing performance. However, conducting human experiments can be costly and time-consuming, especially when investigating broad sets of parameters. Attempts to utilize previously collected data in offline analysis lack a co-adaptive feedback loop between the system and the user present online, limiting the applicability of the conclusions obtained to real-world uses of BCI. As such, a number of studies have attempted to address this cost-wise middle ground between offline and live experimentation with real-time neural activity simulators. We present one such system which generates motor imagery electroencephalography (EEG) via forward modeling and novel motor intention encoding models for conducting sensorimotor rhythm (SMR)-based continuous cursor control experiments in a closed-loop setting. We use the proposed simulator with 10 healthy human subjects to test the effect of three decoder and task parameters across 10 different values. Our simulated approach produces similar statistical conclusions to those produced during parallel, paired, online experimentation, but in 55% of the time. Notably, both online and simulated experimentation expressed a positive effect of cursor velocity limit on performance regardless of subject average performance, supporting the idea of relaxing constraints on cursor gain in online continuous cursor control. We demonstrate the merits of our closed-loop motor imagery EEG simulation, and provide an open-source framework to the community for closed-loop SMR-based BCI studies in the future. All code including the simulator have been made available on GitHub.

A systematic review of research on augmentative and alternative communication brain-computer interface systems for individuals with disabilities

Augmentative and alternative communication brain-computer interface (AAC-BCI) systems are intended to offer communication access to people with severe speech and physical impairment (SSPI) without requiring volitional movement. As the field moves toward clinical implementation of AAC-BCI systems, research involving participants with SSPI is essential. Research has demonstrated variability in AAC-BCI system performance across users, and mixed results for comparisons of performance for users with and without disabilities. The aims of this systematic review were to (1) describe study, system, and participant characteristics reported in BCI research, (2) summarize the communication task performance of participants with disabilities using AAC-BCI systems, and (3) explore any differences in performance for participants with and without disabilities. Electronic databases were searched in May, 2018, and March, 2021, identifying 6065 records, of which 73 met inclusion criteria. Non-experimental study designs were common and sample sizes were typically small, with approximately half of studies involving five or fewer participants with disabilities. There was considerable variability in participant characteristics, and in how those characteristics were reported. Over 60% of studies reported an average selection accuracy ≤70% for participants with disabilities in at least one tested condition. However, some studies excluded participants who did not reach a specific system performance criterion, and others did not state whether any participants were excluded based on performance. Twenty-nine studies included participants both with and without disabilities, but few reported statistical analyses comparing performance between the two groups. Results suggest that AAC-BCI systems show promise for supporting communication for people with SSPI, but they remain ineffective for some individuals. The lack of standards in reporting outcome measures makes it difficult to synthesize data across studies. Further research is needed to demonstrate efficacy of AAC-BCI systems for people who experience SSPI of varying etiologies and severity levels, and these individuals should be included in system design and testing. Consensus in terminology and consistent participant, protocol, and performance description will facilitate the exploration of user and system characteristics that positively or negatively affect AAC-BCI use, and support innovations that will make this technology more useful to a broader group of people.

Clinical trial registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018095345, PROSPERO: CRD42018095345.

Iterative Development of a Software to Facilitate Independent Home Use of BCI Technologies for Children with Quadriplegic Cerebral Palsy

Brain-computer interfaces (BCIs) are emerging as a new solution for children with severe disabilities to interact with the world. However, BCI technologies have yet to reach end-users in their daily lives due to significant translational gaps. To address these gaps, we applied user-centered design principles to establish a home BCI program for children with quadriplegic cerebral palsy. This work describes the technical development of the software we designed to facilitate BCI use at home. Children and their families were involved at each design stage to evaluate and provide feedback. Since deployment, seven families have successfully used the system independently at home and continue to use BCI at home to further enable participation and independence for their children. Clinical relevance- The design and successful implementation of user-centered software for home use will both inform on the feasibility of BCI as a long-term access solution for children with neurological disabilities as well as decrease barriers of accessibility and availability of BCI technologies for end-users.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Image-Based Learning Using Gradient Class Activation Maps for Enhanced Physiological Interpretability of Motor Imagery Skills

Brain activity stimulated by the motor imagery paradigm (MI) is measured by Electroencephalography (EEG), which has several advantages to be implemented with the widely used Brain–Computer Interfaces (BCIs) technology. However, the substantial inter/intra variability of recorded data significantly influences individual skills on the achieved performance. This study explores the ability to distinguish between MI tasks and the interpretability of the brain’s ability to produce elicited mental responses with improved accuracy. We develop a Deep and Wide Convolutional Neuronal Network fed by a set of topoplots extracted from the multichannel EEG data. Further, we perform a visualization technique based on gradient-based class activation maps (namely, GradCam++) at different intervals along the MI paradigm timeline to account for intra-subject variability in neural responses over time. We also cluster the dynamic spatial representation of the extracted maps across the subject set to come to a deeper understanding of MI-BCI coordination skills. According to the results obtained from the evaluated GigaScience Database of motor-evoked potentials, the developed approach enhances the physiological explanation of motor imagery in aspects such as neural synchronization between rhythms, brain lateralization, and the ability to predict the MI onset responses and their evolution during training sessions.

Applying Implementation Science to Support Active Collaboration in Noninvasive Brain-Computer Interface Development and Translation for Augmentative and Alternative Communication

PURPOSE

The purpose of this article is to consider how, alongside engineering advancements, noninvasive brain-computer interface (BCI) for augmentative and alternative communication (AAC; BCI-AAC) developments can leverage implementation science to increase the clinical impact of this technology. We offer the Consolidated Framework for Implementation Research (CFIR) as a structure to help guide future BCI-AAC research. Specifically, we discuss CFIR primary domains that include intervention characteristics, the outer and inner settings, the individuals involved in the intervention, and the process of implementation, alongside pertinent subdomains including adaptability, cost, patient needs and recourses, implementation climate, other personal attributes, and the process of engaging. The authors support their view with current citations from both the AAC and BCI-AAC fields.

CONCLUSIONS

The article aimed to provide thoughtful considerations for how future research may leverage the CFIR to support meaningful BCI-AAC translation for those with severe physical impairments. We believe that, although significant barriers to BCI-AAC development still exist, incorporating implementation research may be timely for the field of BCI-AAC and help account for diversity in end users, navigate implementation obstacles, and support a smooth and efficient translation of BCI-AAC technology. Moreover, the sooner clinicians, individuals who use AAC, their support networks, and engineers collectively improve BCI-AAC outcomes and the efficiency of translation, the sooner BCI-AAC may become an everyday tool in the AAC arsenal.

An Impending Paradigm Shift in Motor Imagery Based Brain-Computer Interfaces

The development of reliable assistive devices for patients that suffer from motor impairments following central nervous system lesions remains a major challenge in the field of non-invasive Brain-Computer Interfaces (BCIs). These approaches are predominated by electroencephalography and rely on advanced signal processing and machine learning methods to extract neural correlates of motor activity. However, despite tremendous and still ongoing efforts, their value as effective clinical tools remains limited. We advocate that a rather overlooked research avenue lies in efforts to question neurophysiological markers traditionally targeted in non-invasive motor BCIs. We propose an alternative approach grounded by recent fundamental advances in non-invasive neurophysiology, specifically subject-specific feature extraction of sensorimotor bursts of activity recorded via (possibly magnetoencephalography-optimized) electroencephalography. This path holds promise in overcoming a significant proportion of existing limitations, and could foster the wider adoption of online BCIs in rehabilitation protocols.

A Bibliometric Analysis of Human-Machine Interaction Methodology for Electric-Powered Wheelchairs Driving from 1998 to 2020

Electric power wheelchairs (EPWs) enhance the mobility capability of the elderly and the disabled, while the human-machine interaction (HMI) determines how well the human intention will be precisely delivered and how human-machine system cooperation will be efficiently conducted. A bibliometric quantitative analysis of 1154 publications related to this research field, published between 1998 and 2020, was conducted. We identified the development status, contributors, hot topics, and potential future research directions of this field. We believe that the combination of intelligence and humanization of an EPW HMI system based on human-machine collaboration is an emerging trend in EPW HMI methodology research. Particular attention should be paid to evaluating the applicability and benefits of the EPW HMI methodology for the users, as well as how much it contributes to society. This study offers researchers a comprehensive understanding of EPW HMI studies in the past 22 years and latest trends from the evolutionary footprints and forward-thinking insights regarding future research.

Challenges and Opportunities for the Future of Brain-Computer Interface in Neurorehabilitation

Brain-computer interfaces (BCIs) provide a unique technological solution to circumvent the damaged motor system. For neurorehabilitation, the BCI can be used to translate neural signals associated with movement intentions into tangible feedback for the patient, when they are unable to generate functional movement themselves. Clinical interest in BCI is growing rapidly, as it would facilitate rehabilitation to commence earlier following brain damage and provides options for patients who are unable to partake in traditional physical therapy. However, substantial challenges with existing BCI implementations have prevented its widespread adoption. Recent advances in knowledge and technology provide opportunities to facilitate a change, provided that researchers and clinicians using BCI agree on standardisation of guidelines for protocols and shared efforts to uncover mechanisms. We propose that addressing the speed and effectiveness of learning BCI control are priorities for the field, which may be improved by multimodal or multi-stage approaches harnessing more sensitive neuroimaging technologies in the early learning stages, before transitioning to more practical, mobile implementations. Clarification of the neural mechanisms that give rise to improvement in motor function is an essential next step towards justifying clinical use of BCI. In particular, quantifying the unknown contribution of non-motor mechanisms to motor recovery calls for more stringent control conditions in experimental work. Here we provide a contemporary viewpoint on the factors impeding the scalability of BCI. Further, we provide a future outlook for optimal design of the technology to best exploit its unique potential, and best practices for research and reporting of findings.

Design Considerations for Long Term Non-invasive Brain Computer Interface Training With Tetraplegic CYBATHLON Pilot

Several studies in the recent past have demonstrated how Brain Computer Interface (BCI) technology can uncover the neural mechanisms underlying various tasks and translate them into control commands. While a multitude of studies have demonstrated the theoretic potential of BCI, a point of concern is that the studies are still confined to lab settings and mostly limited to healthy, able-bodied subjects. The CYBATHLON 2020 BCI race represents an opportunity to further develop BCI design strategies for use in real-time applications with a tetraplegic end user. In this study, as part of the preparation to participate in CYBATHLON 2020 BCI race, we investigate the design aspects of BCI in relation to the choice of its components, in particular, the type of calibration paradigm and its relevance for long-term use. The end goal was to develop a user-friendly and engaging interface suited for long-term use, especially for a spinal-cord injured (SCI) patient. We compared the efficacy of conventional open-loop calibration paradigms with real-time closed-loop paradigms, using pre-trained BCI decoders. Various indicators of performance were analyzed for this study, including the resulting classification performance, game completion time, brain activation maps, and also subjective feedback from the pilot. Our results show that the closed-loop calibration paradigms with real-time feedback is more engaging for the pilot. They also show an indication of achieving better online median classification performance as compared to conventional calibration paradigms (p = 0.0008). We also observe that stronger and more localized brain activation patterns are elicited in the closed-loop paradigm in which the experiment interface closely resembled the end application. Thus, based on this longitudinal evaluation of single-subject data, we demonstrate that BCI-based calibration paradigms with active user-engagement, such as with real-time feedback, could help in achieving better user acceptability and performance.

An Online Data Visualization Feedback Protocol for Motor Imagery-Based BCI Training

Brain-computer interface (BCI) has developed rapidly over the past two decades, mainly due to advancements in machine learning. Subjects must learn to modulate their brain activities to ensure a successful BCI. Feedback training is a practical approach to this learning process; however, the commonly used classifier-dependent approaches have inherent limitations such as the need for calibration and a lack of continuous feedback over long periods of time. This paper proposes an online data visualization feedback protocol that intuitively reflects the EEG distribution in Riemannian geometry in real time. Rather than learning a hyperplane, the Riemannian geometry formulation allows iterative learning of prototypical covariance matrices that are translated into visualized feedback through diffusion map process. Ten subjects were recruited for MI-BCI (motor imagery-BCI) training experiments. The subjects learned to modulate their sensorimotor rhythm to centralize the points within one category and to separate points belonging to different categories. The results show favorable overall training effects in terms of the class distinctiveness and EEG feature discriminancy over a 3-day training with 30% learners. A steadily increased class distinctiveness in the last three sessions suggests that the advanced training protocol is effective. The optimal frequency band was consistent during the 3-day training, and the difference between subjects with good or low MI-BCI performance could be clearly observed. We believe that the proposed feedback protocol has promising application prospect.

[Human factors engineering of brain-computer interface and its applications: Human-centered brain-computer interface design and evaluation methodology]

Brain-computer interface (BCI) is a revolutionizing human-computer Interaction, which is developing towards the direction of intelligent brain-computer interaction and brain-computer intelligent integration. However, the practical application of BCI is facing great challenges. The maturity of BCI technology has not yet reached the needs of users. The traditional design method of BCI needs to be improved. It is necessary to pay attention to BCI human factors engineering, which plays an important role in narrowing the gap between research and practical application, but it has not attracted enough attention and has not been specifically discussed in depth. Aiming at BCI human factors engineering, this article expounds the design requirements (from users), design ideas, objectives and methods, as well as evaluation indexes of BCI with the human-centred-design. BCI human factors engineering is expected to make BCI system design under different use conditions more in line with human characteristics, abilities and needs, improve the user satisfaction of BCI system, enhance the user experience of BCI system, improve the intelligence of BCI, and make BCI move towards practical application.

Long-Term BCI Training of a Tetraplegic User: Adaptive Riemannian Classifiers and User Training

While often presented as promising assistive technologies for motor-impaired users, electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) remain barely used outside laboratories due to low reliability in real-life conditions. There is thus a need to design long-term reliable BCIs that can be used outside-of-the-lab by end-users, e.g., severely motor-impaired ones. Therefore, we propose and evaluate the design of a multi-class Mental Task (MT)-based BCI for longitudinal training (20 sessions over 3 months) of a tetraplegic user for the CYBATHLON BCI series 2019. In this BCI championship, tetraplegic pilots are mentally driving a virtual car in a racing video game. We aimed at combining a progressive user MT-BCI training with a newly designed machine learning pipeline based on adaptive Riemannian classifiers shown to be promising for real-life applications. We followed a two step training process: the first 11 sessions served to train the user to control a 2-class MT-BCI by performing either two cognitive tasks (REST and MENTAL SUBTRACTION) or two motor-imagery tasks (LEFT-HAND and RIGHT-HAND). The second training step (9 remaining sessions) applied an adaptive, session-independent Riemannian classifier that combined all 4 MT classes used before. Moreover, as our Riemannian classifier was incrementally updated in an unsupervised way it would capture both within and between-session non-stationarity. Experimental evidences confirm the effectiveness of this approach. Namely, the classification accuracy improved by about 30% at the end of the training compared to initial sessions. We also studied the neural correlates of this performance improvement. Using a newly proposed BCI user learning metric, we could show our user learned to improve his BCI control by producing EEG signals matching increasingly more the BCI classifier training data distribution, rather than by improving his EEG class discrimination. However, the resulting improvement was effective only on synchronous (cue-based) BCI and it did not translate into improved CYBATHLON BCI game performances. For the sake of overcoming this in the future, we unveil possible reasons for these limited gaming performances and identify a number of promising future research directions. Importantly, we also report on the evolution of the user's neurophysiological patterns and user experience throughout the BCI training and competition.

A multi-modal modified feedback self-paced BCI to control the gait of an avatar

Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation.

OBJECTIVE

to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems.

APPROACH

This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants.

MAIN RESULTS

All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode.

SIGNIFICANCE

This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.

A review of user training methods in brain computer interfaces based on mental tasks

Mental-Tasks based Brain-Computer Interfaces (MT-BCIs) allow their users to interact with an external device solely by using brain signals produced through mental tasks. While MT-BCIs are promising for many applications, they are still barely used outside laboratories due to their lack of reliability. MT-BCIs require their users to develop the ability to self-regulate specific brain signals. However, the human learning process to control a BCI is still relatively poorly understood and how to optimally train this ability is currently under investigation. Despite their promises and achievements, traditional training programs have been shown to be sub-optimal and could be further improved. In order to optimize user training and improve BCI performance, human factors should be taken into account. An interdisciplinary approach should be adopted to provide learners with appropriate and/or adaptive training. In this article, we provide an overview of existing methods for MT-BCI user training - notably in terms of environment, instructions, feedback and exercises. We present a categorization and taxonomy of these training approaches, provide guidelines on how to choose the best methods and identify open challenges and perspectives to further improve MT-BCI user training.

Effects of Gamification in BCI Functional Rehabilitation

Objective

To evaluate whether introducing gamification in BCI rehabilitation of the upper limbs of post-stroke patients has a positive impact on their experience without altering their efficacy in creating motor mental images (MI).

Design

A game was designed purposely adapted to the pace and goals of an established BCI-rehabilitation protocol. Rehabilitation was based on a double feedback: functional electrostimulation and animation of a virtual avatar of the patient’s limbs. The game introduced a narrative on top of this visual feedback with an external goal to achieve (protecting bits of cheese from a rat character). A pilot study was performed with 10 patients and a control group of six volunteers. Two rehabilitation sessions were done, each made up of one stage of calibration and two training stages, some stages with the game and others without. The accuracy of the classification computed was taken as a measure to compare the efficacy of MI. Users’ opinions were gathered through a questionnaire. No potentially identifiable human images or data are presented in this study.

Results

The gamified rehabilitation presented in the pilot study does not impact on the efficacy of MI, but it improves users experience making it more fun.

Conclusion

These preliminary results are encouraging to continue investigating how game narratives can be introduced in BCI rehabilitation to make it more gratifying and engaging.

User Adaptation to Closed-Loop Decoding of Motor Imagery Termination

One of the most popular methods in non-invasive brain machine interfaces (BMI) relies on the decoding of sensorimotor rhythms associated to sustained motor imagery. Although motor imagery has been intensively studied, its termination is mostly neglected.

OBJECTIVE

Here, we provide insights in the decoding of motor imagery termination and investigate the use of such decoder in closed-loop BMI.

METHODS

Participants (N = 9) were asked to perform kinesthetic motor imagery of both hands simultaneously cued with a clock indicating the initiation and termination of the action. Using electroencephalogram (EEG) signals, we built a decoder to detect the transition between event-related desynchronization and event-related synchronization. Features for this decoder were correlates of motor termination in the upper μ and β bands.

RESULTS

The decoder reached an accuracy of 76.2% (N = 9), revealing the high robustness of our approach. More importantly, this paper shows that the decoding of motor termination has an intrinsic latency mainly due to the delayed appearance of its correlates. Because the latency was consistent and thus predictable, users were able to compensate it after training.

CONCLUSION

Using our decoding system, BMI users were able to adapt their behavior and modulate their sensorimotor rhythm to stop the device (clock) accurately on time.

SIGNIFICANCE

These results show the importance of closed-loop evaluations of BMI decoders and open new possibilities for BMI control using decoding of movement termination.

Stimulus modality influences session-to-session transfer of training effects in auditory and tactile streaming-based P300 brain-computer interfaces

Despite recent successes, patients suffering from locked-in syndrome (LIS) still struggle to communicate using vision-independent brain–computer interfaces (BCIs). In this study, we compared auditory and tactile BCIs, regarding training effects and cross-stimulus-modality transfer effects, when switching between stimulus modalities. We utilized a streaming-based P300 BCI, which was developed as a low workload approach to prevent potential BCI-inefficiency. We randomly assigned 20 healthy participants to two groups. The participants received three sessions of training either using an auditory BCI or using a tactile BCI. In an additional fourth session, BCI versions were switched to explore possible cross-stimulus-modality transfer effects. Both BCI versions could be operated successfully in the first session by the majority of the participants, with the tactile BCI being experienced as more intuitive. Significant training effects were found mostly in the auditory BCI group and strong evidence for a cross-stimulus-modality transfer occurred for the auditory training group that switched to the tactile version but not vice versa. All participants were able to control at least one BCI version, suggesting that the investigated paradigms are generally feasible and merit further research into their applicability with LIS end-users. Individual preferences regarding stimulus modality should be considered.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Clinician awareness of brain computer interfaces: a Canadian national survey

Background

Individuals with severe neurological disabilities but preserved cognition, including children, are often precluded from connecting with their environments. Brain computer interfaces (BCI) are a potential solution where advancing technologies create new clinical opportunities. We evaluated clinician awareness as a modifiable barrier to progress and identified eligible populations.

Methods

We executed a national, population-based, cross-sectional survey of physician specialists caring for persons with severe disability. An evidence- and experience-based survey had three themes: clinician BCI knowledge, eligible populations, and potential impact. A BCI knowledge index was created and scored. Canadian adult and pediatric neurologists, physiatrists and a subset of developmental pediatricians were contacted. Secure, web-based software administered the survey via email with online data collection.

Results

Of 922 valid emails (664 neurologists, 253 physiatrists), 137 (15%) responded. One third estimated that ≥10% of their patients had severe neurological disability with cognitive capacity. BCI knowledge scores were low with > 40% identifying as less than “vaguely aware” and only 15% as “somewhat familiar” or better. Knowledge did not differ across specialties. Only 6 physicians (4%) had patients using BCI. Communication and wheelchair control rated highest for potentially improving quality of life. Most (81%) felt BCI had high potential to improve quality of life. Estimates suggested that > 13,000 Canadians (36 M population) might benefit from BCI technologies.

Conclusions

Despite high potential and thousands of patients who might benefit, BCI awareness among clinicians caring for disabled persons is poor. Further, functional priorities for BCI applications may differ between medical professionals and potential BCI users, perhaps reflecting that clinicians possess a less accurate understanding of the desires and needs of potential end-users. Improving knowledge and engaging both clinicians and patients could facilitate BCI program development to improve patient outcomes.

Hearing the needs of clinical users

In the past 10 years, brain-computer interfaces (BCIs) for controlling assistive devices have seen tremendous progress with respect to reliability and learnability, and numerous exemplary applications were demonstrated to be controllable by a BCI. Yet, BCI-controlled applications are hardly used for patients with neurologic or neurodegenerative disease. Such patient groups are considered potential end-users of BCI, specifically for replacing or improving lost function. We argue that BCI research and development still faces a translational gap, i.e., the knowledge of how to bring BCIs from the laboratory to the field is insufficient. BCI-controlled applications lack usability and accessibility; both constitute two sides of one coin, which is the key to use in daily life and to prevent nonuse. To increase usability, we suggest rigorously adopting the user-centered design in applied BCI research and development. To provide accessibility, assistive technology (AT) experts, providers, and other stakeholders have to be included in the user-centered process. BCI experts have to ensure the transfer of knowledge to AT professionals, and listen to the needs of primary, secondary, and tertiary end-users of BCI technology. Addressing both, usability and accessibility, in applied BCI research and development will bridge the translational gap and ensure that the needs of clinical end-users are heard, understood, addressed, and fulfilled.

Using Robust Principal Component Analysis to Reduce EEG Intra-Trial Variability

Practical brain-computer interfaces need to overcome several challenges, including tolerance to signal variability within- and across sessions. We introduce Robust Principal Component Analysis (RPCA) as a potential approach to tackle intra-trial variability. Assuming that subjects undergo the same cognitive process or perform the same task in a short period (e.g., a few seconds), as a result, the signal of interest should be represented by only a few components. We verified this approach on a workload detection task, where subjects needed to pilot a simulated drone. We used RPCA as a processing step to decrease trial variability and assessed its impact on classification accuracy. Our results showed that RPCA significantly increased performance in both at group and subject level analysis. On average, class-balanced accuracy when simulating RPCA online increased from 63.9% up to 70.6% $(p~ 0 . 001)$.

Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment

This paper presents a gamified motor imagery brain-computer interface (MI-BCI) training in immersive virtual reality. The aim of the proposed training method is to increase engagement, attention, and motivation in co-adaptive event-driven MI-BCI training. This was achieved using gamification, progressive increase of the training pace, and virtual reality design reinforcing body ownership transfer (embodiment) into the avatar. From the 20 healthy participants performing 6 runs of 2-class MI-BCI training (left/right hand), 19 were trained for a basic level of MI-BCI operation, with average peak accuracy in the session = 75.84%. This confirms the proposed training method succeeded in improvement of the MI-BCI skills; moreover, participants were leaving the session in high positive affect. Although the performance was not directly correlated to the degree of embodiment, subjective magnitude of the body ownership transfer illusion correlated with the ability to modulate the sensorimotor rhythm.

Critiquing the Concept of BCI Illiteracy

Brain-computer interfaces (BCIs) are a form of technology that read a user's neural signals to perform a task, often with the aim of inferring user intention. They demonstrate potential in a wide range of clinical, commercial, and personal applications. But BCIs are not always simple to operate, and even with training some BCI users do not operate their systems as intended. Many researchers have described this phenomenon as "BCI illiteracy," and a body of research has emerged aiming to characterize, predict, and solve this perceived problem. However, BCI illiteracy is an inadequate concept for explaining difficulty that users face in operating BCI systems. BCI illiteracy is a methodologically weak concept; furthermore, it relies on the flawed assumption that BCI users possess physiological or functional traits that prevent proficient performance during BCI use. Alternative concepts to BCI illiteracy may offer better outcomes for prospective users and may avoid the conceptual pitfalls that BCI illiteracy brings to the BCI research process.

Histogram of Gradient Orientations of Signal Plots Applied to P300 Detection

The analysis of Electroencephalographic (EEG) signals is of ulterior importance to aid in the diagnosis of mental disease and to increase our understanding of the brain. Traditionally, clinical EEG has been analyzed in terms of temporal waveforms, looking at rhythms in spontaneous activity, subjectively identifying troughs and peaks in Event-Related Potentials (ERP), or by studying graphoelements in pathological sleep stages. Additionally, the discipline of Brain Computer Interfaces (BCI) requires new methods to decode patterns from non-invasive EEG signals. This field is developing alternative communication pathways to transmit volitional information from the Central Nervous System. The technology could potentially enhance the quality of life of patients affected by neurodegenerative disorders and other mental illness. This work mimics what electroencephalographers have been doing clinically, visually inspecting, and categorizing phenomena within the EEG by the extraction of features from images of signal plots. These features are constructed based on the calculation of histograms of oriented gradients from pixels around the signal plot. It aims to provide a new objective framework to analyze, characterize and classify EEG signal waveforms. The feasibility of the method is outlined by detecting the P300, an ERP elicited by the oddball paradigm of rare events, and implementing an offline P300-based BCI Speller. The validity of the proposal is shown by offline processing a public dataset of Amyotrophic Lateral Sclerosis (ALS) patients and an own dataset of healthy subjects.

Thought-based interaction: Same data, same methods, different results?

Restoration of communication in people with complete motor paralysis—a condition called complete locked-in state (CLIS)—is one of the greatest challenges of brain-computer interface (BCI) research. New findings have recently been presented that bring us one step closer to this goal. However, the validity of the evidence has been questioned: independent reanalysis of the same data yielded significantly different results. Reasons for the failure to replicate the findings must be of a methodological nature. What is the best practice to ensure that results are stringent and conclusive and analyses replicable? Confirmation bias and the counterintuitive nature of probability may lead to an overly optimistic interpretation of new evidence. Lack of detail complicates replicability.

This Primer explores a recent debate about brain-computer interface studies, observing that confirmation bias and the counter-intuitive nature of probability may lead to an overly optimistic interpretation of scientific results; furthermore, a lack of details complicates replicability of analyses and experiments.

New and emerging access technologies for adults with complex communication needs and severe motor impairments: State of the science

Individuals with complex communication needs often use alternative access technologies to control their augmentative and alternative communication (AAC) devices, their computers, and mobile technologies. While a range of access devices is available, many challenges continue to exist, particularly for those with severe motor-control limitations. For some, access options may not be readily available or access itself may be inaccurate and frustrating. For others, access may be available but only under optimal conditions and support. There is an urgent need to develop new options for individuals with severe motor impairments and to leverage existing technology to improve efficiency, increase accuracy, and decrease fatigue of access. This paper describes person-centred research and development activities related to new and emerging access technologies, with a particular focus on adults with acquired neurological conditions.

Considering Augmentative and Alternative Communication Research for Brain-Computer Interface Practice

PURPOSE

Brain-computer interfaces (BCIs) aim to provide access to augmentative and alternative communication (AAC) devices via brain activity alone. However, while BCI technology is expanding in the laboratory setting there is minimal incorporation into clinical practice. Building upon established AAC research and clinical best practices may aid the clinical translation of BCI practice, allowing advancements in both fields to be fully leveraged.

METHOD

A multidisciplinary team developed considerations for how BCI products, practice, and policy may build upon existing AAC research, based upon published reports of existing AAC and BCI procedures.

OUTCOMES/BENEFITS

Within each consideration, a review of BCI research is provided, along with considerations regarding how BCI procedures may build upon existing AAC methods. The consistent use of clinical/research procedures across disciplines can help facilitate collaborative efforts, engaging a range-individuals within the AAC community in the transition of BCI into clinical practice.

HD-EEG Based Classification of Motor-Imagery Related Activity in Patients With Spinal Cord Injury

Brain computer interfaces (BCIs) are thought to revolutionize rehabilitation after SCI, e.g., by controlling neuroprostheses, exoskeletons, functional electrical stimulation, or a combination of these components. However, most BCI research was performed in healthy volunteers and it is unknown whether these results can be translated to patients with spinal cord injury because of neuroplasticity. We sought to examine whether high-density EEG (HD-EEG) could improve the performance of motor-imagery classification in patients with SCI. We recorded HD-EEG with 256 channels in 22 healthy controls and 7 patients with 14 recordings (4 patients had more than one recording) in an event related design. Participants were instructed acoustically to either imagine, execute, or observe foot and hand movements, or to rest. We calculated Fast Fourier Transform (FFT) and full frequency directed transfer function (ffDTF) for each condition and classified conditions pairwise with support vector machines when using only 2 channels over the sensorimotor area, full 10-20 montage, high-density montage of the sensorimotor cortex, and full HD-montage. Classification accuracies were comparable between patients and controls, with an advantage for controls for classifications that involved the foot movement condition. Full montages led to better results for both groups (p < 0.001), and classification accuracies were higher for FFT than for ffDTF (p < 0.001), for which the feature vector might be too long. However, full-montage 10–20 montage was comparable to high-density configurations. Motor-imagery driven control of neuroprostheses or BCI systems may perform as well in patients as in healthy volunteers with adequate technical configuration. We suggest the use of a whole-head montage and analysis of a broad frequency range.

EEG Waveform Analysis of P300 ERP with Applications to Brain Computer Interfaces

The Electroencephalography (EEG) is not just a mere clinical tool anymore. It has become the de-facto mobile, portable, non-invasive brain imaging sensor to harness brain information in real time. It is now being used to translate or decode brain signals, to diagnose diseases or to implement Brain Computer Interface (BCI) devices. The automatic decoding is mainly implemented by using quantitative algorithms to detect the cloaked information buried in the signal. However, clinical EEG is based intensively on waveforms and the structure of signal plots. Hence, the purpose of this work is to establish a bridge to fill this gap by reviewing and describing the procedures that have been used to detect patterns in the electroencephalographic waveforms, benchmarking them on a controlled pseudo-real dataset of a P300-Based BCI Speller and verifying their performance on a public dataset of a BCI Competition.