Performance measurement for brain-computer or brain-machine interfaces: a tutorial

A 10-year journey towards clinical translation of an implantable endovascular BCI a keynote lecture given at the BCI society meeting in Brussels

In the rapidly evolving field of brain-computer interfaces (BCIs), a novel modality for recording electrical brain signals has quietly emerged over the past decade. The technology is endovascular electrocorticography (ECoG), an innovation that stands alongside well-established methods such as electroencephalography, traditional ECoG, and single/multi-unit activity recording. This system was inspired by advancements in interventional cardiology, particularly the integration of electronics into various medical interventions. The breakthrough led to the development of the Stentrode system, which employs stent-mounted electrodes to record electrical brain activity for applications in a motor neuroprosthesis. This perspective explores four key areas in our quest to bring the Stentrode BCI to market: the critical patient need for autonomy driving our efforts, the hurdles and achievements in assessing BCI performance, the compelling advantages of our unique endovascular approach, and the essential steps for clinical translation and product commercialization.

Brain computer interfaces for cognitive enhancement in older people - challenges and applications: a systematic review

BACKGROUND

Brain-computer interface (BCI) offers promising solutions to cognitive enhancement in older people. Despite the clear progress received, there is limited evidence of BCI implementation for rehabilitation. This systematic review addresses BCI applications and challenges in the standard practice of EEG-based neurofeedback (NF) training in healthy older people or older people with mild cognitive impairment (MCI).

METHODS

Articles were searched via MEDLINE, PubMed, SCOPUS, SpringerLink, and Web of Science. 16 studies between 1st January 2010 to 1st November 2024 are included after screening using PRISMA. The risk of bias, system design, and neurofeedback protocols are reviewed.

RESULTS

The successful BCI applications in NF trials in older people were biased by the randomisation process and outcome measurement. Although the studies demonstrate promising results in effectiveness of research-grade BCI for cognitive enhancement in older people, it is premature to make definitive claims about widespread BCI usability and applicability.

SIGNIFICANCE

This review highlights the common issues in the field of EEG-based BCI for older people. Future BCI research could focus on trial design and BCI performance gaps between the old and the young to develop a robust BCI system that compensates for age-related declines in cognitive and motor functions.

Decoding Continuous Tracking Eye Movements from Cortical Spiking Activity

Eye movements are the primary way primates interact with the world. Understanding how the brain controls the eyes is therefore crucial for improving human health and designing visual rehabilitation devices. However, brain activity is challenging to decipher. Here, we leveraged machine learning algorithms to reconstruct tracking eye movements from high-resolution neuronal recordings. We found that continuous eye position could be decoded with high accuracy using spiking data from only a few dozen cortical neurons. We tested eight decoders and found that neural network models yielded the highest decoding accuracy. Simpler models performed well above chance with a substantial reduction in training time. We measured the impact of data quantity (e.g. number of neurons) and data format (e.g. bin width) on training time, inference time, and generalizability. Training models with more input data improved performance, as expected, but the format of the behavioral output was critical for emphasizing or omitting specific oculomotor events. Our results provide the first demonstration, to our knowledge, of continuously decoded eye movements across a large field of view. Our comprehensive investigation of predictive power and computational efficiency for common decoder architectures provides a much-needed foundation for future work on real-time gaze-tracking devices.

Intracortical brain-computer interfaces for improved motor function: a systematic review

In this systematic review, we address the status of intracortical brain-computer interfaces (iBCIs) applied to the motor cortex to improve function in patients with impaired motor ability. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 Guidelines for Systematic Reviews. Risk Of Bias In Non-randomized Studies - of Interventions (ROBINS-I) and the Effective Public Health Practice Project (EPHPP) were used to assess bias and quality. Advances in iBCIs in the last two decades demonstrated the use of iBCI to activate limbs for functional tasks, achieve neural typing for communication, and other applications. However, the inconsistency of performance metrics employed by these studies suggests the need for standardization. Each study was a pilot clinical trial consisting of 1-4, majority male (64.28 %) participants, with most trials featuring participants treated for more than 12 months (55.55 %). The systems treated patients with various conditions: amyotrophic lateral sclerosis, stroke, spinocerebellar degeneration without cerebellar involvement, and spinal cord injury. All participants presented with tetraplegia at implantation and were implanted with microelectrode arrays via pneumatic insertion, with nearly all electrode locations solely at the precentral gyrus of the motor cortex (88.88 %). The development of iBCI devices using neural signals from the motor cortex to improve motor-impaired patients has enhanced the ability of these systems to return ability to their users. However, many milestones remain before these devices can prove their feasibility for recovery. This review summarizes the achievements and shortfalls of these systems and their respective trials.

A novel theta-controlled vibrotactile brain-computer interface to treat chronic pain: a pilot study

Limitations in chronic pain therapies necessitate novel interventions that are effective, accessible, and safe. Brain-computer interfaces (BCIs) provide a promising modality for targeting neuropathology underlying chronic pain by converting recorded neural activity into perceivable outputs. Recent evidence suggests that increased frontal theta power (4-7 Hz) reflects pain relief from chronic and acute pain. Further studies have suggested that vibrotactile stimulation decreases pain intensity in experimental and clinical models. This longitudinal, non-randomized, open-label pilot study's objective was to reinforce frontal theta activity in six patients with chronic upper extremity pain using a novel vibrotactile neurofeedback BCI system. Patients increased their BCI performance, reflecting thought-driven control of neurofeedback, and showed a significant decrease in pain severity (1.29 ± 0.25 MAD, p = 0.03, q = 0.05) and pain interference (1.79 ± 1.10 MAD p = 0.03, q = 0.05) scores without any adverse events. Pain relief significantly correlated with frontal theta modulation. These findings highlight the potential of BCI-mediated cortico-sensory coupling of frontal theta with vibrotactile stimulation for alleviating chronic pain.

BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems

The Brain-Computer Interface (BCI) was envisioned as an assistive technology option for people with severe movement impairments. The traditional synchronous event-related potential (ERP) BCI design uses a fixed communication speed and is vulnerable to variations in attention. Recent ERP BCI designs have added asynchronous features, including abstention and dynamic stopping, but it remains a open question of how to evaluate asynchronous BCI performance. In this work, we build on the BCI-Utility metric to create the first evaluation metric with special consideration of the asynchronous features of self-paced BCIs. This metric considers accuracy as all of the following three - probability of a correct selection when a selection was intended, probability of making a selection when a selection was intended, and probability of an abstention when an abstention was intended. Further, it considers the average time required for a selection when using dynamic stopping and the proportion of intended selections versus abstentions. We establish the validity of the derived metric via extensive simulations, and illustrate and discuss its practical usage on real-world BCI data. We describe the relative contribution of different inputs with plots of BCI-Utility curves under different parameter settings. Generally, the BCI-Utility metric increases as any of the accuracy values increase and decreases as the expected time for an intended selection increases. Furthermore, in many situations, we find shortening the expected time of an intended selection is the most effective way to improve the BCI-Utility, which necessitates the advancement of asynchronous BCI systems capable of accurate abstention and dynamic stopping.

A Calibration-Free Hybrid Approach Combining SSVEP and EOG for Continuous Control

While SSVEP-BCI has been widely developed to control external devices, most of them rely on the discrete control strategy. The continuous SSVEP-BCI enables users to continuously deliver commands and receive real-time feedback from the devices, but it suffers from the transition state problem, a period the erroneous recognition, when users shift their gazes between targets. To resolve this issue, we proposed a novel calibration-free Bayesian approach by hybridizing SSVEP and electrooculography (EOG). First, canonical correlation analysis (CCA) was applied to detect the evoked SSVEPs, and saccade during the gaze shift was detected by EOG data using an adaptive threshold method. Then, the new target after the gaze shift was recognized based on a Bayesian optimization approach, which combined the detection of SSVEP and saccade together and calculated the optimized probability distribution of the targets. Eighteen healthy subjects participated in the offline and online experiments. The offline experiments showed that the proposed hybrid BCI had significantly higher overall continuous accuracy and shorter gaze-shifting time compared to FBCCA, CCA, MEC, and PSDA. In online experiments, the proposed hybrid BCI significantly outperformed CCA-based SSVEP-BCI in terms of continuous accuracy (77.61 ± 1.36%vs. 68.86 ± 1.08% and gaze-shifting time (0.93 ± 0.06s vs. 1.94 ± 0.08s). Additionally, participants also perceived a significant improvement over the CCA-based SSVEP-BCI when the newly proposed decoding approach was used. These results validated the efficacy of the proposed hybrid Bayesian approach for the BCI continuous control without any calibration. This study provides an effective framework for combining SSVEP and EOG, and promotes the potential applications of plug-and-play BCIs in continuous control.

Brain-Computer interface control of stepping from invasive electrocorticography upper-limb motor imagery in a patient with quadriplegia

Introduction: Most spinal cord injuries (SCI) result in lower extremities paralysis, thus diminishing ambulation. Using brain-computer interfaces (BCI), patients may regain leg control using neural signals that actuate assistive devices. Here, we present a case of a subject with cervical SCI with an implanted electrocorticography (ECoG) device and determined whether the system is capable of motor-imagery-initiated walking in an assistive ambulator. Methods: A 24-year-old male subject with cervical SCI (C5 ASIA A) was implanted before the study with an ECoG sensing device over the sensorimotor hand region of the brain. The subject used motor-imagery (MI) to train decoders to classify sensorimotor rhythms. Fifteen sessions of closed-loop trials followed in which the subject ambulated for one hour on a robotic-assisted weight-supported treadmill one to three times per week. We evaluated the stability of the best-performing decoder over time to initiate walking on the treadmill by decoding upper-limb (UL) MI. Results: An online bagged trees classifier performed best with an accuracy of 84.15% averaged across 9 weeks. Decoder accuracy remained stable following throughout closed-loop data collection. Discussion: These results demonstrate that decoding UL MI is a feasible control signal for use in lower-limb motor control. Invasive BCI systems designed for upper-extremity motor control can be extended for controlling systems beyond upper extremity control alone. Importantly, the decoders used were able to use the invasive signal over several weeks to accurately classify MI from the invasive signal. More work is needed to determine the long-term consequence between UL MI and the resulting lower-limb control.

Cortico-muscular coupling to control a hybrid brain-computer interface for upper limb motor rehabilitation: A pseudo-online study on stroke patients

Brain-Computer Interface (BCI) systems for motor rehabilitation after stroke have proven their efficacy to enhance upper limb motor recovery by reinforcing motor related brain activity. Hybrid BCIs (h-BCIs) exploit both central and peripheral activation and are frequently used in assistive BCIs to improve classification performances. However, in a rehabilitative context, brain and muscular features should be extracted to promote a favorable motor outcome, reinforcing not only the volitional control in the central motor system, but also the effective projection of motor commands to target muscles, i.e., central-to-peripheral communication. For this reason, we considered cortico-muscular coupling (CMC) as a feature for a h-BCI devoted to post-stroke upper limb motor rehabilitation. In this study, we performed a pseudo-online analysis on 13 healthy participants (CTRL) and 12 stroke patients (EXP) during executed (CTRL, EXP unaffected arm) and attempted (EXP affected arm) hand grasping and extension to optimize the translation of CMC computation and CMC-based movement detection from offline to online. Results showed that updating the CMC computation every 125 ms (shift of the sliding window) and accumulating two predictions before a final classification decision were the best trade-off between accuracy and speed in movement classification, independently from the movement type. The pseudo-online analysis on stroke participants revealed that both attempted and executed grasping/extension can be classified through a CMC-based movement detection with high performances in terms of classification speed (mean delay between movement detection and EMG onset around 580 ms) and accuracy (hit rate around 85%). The results obtained by means of this analysis will ground the design of a novel non-invasive h-BCI in which the control feature is derived from a combined EEG and EMG connectivity pattern estimated during upper limb movement attempts.

Recognition of Attentional States in VR Environment: An fNIRS Study

An improvement in ecological validity is one of the significant challenges for 21st-century neuroscience. At the same time, the study of neurocognitive processes in real-life situations requires good control of all variables relevant to the results. One possible solution that combines the capability of creating realistic experimental scenarios with adequate control of the test environment is virtual reality. Our goal was to develop an integrative research workspace involving a CW-fNIRS and head-mounted-display (HMD) technology dedicated to offline and online cognitive experiments. We designed an experimental study in a repeated-measures model on a group of BCI-naïve participants to verify our assumptions. The procedure included a 3D environment-adapted variant of the classic n-back task (2-back version). Tasks were divided into offline (calibration) and online (feedback) sessions. In both sessions, the signal was recorded during the cognitive task for within-group comparisons of changes in oxy-Hb concentration in the regions of interest (the dorsolateral prefrontal cortex-DLPFC and middle frontal gyrus-MFG). In the online session, the recorded signal changes were translated into real-time feedback. We hypothesized that it would be possible to obtain significantly higher than the level-of-chance threshold classification accuracy for the enhanced attention engagement (2-back task) vs. relaxed state in both conditions. Additionally, we measured participants' subjective experiences of the BCI control in terms of satisfaction. Our results confirmed hypotheses regarding the offline condition. In accordance with the hypotheses, combining fNIRS and HMD technologies enables the effective transfer of experimental cognitive procedures to a controlled VR environment. This opens the new possibility of creating more ecologically valid studies and training procedures.

Evaluating the clinical benefit of brain-computer interfaces for control of a personal computer

Brain-computer interfaces (BCIs) enabling the control of a personal computer could provide myriad benefits to individuals with disabilities including paralysis. However, to realize this potential, these BCIs must gain regulatory approval and be made clinically available beyond research participation. Therefore, a transition from engineering-oriented to clinically oriented outcome measures will be required in the evaluation of BCIs. This review examined how to assess the clinical benefit of BCIs for the control of a personal computer. We report that: 1) a variety of different patient-reported outcome measures can be used to evaluate improvements in how a patient feels, and we offer some considerations that should guide instrument selection. 2) Activities of daily living can be assessed to demonstrate improvements in how a patient functions, however, new instruments that are sensitive to increases in functional independence via the ability to perform digital tasks may be needed. 3) Benefits to how a patient survives has not previously been evaluated, but establishing patient-initiated communication channels using BCIs might facilitate quantifiable improvements in health outcomes.

Applying Implementation Science to Support Active Collaboration in Noninvasive Brain-Computer Interface Development and Translation for Augmentative and Alternative Communication

PURPOSE

The purpose of this article is to consider how, alongside engineering advancements, noninvasive brain-computer interface (BCI) for augmentative and alternative communication (AAC; BCI-AAC) developments can leverage implementation science to increase the clinical impact of this technology. We offer the Consolidated Framework for Implementation Research (CFIR) as a structure to help guide future BCI-AAC research. Specifically, we discuss CFIR primary domains that include intervention characteristics, the outer and inner settings, the individuals involved in the intervention, and the process of implementation, alongside pertinent subdomains including adaptability, cost, patient needs and recourses, implementation climate, other personal attributes, and the process of engaging. The authors support their view with current citations from both the AAC and BCI-AAC fields.

CONCLUSIONS

The article aimed to provide thoughtful considerations for how future research may leverage the CFIR to support meaningful BCI-AAC translation for those with severe physical impairments. We believe that, although significant barriers to BCI-AAC development still exist, incorporating implementation research may be timely for the field of BCI-AAC and help account for diversity in end users, navigate implementation obstacles, and support a smooth and efficient translation of BCI-AAC technology. Moreover, the sooner clinicians, individuals who use AAC, their support networks, and engineers collectively improve BCI-AAC outcomes and the efficiency of translation, the sooner BCI-AAC may become an everyday tool in the AAC arsenal.

Information Bottleneck as Optimisation Method for SSVEP-Based BCI

In this study, the information bottleneck method is proposed as an optimisation method for steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI). The information bottleneck is an information-theoretic optimisation method for solving problems with a trade-off between preserving meaningful information and compression. Its main practical application in machine learning is in representation learning or feature extraction. In this study, we use the information bottleneck to find optimal classification rule for a BCI. This is a novel application for the information bottleneck. This approach is particularly suitable for BCIs since the information bottleneck optimises the amount of information transferred by the BCI. Steady-state visual evoked potential-based BCIs often classify targets using very simple rules like choosing the class corresponding to the largest feature value. We call this classifier the arg max classifier. It is unlikely that this approach is optimal, and in this study, we propose a classification method specifically designed to optimise the performance measure of BCIs. This approach gives an advantage over standard machine learning methods, which aim to optimise different measures. The performance of the proposed algorithm is tested on two publicly available datasets in offline experiments. We use the standard power spectral density analysis (PSDA) and canonical correlation analysis (CCA) feature extraction methods on one dataset and show that the current approach outperforms most of the related studies on this dataset. On the second dataset, we use the task-related component analysis (TRCA) method and demonstrate that the proposed method outperforms the standard argmax classification rule in terms of information transfer rate when using a small number of classes. To our knowledge, this is the first time the information bottleneck is used in the context of SSVEP-based BCIs. The approach is unique in the sense that optimisation is done over the space of classification functions. It potentially improves the performance of BCIs and makes it easier to calibrate the system for different subjects.

Brain-Computer Interface: Advancement and Challenges

Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been conducted yet. Hence, a comprehensive overview of the BCI domain is presented in this study. This study covers several applications of BCI and upholds the significance of this domain. Then, each element of BCI systems, including techniques, datasets, feature extraction methods, evaluation measurement matrices, existing BCI algorithms, and classifiers, are explained concisely. In addition, a brief overview of the technologies or hardware, mostly sensors used in BCI, is appended. Finally, the paper investigates several unsolved challenges of the BCI and explains them with possible solutions.

Brain-Computer Interfaces for Children With Complex Communication Needs and Limited Mobility: A Systematic Review

Brain-computer interfaces (BCIs) represent a new frontier in the effort to maximize the ability of individuals with profound motor impairments to interact and communicate. While much literature points to BCIs' promise as an alternative access pathway, there have historically been few applications involving children and young adults with severe physical disabilities. As research is emerging in this sphere, this article aims to evaluate the current state of translating BCIs to the pediatric population. A systematic review was conducted using the Scopus, PubMed, and Ovid Medline databases. Studies of children and adolescents that reported BCI performance published in English in peer-reviewed journals between 2008 and May 2020 were included. Twelve publications were identified, providing strong evidence for continued research in pediatric BCIs. Research evidence was generally at multiple case study or exploratory study level, with modest sample sizes. Seven studies focused on BCIs for communication and five on mobility. Articles were categorized and grouped based on type of measurement (i.e., non-invasive and invasive), and the type of brain signal (i.e., sensory evoked potentials or movement-related potentials). Strengths and limitations of studies were identified and used to provide requirements for clinical translation of pediatric BCIs. This systematic review presents the state-of-the-art of pediatric BCIs focused on developing advanced technology to support children and youth with communication disabilities or limited manual ability. Despite a few research studies addressing the application of BCIs for communication and mobility in children, results are encouraging and future works should focus on customizable pediatric access technologies based on brain activity.

A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces

The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.

30+ years of P300 brain-computer interfaces

Brain-computer interfaces (BCIs) directly measure brain activity with no physical movement and translate the neural signals into messages. BCIs that employ the P300 event-related brain potential often have used the visual modality. The end user is presented with flashing stimuli that indicate selections for communication, control, or both. Counting each flash that corresponds to a specific target selection while ignoring other flashes will elicit P300s to only the target selection. P300 BCIs also have been implemented using auditory or tactile stimuli. P300 BCIs have been used with a variety of applications for severely disabled end users in their homes without frequent expert support. P300 BCI research and development has made substantial progress, but challenges remain before these tools can become practical devices for impaired patients and perhaps healthy people.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Thought-based interaction: Same data, same methods, different results?

Restoration of communication in people with complete motor paralysis—a condition called complete locked-in state (CLIS)—is one of the greatest challenges of brain-computer interface (BCI) research. New findings have recently been presented that bring us one step closer to this goal. However, the validity of the evidence has been questioned: independent reanalysis of the same data yielded significantly different results. Reasons for the failure to replicate the findings must be of a methodological nature. What is the best practice to ensure that results are stringent and conclusive and analyses replicable? Confirmation bias and the counterintuitive nature of probability may lead to an overly optimistic interpretation of new evidence. Lack of detail complicates replicability.

This Primer explores a recent debate about brain-computer interface studies, observing that confirmation bias and the counter-intuitive nature of probability may lead to an overly optimistic interpretation of scientific results; furthermore, a lack of details complicates replicability of analyses and experiments.

Closed-Loop Continuous Hand Control via Chronic Recording of Regenerative Peripheral Nerve Interfaces

Loss of the upper limb imposes a devastating interruption to everyday life. Full restoration of natural arm control requires the ability to simultaneously control multiple degrees of freedom of the prosthetic arm and maintain that control over an extended period of time. Current clinically available myoelectric prostheses do not provide simultaneous control or consistency for transradial amputees. To address this issue, we have implemented a standard Kalman filter for continuous hand control using intramuscular electromyography (EMG) from both regenerative peripheral nerve interfaces (RPNI) and an intact muscle within non-human primates. Seven RPNIs and one intact muscle were implanted with indwelling bipolar intramuscular electrodes in two rhesus macaques. Following recuperations, function-specific EMG signals were recorded and then fed through the Kalman filter during a hand-movement behavioral task to continuously predict the monkey's finger position. We were able to reconstruct continuous finger movement offline with an average correlation of and a root mean squared error (RMSE) of 0.12 between actual and predicted position from two macaques. This finger movement prediction was also performed in real time to enable closed-loop neural control of a virtual hand. Compared with physical hand control, neural control performance was slightly slower but maintained an average target hit success rate of 96.70%. Recalibration longevity measurements maintained consistent average correlation over time but had a significant change in RMSE ( ). Additionally, extracted single units varied in amplitude by a factor of +18.65% and -25.85% compared with its mean. This is the first demonstration of chronic indwelling electrodes being used for continuous position control via the Kalman filter. Combining these analyses with our novel peripheral nerve interface, we believe that this demonstrates an important step in providing patients with more naturalistic control of their prosthetic limbs.

An Online Interactive Paradigm for P300 Brain-Computer Interface Speller

For each brain-computer interface system, efficiency is a key issue that considers both accuracy and speed. The P300 spellers built upon oddball paradigm are usually less efficient due to the repetitive stimulation of multiple characters for reliable detection. In this paper, based on the online EEG signal, we propose an interactive paradigm for P300 speller to improve its efficiency, primarily focusing within the single characterP300 paradigm. Specifically, after each stimulation, we first evaluate the posterior probability of each character in the stimuli set to be the target. The lowprobability characters are then removed fromthe stimuli set in the subsequent round(s), as character flash continues until the probability of any character surpasses a predefined threshold. Then, the character is selected as the target and data collection for the trial terminates. By reducing stimulus sequence characters, the system efficiency can be substantially improved. The spelling accuracy is insignificantly affected as the characters being removed have low probability to be the target. The online experimental results from a total of eight subjects show that an average practical information transfer rate of 50.26 bits/min (i.e. 9.07 characters/min) has been achieved, with 91% average spelling accuracy rate.

Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter

Objective: To date, many brain-machine interface (BMI) studies have developed decoding algorithms for neuroprostheses that provide users with precise control of upper arm reaches with some limited grasping capabilities. However, comparatively few have focused on quantifying the performance of precise finger control. Here we expand upon this work by investigating online control of individual finger groups.

Approach: We have developed a novel training manipulandum for non-human primate (NHP) studies to isolate the movements of two specific finger groups: index and middle-ring-pinkie (MRP) fingers. We use this device in combination with the ReFIT (Recalibrated Feedback Intention-Trained) Kalman filter to decode the position of each finger group during a single degree of freedom task in two rhesus macaques with Utah arrays in motor cortex. The ReFIT Kalman filter uses a two-stage training approach that improves online control of upper arm tasks with substantial reductions in orbiting time, thus making it a logical first choice for precise finger control.

Results: Both animals were able to reliably acquire fingertip targets with both index and MRP fingers, which they did in blocks of finger group specific trials. Decoding from motor signals online, the ReFIT Kalman filter reliably outperformed the standard Kalman filter, measured by bit rate, across all tested finger groups and movements by 31.0 and 35.2%. These decoders were robust when the manipulandum was removed during online control. While index finger movements and middle-ring-pinkie finger movements could be differentiated from each other with 81.7% accuracy across both subjects, the linear Kalman filter was not sufficient for decoding both finger groups together due to significant unwanted movement in the stationary finger, potentially due to co-contraction.

Significance: To our knowledge, this is the first systematic and biomimetic separation of digits for continuous online decoding in a NHP as well as the first demonstration of the ReFIT Kalman filter improving the performance of precise finger decoding. These results suggest that novel nonlinear approaches, apparently not necessary for center out reaches or gross hand motions, may be necessary to achieve independent and precise control of individual fingers.

A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent

Laboratory demonstrations of brain-computer interface (BCI) systems show promise for reducing disability associated with paralysis by directly linking neural activity to the control of assistive devices. Surveys of potential users have revealed several key BCI performance criteria for clinical translation of such a system. Of these criteria, high accuracy, short response latencies, and multi-functionality are three key characteristics directly impacted by the neural decoding component of the BCI system, the algorithm that translates neural activity into control signals. Building a decoder that simultaneously addresses these three criteria is complicated because optimizing for one criterion may lead to undesirable changes in the other criteria. Unfortunately, there has been little work to date to quantify how decoder design simultaneously affects these performance characteristics. Here, we systematically explore the trade-off between accuracy, response latency, and multi-functionality for discrete movement classification using two different decoding strategies-a support vector machine (SVM) classifier which represents the current state-of-the-art for discrete movement classification in laboratory demonstrations and a proposed deep neural network (DNN) framework. We utilized historical intracortical recordings from a human tetraplegic study participant, who imagined performing several different hand and finger movements. For both decoders, we found that response time increases (i.e., slower reaction) and accuracy decreases as the number of functions increases. However, we also found that both the increase of response times and the decline in accuracy with additional functions is less for the DNN than the SVM. We also show that data preprocessing steps can affect the performance characteristics of the two decoders in drastically different ways. Finally, we evaluated the performance of our tetraplegic participant using the DNN decoder in real-time to control functional electrical stimulation (FES) of his paralyzed forearm. We compared his performance to that of able-bodied participants performing the same task, establishing a quantitative target for ideal BCI-FES performance on this task. Cumulatively, these results help quantify BCI decoder performance characteristics relevant to potential users and the complex interactions between them.

Online classification of imagined speech using functional near-infrared spectroscopy signals

OBJECTIVE

Most brain-computer interfaces (BCIs) based on functional near-infrared spectroscopy (fNIRS) require that users perform mental tasks such as motor imagery, mental arithmetic, or music imagery to convey a message or to answer simple yes or no questions. These cognitive tasks usually have no direct association with the communicative intent, which makes them difficult for users to perform.

APPROACH

In this paper, a 3-class intuitive BCI is presented which enables users to directly answer yes or no questions by covertly rehearsing the word 'yes' or 'no' for 15 s. The BCI also admits an equivalent duration of unconstrained rest which constitutes the third discernable task. Twelve participants each completed one offline block and six online blocks over the course of two sessions. The mean value of the change in oxygenated hemoglobin concentration during a trial was calculated for each channel and used to train a regularized linear discriminant analysis (RLDA) classifier.

MAIN RESULTS

By the final online block, nine out of 12 participants were performing above chance (p  <  0.001 using the binomial cumulative distribution), with a 3-class accuracy of 83.8%  ±  9.4%. Even when considering all participants, the average online 3-class accuracy over the last three blocks was 64.1 %  ±  20.6%, with only three participants scoring below chance (p  <  0.001). For most participants, channels in the left temporal and temporoparietal cortex provided the most discriminative information.

SIGNIFICANCE

To our knowledge, this is the first report of an online 3-class imagined speech BCI. Our findings suggest that imagined speech can be used as a reliable activation task for selected users for development of more intuitive BCIs for communication.

Defining and quantifying users' mental imagery-based BCI skills: a first step

OBJECTIVE

While promising for many applications, electroencephalography (EEG)-based brain-computer interfaces (BCIs) are still scarcely used outside laboratories, due to a poor reliability. It is thus necessary to study and fix this reliability issue. Doing so requires the use of appropriate reliability metrics to quantify both the classification algorithm and the BCI user's performances. So far, classification accuracy (CA) is the typical metric used for both aspects. However, we argue in this paper that CA is a poor metric to study BCI users' skills. Here, we propose a definition and new metrics to quantify such BCI skills for mental imagery (MI) BCIs, independently of any classification algorithm.

APPROACH

We first show in this paper that CA is notably unspecific, discrete, training data and classifier dependent, and as such may not always reflect successful self-modulation of EEG patterns by the user. We then propose a definition of MI-BCI skills that reflects how well the user can self-modulate EEG patterns, and thus how well he could control an MI-BCI. Finally, we propose new performance metrics, classDis, restDist and classStab that specifically measure how distinct and stable the EEG patterns produced by the user are, independently of any classifier.

MAIN RESULTS

By re-analyzing EEG data sets with such new metrics, we indeed confirmed that CA may hide some increase in MI-BCI skills or hide the user inability to self-modulate a given EEG pattern. On the other hand, our new metrics could reveal such skill improvements as well as identify when a mental task performed by a user was no different than rest EEG.

SIGNIFICANCE

Our results showed that when studying MI-BCI users' skills, CA should be used with care, and complemented with metrics such as the new ones proposed. Our results also stressed the need to redefine BCI user training by considering the different BCI subskills and their measures. To promote the complementary use of our new metrics, we provide the Matlab code to compute them for free and open-source.

Neural control of finger movement via intracortical brain-machine interface

OBJECTIVE

Intracortical brain-machine interfaces (BMIs) are a promising source of prosthesis control signals for individuals with severe motor disabilities. Previous BMI studies have primarily focused on predicting and controlling whole-arm movements; precise control of hand kinematics, however, has not been fully demonstrated. Here, we investigate the continuous decoding of precise finger movements in rhesus macaques.

APPROACH

In order to elicit precise and repeatable finger movements, we have developed a novel behavioral task paradigm which requires the subject to acquire virtual fingertip position targets. In the physical control condition, four rhesus macaques performed this task by moving all four fingers together in order to acquire a single target. This movement was equivalent to controlling the aperture of a power grasp. During this task performance, we recorded neural spikes from intracortical electrode arrays in primary motor cortex.

MAIN RESULTS

Using a standard Kalman filter, we could reconstruct continuous finger movement offline with an average correlation of ρ  =  0.78 between actual and predicted position across four rhesus macaques. For two of the monkeys, this movement prediction was performed in real-time to enable direct brain control of the virtual hand. Compared to physical control, neural control performance was slightly degraded; however, the monkeys were still able to successfully perform the task with an average target acquisition rate of 83.1%. The monkeys' ability to arbitrarily specify fingertip position was also quantified using an information throughput metric. During brain control task performance, the monkeys achieved an average 1.01 bits s-1 throughput, similar to that achieved in previous studies which decoded upper-arm movements to control computer cursors using a standard Kalman filter.

SIGNIFICANCE

This is, to our knowledge, the first demonstration of brain control of finger-level fine motor skills. We believe that these results represent an important step towards full and dexterous control of neural prosthetic devices.

Development of a Ternary Near-Infrared Spectroscopy Brain-Computer Interface: Online Classification of Verbal Fluency Task, Stroop Task and Rest

The majority of proposed NIRS-BCIs has considered binary classification. Studies considering high-order classification problems have yielded average accuracies that are less than favorable for practical communication. Consequently, there is a paucity of evidence supporting online classification of more than two mental states using NIRS. We developed an online ternary NIRS-BCI that supports the verbal fluency task (VFT), Stroop task and rest. The system utilized two sessions dedicated solely to classifier training. Additionally, samples were collected prior to each period of online classification to update the classifier. Using a continuous-wave spectrometer, measurements were collected from the prefrontal and parietal cortices while 11 able-bodied adult participants were cued to perform one of the two cognitive tasks or rests. Each task was used to indicate the desire to select a particular letter on a scanning interface, while rest avoided selection. Classification was performed using 25 iteration of bagging with a linear discriminant base classifier. Classifiers were trained on 10-dimensional feature sets. The BCI's classification decision was provided as feedback. An average online classification accuracy of [Formula: see text]% was achieved, representing an ITR of [Formula: see text] bits/min. The results demonstrate that online communication can be achieved with a ternary NIRS-BCI that supports VFT, Stroop task and rest. Our findings encourage continued efforts to enhance the ITR of NIRS-BCIs.

Review: Human Intracortical Recording and Neural Decoding for Brain-Computer Interfaces

Brain-computer interfaces (BCIs) use neural information recorded from the brain for the voluntary control of external devices. The development of BCI systems has largely focused on improving functional independence for individuals with severe motor impairments, including providing tools for communication and mobility. In this review, we describe recent advances in intracortical BCI technology and provide potential directions for further research.

Enhancing P300-BCI performance using latency estimation

Brain Computer Interfaces (BCIs) offer restoration of communication to those with the most severe movement impairments, but performance is not yet ideal. Previous work has demonstrated that latency jitter, the variation in timing of the brain responses, plays a critical role in determining BCI performance. In this study, we used Classifier-Based Latency Estimation (CBLE) and a wavelet transform to provide information about latency jitter to a second-level classifier. Three second-level classifiers were tested: least squares (LS), step-wise linear discriminant analysis (SWLDA), and support vector machine (SVM). Of these three, LS and SWLDA performed better than the original online classifier. The resulting combination demonstrated improved detection of brain responses for many participants, resulting in better BCI performance. Interestingly, the performance gain was greatest for those individuals for whom the BCI did not work well online, indicating that this method may be most suitable for improving performance of otherwise marginal participants.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Objective

Using traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.

Method

We introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.

Results

Theoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.

Significance

The continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

Support vector machines to detect physiological patterns for EEG and EMG-based human-computer interaction: a review

Support vector machines (SVMs) are widely used classifiers for detecting physiological patterns in human-computer interaction (HCI). Their success is due to their versatility, robustness and large availability of free dedicated toolboxes. Frequently in the literature, insufficient details about the SVM implementation and/or parameters selection are reported, making it impossible to reproduce study analysis and results. In order to perform an optimized classification and report a proper description of the results, it is necessary to have a comprehensive critical overview of the applications of SVM. The aim of this paper is to provide a review of the usage of SVM in the determination of brain and muscle patterns for HCI, by focusing on electroencephalography (EEG) and electromyography (EMG) techniques. In particular, an overview of the basic principles of SVM theory is outlined, together with a description of several relevant literature implementations. Furthermore, details concerning reviewed papers are listed in tables and statistics of SVM use in the literature are presented. Suitability of SVM for HCI is discussed and critical comparisons with other classifiers are reported.

Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface

Brain-computer interfaces (BCIs) promise to restore independence for people with severe motor disabilities by translating decoded neural activity directly into the control of a computer. However, recorded neural signals are not stationary (that is, can change over time), degrading the quality of decoding. Requiring users to pause what they are doing whenever signals change to perform decoder recalibration routines is time-consuming and impractical for everyday use of BCIs. We demonstrate that signal nonstationarity in an intracortical BCI can be mitigated automatically in software, enabling long periods (hours to days) of self-paced point-and-click typing by people with tetraplegia, without degradation in neural control. Three key innovations were included in our approach: tracking the statistics of the neural activity during self-timed pauses in neural control, velocity bias correction during neural control, and periodically recalibrating the decoder using data acquired during typing by mapping neural activity to movement intentions that are inferred retrospectively based on the user's self-selected targets. These methods, which can be extended to a variety of neurally controlled applications, advance the potential for intracortical BCIs to help restore independent communication and assistive device control for people with paralysis.

Empirical Movement Models for Brain Computer Interfaces

For brain-computer interfaces (BCIs) which provide the user continuous position control, there is little standardization of performance metrics or evaluative tasks. One candidate metric is Fitts's law, which has been used to describe aimed movements across a range of computer interfaces, and has recently been applied to BCI tasks. Reviewing selected studies, we identify two basic problems with Fitts's law: its predictive performance is fragile, and the estimation of 'information transfer rate' from the model is unsupported. Our main contribution is the adaptation and validation of an alternative model to Fitts's law in the BCI context. We show that the Shannon-Welford model outperforms Fitts's law, showing robust predictive power when target distance and width have disproportionate effects on difficulty. Building on a prior study of the Shannon-Welford model, we show that identified model parameters offer a novel approach to quantitatively assess the role of control-display gain in speed/accuracy performance tradeoffs during brain control.

Autonomous Parameter Adjustment for SSVEP-Based BCIs with a Novel BCI Wizard

Brain-Computer Interfaces (BCIs) transfer human brain activities into computer commands and enable a communication channel without requiring movement. Among other BCI approaches, steady-state visual evoked potential (SSVEP)-based BCIs have the potential to become accurate, assistive technologies for persons with severe disabilities. Those systems require customization of different kinds of parameters (e.g., stimulation frequencies). Calibration usually requires selecting predefined parameters by experienced/trained personnel, though in real-life scenarios an interface allowing people with no experience in programming to set up the BCI would be desirable. Another occurring problem regarding BCI performance is BCI illiteracy (also called BCI deficiency). Many articles reported that BCI control could not be achieved by a non-negligible number of users. In order to bypass those problems we developed a SSVEP-BCI wizard, a system that automatically determines user-dependent key-parameters to customize SSVEP-based BCI systems. This wizard was tested and evaluated with 61 healthy subjects. All subjects were asked to spell the phrase "RHINE WAAL UNIVERSITY" with a spelling application after key parameters were determined by the wizard. Results show that all subjects were able to control the spelling application. A mean (SD) accuracy of 97.14 (3.73)% was reached (all subjects reached an accuracy above 85% and 25 subjects even reached 100% accuracy).

Towards a ternary NIRS-BCI: single-trial classification of verbal fluency task, Stroop task and unconstrained rest

OBJECTIVE

The majority of near-infrared spectroscopy (NIRS) brain-computer interface (BCI) studies have investigated binary classification problems. Limited work has considered differentiation of more than two mental states, or multi-class differentiation of higher-level cognitive tasks using measurements outside of the anterior prefrontal cortex. Improvements in accuracies are needed to deliver effective communication with a multi-class NIRS system. We investigated the feasibility of a ternary NIRS-BCI that supports mental states corresponding to verbal fluency task (VFT) performance, Stroop task performance, and unconstrained rest using prefrontal and parietal measurements.

APPROACH

Prefrontal and parietal NIRS signals were acquired from 11 able-bodied adults during rest and performance of the VFT or Stroop task. Classification was performed offline using bagging with a linear discriminant base classifier trained on a 10 dimensional feature set.

MAIN RESULTS

VFT, Stroop task and rest were classified at an average accuracy of 71.7% ± 7.9%. The ternary classification system provided a statistically significant improvement in information transfer rate relative to a binary system controlled by either mental task (0.87 ± 0.35 bits/min versus 0.73 ± 0.24 bits/min).

SIGNIFICANCE

These results suggest that effective communication can be achieved with a ternary NIRS-BCI that supports VFT, Stroop task and rest via measurements from the frontal and parietal cortices. Further development of such a system is warranted. Accurate ternary classification can enhance communication rates offered by NIRS-BCIs, improving the practicality of this technology.

Motor imagery based brain-computer interfaces: An emerging technology to rehabilitate motor deficits

When the sensory-motor integration system is malfunctioning provokes a wide variety of neurological disorders, which in many cases cannot be treated with conventional medication, or via existing therapeutic technology. A brain-computer interface (BCI) is a tool that permits to reintegrate the sensory-motor loop, accessing directly to brain information. A potential, promising and quite investigated application of BCI has been in the motor rehabilitation field. It is well-known that motor deficits are the major disability wherewith the worldwide population lives. Therefore, this paper aims to specify the foundation of motor rehabilitation BCIs, as well as to review the recent research conducted so far (specifically, from 2007 to date), in order to evaluate the suitability and reliability of this technology. Although BCI for post-stroke rehabilitation is still in its infancy, the tendency is towards the development of implantable devices that encompass a BCI module plus a stimulation system.

Initial constructs for patient-centered outcome measures to evaluate brain-computer interfaces

PURPOSE

The authors describe preliminary work toward the creation of patient-centered outcome (PCO) measures to evaluate brain-computer interface (BCI) as an assistive technology (AT) for individuals with severe speech and physical impairments (SSPI).

METHOD

In Phase 1, 591 items from 15 existing measures were mapped to the International Classification of Functioning, Disability and Health (ICF). In Phase 2, qualitative interviews were conducted with eight people with SSPI and seven caregivers. Resulting text data were coded in an iterative analysis.

RESULTS

Most items (79%) were mapped to the ICF environmental domain; over half (53%) were mapped to more than one domain. The ICF framework was well suited for mapping items related to body functions and structures, but less so for items in other areas, including personal factors. Two constructs emerged from qualitative data: quality of life (QOL) and AT. Component domains and themes were identified for each.

CONCLUSIONS

Preliminary constructs, domains and themes were generated for future PCO measures relevant to BCI. Existing instruments are sufficient for initial items but do not adequately match the values of people with SSPI and their caregivers. Field methods for interviewing people with SSPI were successful, and support the inclusion of these individuals in PCO research. Implications for Rehabilitation Adapted interview methods allow people with severe speech and physical impairments to participate in patient-centered outcomes research. Patient-centered outcome measures are needed to evaluate the clinical implementation of brain-computer interface as an assistive technology.