Bioinspired Stretchable Fiber-Based Sensor toward Intelligent Human-Machine Interactions

Wearable integrated sensing devices with flexible electronic elements exhibit enormous potential in human-machine interfaces (HMI), but they have limitations such as complex structures, poor waterproofness, and electromagnetic interference. Herein, inspired by the profile of Lindernia nummularifolia (LN), a bionic stretchable optical strain (BSOS) sensor composed of an LN-shaped optical fiber incorporated with a stretchable substrate is developed for intelligent HMI. Such a sensor enables large strain and bending angle measurements with temperature self-compensation by the intensity difference of two fiber Bragg gratings' (FBGs') center wavelength. Such configurations enable an excellent tensile strain range of up to 80%, moreover, leading to ultrasensitivity, durability (≥20,000 cycles), and waterproofness. The sensor is also capable of measuring different human activities and achieving HMI control, including immersive virtual reality, robot remote interactive control, and personal hands-free communication. Combined with the machine learning technique, gesture classification can be achieved using muscle activity signals captured from the BSOS sensor, which can be employed to obtain the motion intention of the prosthetic. These merits effectively indicate its potential as a solution for medical care HMI and show promise in smart medical and rehabilitation medicine.

Usability, occupational performance and satisfaction evaluation of a smart environment controlled by infrared oculography by people with severe motor disabilities

A smart environment is an assistive technology space that can enable people with motor disabilities to control their equipment (TV, radio, fan, etc.) through a human-machine interface activated by different inputs. However, assistive technology resources are not always considered useful, reaching quite high abandonment rate. This study aims to evaluate the effectiveness of a smart environment controlled through infrared oculography by people with severe motor disabilities. The study sample was composed of six individuals with motor disabilities. Initially, sociodemographic data forms, the Functional Independence Measure (FIM^TM), and the Canadian Occupational Performance Measure (COPM) were applied. The participants used the system in their domestic environment for a week. Afterwards, they were reevaluated with regards to occupational performance (COPM), satisfaction with the use of the assistive technology resource (QUEST 2.0), psychosocial impact (PIADS) and usability of the system (SUS), as well as through semi-structured interviews for suggestions or complaints. The most common demand from the participants of this research was ‘control of the TV’. Two participants did not use the system. All participants who used the system (four) presented positive results in all assessment protocols, evidencing greater independence in the control of the smart environment equipment. In addition, they evaluated the system as useful and with good usability. Non-acceptance of disability and lack of social support may have influenced the results.

Effects of brain-computer interface training on upper limb function recovery in stroke patients: A protocol for systematic review and meta-analysis

BACKGROUND

In recent years, with the development of medical technology and the increase of inter-disciplinary cooperation technology, new methods in the field of artificial intelligence medicine emerge in an endless stream. Brain-computer interface (BCI), as a frontier technology of multidisciplinary integration, has been widely used in various fields. Studies have shown that BCI-assisted training can improve upper limb function in stroke patients, but its effect is still controversial and lacks evidence-based evidence, which requires further exploration and confirmation. Therefore, the main purpose of this paper is to systematically evaluate the efficacy of different BCI-assisted training on upper limb function recovery in stroke patients, to provide a reference for the application of BCI-assisted technology in stroke rehabilitation.

METHODS

We will search PubMed, Web of Science, The Cochrane Library, Chinese National Knowledge Infrastructure Database, Wanfang Data, Weipu Electronics, and other databases (from the establishment to February 2021) for full text in Chinese and English. Randomized controlled trials were collected to examine the effect of BCI-assisted training on upper limb functional recovery in stroke patients. We will consider inclusion, select high-quality articles for data extraction and analysis, and summarize the intervention effect of BCI-assisted training on the upper limb function of stroke patients. Two reviewers will screen titles, abstracts, and full texts independently according to inclusion criteria; Data extraction and risk of bias assessment were performed in the included studies. We will use a hierarchy of recommended assessment, development, and assessment methods to assess the overall certainty of the evidence and report findings accordingly. Endnote X8 will be applied in selecting the study, Review Manager 5.3 will be applied in analyzing and synthesizing.

RESULTS

The results will provide evidence for judging whether BCI is effective and safe in improving upper limb function in patients with stroke.

CONCLUSION

Our study will provide reliable evidence for the effect of BCI technology on the improvement of upper limb function in stroke patients.

PROSPERO REGISTRATION NUMBER

CRD42021250378.

An Experiential Learning-Based Approach to Neurofeedback Visualisation in Serious Games

This study explores brain-computer interfacing, its possible use in serious or educational games and frameworks. Providing real-time feedback regarding cognitive states and behaviours can be a powerful tool for mental health education and games can offer unique and engaging environments for these neurofeedback experiences. We explore how EEG neurofeedback systems can be affordably created for further research and experimentation and suggest design choices that may assist in developing effective experiences of this nature.

Independent home use of a brain-computer interface by people with amyotrophic lateral sclerosis

OBJECTIVE

To assess the reliability and usefulness of an EEG-based brain-computer interface (BCI) for patients with advanced amyotrophic lateral sclerosis (ALS) who used it independently at home for up to 18 months.

METHODS

Of 42 patients consented, 39 (93%) met the study criteria, and 37 (88%) were assessed for use of the Wadsworth BCI. Nine (21%) could not use the BCI. Of the other 28, 27 (men, age 28-79 years) (64%) had the BCI placed in their homes, and they and their caregivers were trained to use it. Use data were collected by Internet. Periodic visits evaluated BCI benefit and burden and quality of life.

RESULTS

Over subsequent months, 12 (29% of the original 42) left the study because of death or rapid disease progression and 6 (14%) left because of decreased interest. Fourteen (33%) completed training and used the BCI independently, mainly for communication. Technical problems were rare. Patient and caregiver ratings indicated that BCI benefit exceeded burden. Quality of life remained stable. Of those not lost to the disease, half completed the study; all but 1 patient kept the BCI for further use.

CONCLUSION

The Wadsworth BCI home system can function reliably and usefully when operated by patients in their homes. BCIs that support communication are at present most suitable for people who are severely disabled but are otherwise in stable health. Improvements in BCI convenience and performance, including some now underway, should increase the number of people who find them useful and the extent to which they are used.

Augmenting intracortical brain-machine interface with neurally driven error detectors

OBJECTIVE

Making mistakes is inevitable, but identifying them allows us to correct or adapt our behavior to improve future performance. Current brain-machine interfaces (BMIs) make errors that need to be explicitly corrected by the user, thereby consuming time and thus hindering performance. We hypothesized that neural correlates of the user perceiving the mistake could be used by the BMI to automatically correct errors. However, it was unknown whether intracortical outcome error signals were present in the premotor and primary motor cortices, brain regions successfully used for intracortical BMIs.

APPROACH

We report here for the first time a putative outcome error signal in spiking activity within these cortices when rhesus macaques performed an intracortical BMI computer cursor task.

MAIN RESULTS

We decoded BMI trial outcomes shortly after and even before a trial ended with 96% and 84% accuracy, respectively. This led us to develop and implement in real-time a first-of-its-kind intracortical BMI error 'detect-and-act' system that attempts to automatically 'undo' or 'prevent' mistakes. The detect-and-act system works independently and in parallel to a kinematic BMI decoder. In a challenging task that resulted in substantial errors, this approach improved the performance of a BMI employing two variants of the ubiquitous Kalman velocity filter, including a state-of-the-art decoder (ReFIT-KF).

SIGNIFICANCE

Detecting errors in real-time from the same brain regions that are commonly used to control BMIs should improve the clinical viability of BMIs aimed at restoring motor function to people with paralysis.

Evaluating brain-computer interface performance using color in the P300 checkerboard speller

OBJECTIVE

Current Brain-Computer Interface (BCI) systems typically flash an array of items from grey to white (GW). The objective of this study was to evaluate BCI performance using uniquely colored stimuli.

METHODS

In addition to the GW stimuli, the current study tested two types of color stimuli (grey to color [GC] and color intensification [CI]). The main hypotheses were that in a checkboard paradigm, unique color stimuli will: (1) increase BCI performance over the standard GW paradigm; (2) elicit larger event-related potentials (ERPs); and, (3) improve offline performance with an electrode selection algorithm (i.e., Jumpwise).

RESULTS

Online results (n=36) showed that GC provides higher accuracy and information transfer rate than the CI and GW conditions. Waveform analysis showed that GC produced higher amplitude ERPs than CI and GW. Information transfer rate was improved by the Jumpwise-selected channel locations in all conditions.

CONCLUSIONS

Unique color stimuli (GC) improved BCI performance and enhanced ERPs. Jumpwise-selected electrode locations improved offline performance.

SIGNIFICANCE

These results show that in a checkerboard paradigm, unique color stimuli increase BCI performance, are preferred by participants, and are important to the design of end-user applications; thus, could lead to an increase in end-user performance and acceptance of BCI technology.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Objective

Using traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.

Method

We introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.

Results

Theoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.

Significance

The continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain-computer interfaces

Brain-computer interfaces (BCIs) aim to restore independence to people with severe motor disabilities by allowing control of acursor on a computer screen or other effectors with neural activity. However, physiological and/or recording-related nonstationarities in neural signals can limit long-term decoding stability, and it would be tedious for users to pause use of the BCI whenever neural control degrades to perform decoder recalibration routines. We recently demonstrated that a kinematic decoder (i.e. a decoder that controls cursor movement) can be recalibrated using data acquired during practical point-and-click control of the BCI by retrospectively inferring users' intended movement directions based on their subsequent selections. Here, we extend these methods to allow the click decoder to also be recalibrated using data acquired during practical BCI use. We retrospectively labeled neural data patterns as corresponding to "click" during all time bins in which the click log-likelihood (decoded using linear discriminant analysis, or LDA) had been above the click threshold that was used during real-time neural control. We labeled as "non-click" those periods that the kinematic decoder's retrospective target inference (RTI) heuristics determined to be consistent with intended cursor movement. Once these neural activity patterns were labeled, the click decoder was calibrated using standard supervised classifier training methods. Combined with real-time bias correction and baseline firing rate tracking, this set of "retrospectively labeled" decoder calibration methods enabled a BrainGate participant with amyotrophic lateral sclerosis (T9) to type freely across 11 research sessions spanning 29days, maintaining high-performance neural control over cursor movement and click without needing to interrupt virtual keyboard use for explicit calibration tasks. By eliminating the need for tedious calibration tasks with prescribed targets and pre-specified click times, this approach advances the potential clinical utility of intracortical BCIs for individuals with severe motor disability.

VEP-based brain-computer interfaces modulated by Golay complementary series for improving performance

BACKGROUND

The goal of a brain-computer interface (BCI) is to enable communication by pure brain activity without neural and muscle control. However, the practical use of BCIs is limited by low information transfer rate. Recently, code modulation visual evoked potential (c-VEP) based BCIs have exhibited great potential in establishing high-rate communication between the brain and the external world.

OBJECTIVE

This study aims at exploring a more effective modulation code than the commonly used pseudorandom M sequence for c-VEP based BCIs (c-VEP BCIs) in order to increase the detection accuracy of stimulus targets and the resulting information transfer rate.

METHOD

Golay complementary sequence pair is used for constructing the modulation code of c-VEP BCIs due to their superior autocorrelation property. The modulation code is created by concatenating a pair of Golay complementary sequences. Sixteen target stimuli are modulated by the Golay code and its time shift versions.

RESULTS

Through offline analysis on data recorded from seven subjects and online test on five subjects, the Golay code modulated BCI yielded higher detection accuracy and information transfer rate than those achieved by M sequence.

CONCLUSION

The Golay code modulated BCI demonstrates a high performance compared with the M sequence modulated systems, and it is applicable to persons with motor disabilities.

Concentration on performance with P300-based BCI systems: a matter of interface features

People who suffer from severe motor disabilities have difficulties to communicate with others or to interact with their environment using natural, i.e., muscular channels. These limitations can be overcome to some extent by using brain-computer interfaces (BCIs), because such systems allow users to communicate on the basis of their brain activity only. Among the several types of BCIs for spelling purposes, those that rely on the P300 event related potential-P300-based spellers-are chosen preferentially due to their high reliability. However, they demand from the user to sustain his/her attention to the desired character over a relatively long period of time. Therefore, the user's capacity to concentrate can affect his/her performance with a P300-based speller. The aim of this study was to test this hypothesis using three different interfaces: one based on the classic P300 speller paradigm, another also based on that speller but including a word predictor, and a third one that was based on the T9 interface developed for mobile phones. User performance was assessed by measuring the time to complete a spelling task and the accuracy of character selection. The d2 test was applied to assess attention and concentration. Sample (N = 14) was divided into two groups basing on of concentration scores. As a result, performance was better with the predictor-enriched interfaces: less time was needed to solve the task and participants made fewer errors (p < .05). There were also significant effects of concentration (p < .05) on performance with the standard P300 speller. In conclusion, the performance of those users with lower concentration level can be improved by providing BCIs with more interactive interfaces. These findings provide substantial evidence in order to highlight the impact of psychological features on BCI performance and should be taken into account for future assistive technology systems.

Developing brain-computer interfaces from a user-centered perspective: Assessing the needs of persons with amyotrophic lateral sclerosis, caregivers, and professionals

By focus group methodology, we examined the opinions and requirements of persons with ALS, their caregivers, and health care assistants with regard to developing a brain-computer interface (BCI) system that fulfills the user's needs. Four overarching topics emerged from this analysis: 1) lack of information on BCI and its everyday applications; 2) importance of a customizable system that supports individuals throughout the various stages of the disease; 3) relationship between affectivity and technology use; and 4) importance of individuals retaining a sense of agency. These findings should be considered when developing new assistive technology. Moreover, the BCI community should acknowledge the need to bridge experimental results and its everyday application.

Exploiting Task Constraints for Self-Calibrated Brain-Machine Interface Control Using Error-Related Potentials

This paper presents a new approach for self-calibration BCI for reaching tasks using error-related potentials. The proposed method exploits task constraints to simultaneously calibrate the decoder and control the device, by using a robust likelihood function and an ad-hoc planner to cope with the large uncertainty resulting from the unknown task and decoder. The method has been evaluated in closed-loop online experiments with 8 users using a previously proposed BCI protocol for reaching tasks over a grid. The results show that it is possible to have a usable BCI control from the beginning of the experiment without any prior calibration. Furthermore, comparisons with simulations and previous results obtained using standard calibration hint that both the quality of recorded signals and the performance of the system were comparable to those obtained with a standard calibration approach.

A general method for assessing brain-computer interface performance and its limitations

OBJECTIVE

When researchers evaluate brain-computer interface (BCI) systems, we want quantitative answers to questions such as: How good is the system's performance? How good does it need to be? and: Is it capable of reaching the desired level in future? In response to the current lack of objective, quantitative, study-independent approaches, we introduce methods that help to address such questions. We identified three challenges: (I) the need for efficient measurement techniques that adapt rapidly and reliably to capture a wide range of performance levels; (II) the need to express results in a way that allows comparison between similar but non-identical tasks; (III) the need to measure the extent to which certain components of a BCI system (e.g. the signal processing pipeline) not only support BCI performance, but also potentially restrict the maximum level it can reach.

APPROACH

For challenge (I), we developed an automatic staircase method that adjusted task difficulty adaptively along a single abstract axis. For challenge (II), we used the rate of information gain between two Bernoulli distributions: one reflecting the observed success rate, the other reflecting chance performance estimated by a matched random-walk method. This measure includes Wolpaw's information transfer rate as a special case, but addresses the latter's limitations including its restriction to item-selection tasks. To validate our approach and address challenge (III), we compared four healthy subjects' performance using an EEG-based BCI, a 'Direct Controller' (a high-performance hardware input device), and a 'Pseudo-BCI Controller' (the same input device, but with control signals processed by the BCI signal processing pipeline).

MAIN RESULTS

Our results confirm the repeatability and validity of our measures, and indicate that our BCI signal processing pipeline reduced attainable performance by about 33% (21 bits min(-1)).

SIGNIFICANCE

Our approach provides a flexible basis for evaluating BCI performance and its limitations, across a wide range of tasks and task difficulties.

Optimizing spatial filters by minimizing within-class dissimilarities in electroencephalogram-based brain-computer interface

A major challenge in electroencephalogram (EEG)-based brain-computer interfaces (BCIs) is the inherent nonstationarities in the EEG data. Variations of the signal properties from intra and inter sessions often lead to deteriorated BCI performances, as features extracted by methods such as common spatial patterns (CSP) are not invariant against the changes. To extract features that are robust and invariant, this paper proposes a novel spatial filtering algorithm called Kullback-Leibler (KL) CSP. The CSP algorithm only considers the discrimination between the means of the classes, but does not consider within-class scatters information. In contrast, the proposed KLCSP algorithm simultaneously maximizes the discrimination between the class means, and minimizes the within-class dissimilarities measured by a loss function based on the KL divergence. The performance of the proposed KLCSP algorithm is compared against two existing algorithms, CSP and stationary CSP (sCSP), using the publicly available BCI competition III dataset IVa and a large dataset from stroke patients performing neuro-rehabilitation. The results show that the proposed KLCSP algorithm significantly outperforms both the CSP and the sCSP algorithms, in terms of classification accuracy, by reducing within-class variations. This results in more compact and separable features.

Detection of error related neuronal responses recorded by electrocorticography in humans during continuous movements

Background

Brain-machine interfaces (BMIs) can translate the neuronal activity underlying a user’s movement intention into movements of an artificial effector. In spite of continuous improvements, errors in movement decoding are still a major problem of current BMI systems. If the difference between the decoded and intended movements becomes noticeable, it may lead to an execution error. Outcome errors, where subjects fail to reach a certain movement goal, are also present during online BMI operation. Detecting such errors can be beneficial for BMI operation: (i) errors can be corrected online after being detected and (ii) adaptive BMI decoding algorithm can be updated to make fewer errors in the future.

Methodology/Principal Findings

Here, we show that error events can be detected from human electrocorticography (ECoG) during a continuous task with high precision, given a temporal tolerance of 300–400 milliseconds. We quantified the error detection accuracy and showed that, using only a small subset of 2×2 ECoG electrodes, 82% of detection information for outcome error and 74% of detection information for execution error available from all ECoG electrodes could be retained.

Conclusions/Significance

The error detection method presented here could be used to correct errors made during BMI operation or to adapt a BMI algorithm to make fewer errors in the future. Furthermore, our results indicate that smaller ECoG implant could be used for error detection. Reducing the size of an ECoG electrode implant used for BMI decoding and error detection could significantly reduce the medical risk of implantation.