Continuous Tactile Feedback for Motor-Imagery Based Brain-Computer Interaction in a Multitasking Context

Influence of feedback transparency on motor imagery neurofeedback performance: the contribution of agency

Objective.Neurofeedback (NF) is a cognitive training procedure based on real-time feedback (FB) of a participant's brain activity that they must learn to self-regulate. A classical visual FB delivered in a NF task is a filling gauge reflecting a measure of brain activity. This abstract visual FB is not transparently linked-from the subject's perspective-to the task performed (e.g., motor imagery (MI)). This may decrease the sense of agency, that is, the participants' reported control over FB. Here, we assessed the influence of FB transparency on NF performance and the role of agency in this relationship.Approach.Participants performed a NF task using MI to regulate brain activity measured using electroencephalography. In separate blocks, participants experienced three different conditions designed to vary transparency: FB was presented as either (1) a swinging pendulum, (2) a clenching virtual hand, (3) a clenching virtual hand combined with a motor illusion induced by tendon vibration. We measured self-reported agency and user experience after each NF block.Main results. We found that FB transparency influences NF performance. Transparent visual FB provided by the virtual hand resulted in significantly better NF performance than the abstract FB of the pendulum. Surprisingly, adding a motor illusion to the virtual hand significantly decreased performance relative to the virtual hand alone. When introduced in incremental linear mixed effect models, self-reported agency was significantly associated with NF performance and it captured the variance related to the effect of FB transparency on NF performance.Significance. Our results highlight the relevance of transparent FB in relation to the sense of agency. This is likely an important consideration in designing FB to improve NF performance and learning outcomes.

Enhanced Motor Imagery Decoding by Calibration Model-Assisted With Tactile ERD

OBJECTIVE

In this study, we propose a tactile-assisted calibration method for a motor imagery (MI) based Brain-Computer Interface (BCI) system.

METHOD

In the proposed calibration, tactile stimulation was applied to the hand wrist to assist the subjects in the MI task, which is named SA-MI task. Then, classifier training in the SA-MI Calibration was performed using the SA-MI data, while the Conventional Calibration employed the MI data. After the classifiers were trained, the performance was evaluated on a common MI dataset.

RESULTS

Our study demonstrated that the SA-MI Calibration significantly improved the performance as compared with the Conventional Calibration, with a decoding accuracy of (78.3% vs. 71.3%). Moreover, the average calibration time could be reduced by 40%. This benefit of the SA-MI Calibration effect was further validated by an independent control group, which showed no improvement when tactile stimulation was not applied during the calibration phase. Further analysis showed that when compared with MI, greater motor-related cortical activation and higher R 2 value in the alpha-beta frequency band were induced in SA-MI.

CONCLUSION

Indeed, the SA-MI Calibration could significantly improve the performance and reduce the calibration time as compared with the Conventional Calibration.

SIGNIFICANCE

The proposed tactile stimulation-assisted MI Calibration method holds great potential for a faster and more accurate system setup at the beginning of BCI usage.

An improved model using convolutional sliding window-attention network for motor imagery EEG classification

Introduction

The classification model of motor imagery-based electroencephalogram (MI-EEG) is a new human-computer interface pattern and a new neural rehabilitation assessment method for diseases such as Parkinson's and stroke. However, existing MI-EEG models often suffer from insufficient richness of spatiotemporal feature extraction, learning ability, and dynamic selection ability.

Methods

To solve these problems, this work proposed a convolutional sliding window-attention network (CSANet) model composed of novel spatiotemporal convolution, sliding window, and two-stage attention blocks.

Results

The model outperformed existing state-of-the-art (SOTA) models in within- and between-individual classification tasks on commonly used MI-EEG datasets BCI-2a and Physionet MI-EEG, with classification accuracies improved by 4.22 and 2.02%, respectively.

Discussion

The experimental results also demonstrated that the proposed type token, sliding window, and local and global multi-head self-attention mechanisms can significantly improve the model's ability to construct, learn, and adaptively select multi-scale spatiotemporal features in MI-EEG signals, and accurately identify electroencephalogram signals in the unilateral motor area. This work provided a novel and accurate classification model for MI-EEG brain-computer interface tasks and proposed a feasible neural rehabilitation assessment scheme based on the model, which could promote the further development and application of MI-EEG methods in neural rehabilitation.

On closed-loop brain stimulation systems for improving the quality of life of patients with neurological disorders

Emerging brain technologies have significantly transformed human life in recent decades. For instance, the closed-loop brain-computer interface (BCI) is an advanced software-hardware system that interprets electrical signals from neurons, allowing communication with and control of the environment. The system then transmits these signals as controlled commands and provides feedback to the brain to execute specific tasks. This paper analyzes and presents the latest research on closed-loop BCI that utilizes electric/magnetic stimulation, optogenetic, and sonogenetic techniques. These techniques have demonstrated great potential in improving the quality of life for patients suffering from neurodegenerative or psychiatric diseases. We provide a comprehensive and systematic review of research on the modalities of closed-loop BCI in recent decades. To achieve this, the authors used a set of defined criteria to shortlist studies from well-known research databases into categories of brain stimulation techniques. These categories include deep brain stimulation, transcranial magnetic stimulation, transcranial direct-current stimulation, transcranial alternating-current stimulation, and optogenetics. These techniques have been useful in treating a wide range of disorders, such as Alzheimer's and Parkinson's disease, dementia, and depression. In total, 76 studies were shortlisted and analyzed to illustrate how closed-loop BCI can considerably improve, enhance, and restore specific brain functions. The analysis revealed that literature in the area has not adequately covered closed-loop BCI in the context of cognitive neural prosthetics and implanted neural devices. However, the authors demonstrate that the applications of closed-loop BCI are highly beneficial, and the technology is continually evolving to improve the lives of individuals with various ailments, including those with sensory-motor issues or cognitive deficiencies. By utilizing emerging techniques of stimulation, closed-loop BCI can safely improve patients' cognitive and affective skills, resulting in better healthcare outcomes.

Toward an Adapted Neurofeedback for Post-stroke Motor Rehabilitation: State of the Art and Perspectives

Stroke is a severe health issue, and motor recovery after stroke remains an important challenge in the rehabilitation field. Neurofeedback (NFB), as part of a brain–computer interface, is a technique for modulating brain activity using on-line feedback that has proved to be useful in motor rehabilitation for the chronic stroke population in addition to traditional therapies. Nevertheless, its use and applications in the field still leave unresolved questions. The brain pathophysiological mechanisms after stroke remain partly unknown, and the possibilities for intervention on these mechanisms to promote cerebral plasticity are limited in clinical practice. In NFB motor rehabilitation, the aim is to adapt the therapy to the patient’s clinical context using brain imaging, considering the time after stroke, the localization of brain lesions, and their clinical impact, while taking into account currently used biomarkers and technical limitations. These modern techniques also allow a better understanding of the physiopathology and neuroplasticity of the brain after stroke. We conducted a narrative literature review of studies using NFB for post-stroke motor rehabilitation. The main goal was to decompose all the elements that can be modified in NFB therapies, which can lead to their adaptation according to the patient’s context and according to the current technological limits. Adaptation and individualization of care could derive from this analysis to better meet the patients’ needs. We focused on and highlighted the various clinical and technological components considering the most recent experiments. The second goal was to propose general recommendations and enhance the limits and perspectives to improve our general knowledge in the field and allow clinical applications. We highlighted the multidisciplinary approach of this work by combining engineering abilities and medical experience. Engineering development is essential for the available technological tools and aims to increase neuroscience knowledge in the NFB topic. This technological development was born out of the real clinical need to provide complementary therapeutic solutions to a public health problem, considering the actual clinical context of the post-stroke patient and the practical limits resulting from it.

Embodiment Comfort Levels During Motor Imagery Training Combined With Immersive Virtual Reality in a Spinal Cord Injury Patient

Brain-machine interfaces combining visual, auditory, and tactile feedback have been previously used to generate embodiment experiences during spinal cord injury (SCI) rehabilitation. It is not known if adding temperature to these modalities can result in discomfort with embodiment experiences. Here, comfort levels with the embodiment experiences were investigated in an intervention that required a chronic pain SCI patient to generate lower limb motor imagery commands in an immersive environment combining visual (virtual reality -VR), auditory, tactile, and thermal feedback. Assessments were made pre-/ post-, throughout the intervention (Weeks 0-5), and at 7 weeks follow up. Overall, high levels of embodiment in the adapted three-domain scale of embodiment were found throughout the sessions. No significant adverse effects of VR were reported. Although sessions induced only a modest reduction in pain levels, an overall reduction occurred in all pain scales (Faces, Intensity, and Verbal) at follow up. A high degree of comfort in the comfort scale for the thermal-tactile sleeve, in both the thermal and tactile feedback components of the sleeve was reported. This study supports the feasibility of combining multimodal stimulation involving visual (VR), auditory, tactile, and thermal feedback to generate embodiment experiences in neurorehabilitation programs.

Influence of the visuo-proprioceptive illusion of movement and motor imagery of the wrist on EEG cortical excitability among healthy participants

INTRODUCTION

Motor Imagery (MI) is a powerful tool to stimulate sensorimotor brain areas and is currently used in motor rehabilitation after a stroke. The aim of our study was to evaluate whether an illusion of movement induced by visuo-proprioceptive immersion (VPI) including tendon vibration (TV) and Virtual moving hand (VR) combined with MI tasks could be more efficient than VPI alone or MI alone on cortical excitability assessed using Electroencephalography (EEG).

METHODS

We recorded EEG signals in 20 healthy participants in 3 different conditions: MI tasks involving their non-dominant wrist (MI condition); VPI condition; and VPI with MI tasks (combined condition). Each condition lasted 3 minutes, and was repeated 3 times in randomized order. Our main judgment criterion was the Event-Related De-synchronization (ERD) threshold in sensori-motor areas in each condition in the brain motor area.

RESULTS

The combined condition induced a greater change in the ERD percentage than the MI condition alone, but no significant difference was found between the combined and the VPI condition (p = 0.07) and between the VPI and MI condition (p = 0.20).

CONCLUSION

This study demonstrated the interest of using a visuo-proprioceptive immersion with MI rather than MI alone in order to increase excitability in motor areas of the brain. Further studies could test this hypothesis among patients with stroke to provide new perspectives for motor rehabilitation in this population.

A BCI-Based Vibrotactile Neurofeedback Training Improves Motor Cortical Excitability During Motor Imagery

In this study, we address the issue of whether vibrotactile feedback can enhance the motor cortex excitability translated into the plastic changes in local cortical areas during motor imagery (MI) BCI-based training. For this purpose, we focused on two of the most notable neurophysiological effects of MI - the event-related desynchronization (ERD) level and the increase in cortical excitability assessed with navigated transcranial magnetic stimulation (nTMS). For TMS navigation, we used individual high-resolution 3D brain MRIs. Ten BCI-naive and healthy adults participated in this study. The MI (rest or left/right hand imagery using Graz-BCI paradigm) tasks were performed separately in the presence and absence of feedback. To investigate how much the presence/absence of vibrotactile feedback in MI BCI-based training could contribute to the sensorimotor cortical activations, we compared the MEPs amplitude during MI after training with and without feedback. In addition, the ERD levels during MI BCI-based training were investigated. Our findings provide evidence that applying vibrotactile feedback during MI training leads to (i) an enhancement of the desynchronization level of mu-rhythm EEG patterns over the contralateral motor cortex area corresponding to the MI of the non-dominant hand; (ii) an increase in motor cortical excitability in hand muscle representation corresponding to a muscle engaged by the MI.

Characterization of kinesthetic motor imagery compared with visual motor imageries

Motor imagery (MI) is the only way for disabled subjects to robustly use a robot arm with a brain-machine interface. There are two main types of MI. Kinesthetic motor imagery (KMI) is proprioceptive (OR somato-) sensory imagination and Visual motor imagery (VMI) represents a visualization of the corresponding movement incorporating the visual network. Because these imagery tactics may use different networks, we hypothesized that the connectivity measures could characterize the two imageries better than the local activity. Electroencephalography data were recorded. Subjects performed different conditions, including motor execution (ME), KMI, VMI, and visual observation (VO). We tried to classify the KMI and VMI by conventional power analysis and by the connectivity measures. The mean accuracies of the classification of the KMI and VMI were 98.5% and 99.29% by connectivity measures (alpha and beta, respectively), which were higher than those by the normalized power (p < 0.01, Wilcoxon paired rank test). Additionally, the connectivity patterns were correlated between the ME-KMI and between the VO-VMI. The degree centrality (DC) was significantly higher in the left-S1 at the alpha-band in the KMI than in the VMI. The MI could be well classified because the KMI recruits a similar network to the ME. These findings could contribute to MI training methods.

A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback

Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.

Electroencephalographic modulations during an open- or closed-eyes motor task

There is fundamental knowledge that during the resting state cerebral activity recorded by electroencephalography (EEG) is strongly modulated by the eyes-closed condition compared to the eyes-open condition, especially in the occipital lobe. However, little research has demonstrated the influence of the eyes-closed condition on the motor cortex, particularly during a self-paced movement. This prompted the question: How does the motor cortex activity change between the eyes-closed and eyes-open conditions? To answer this question, we recorded EEG signals from 15 voluntary healthy subjects who performed a simple motor task (i.e., a voluntary isometric flexion of the right-hand index) under two conditions: eyes-closed and eyes-open. Our results confirmed strong modulation in the mu rhythm (7-13 Hz) with a large event-related desynchronisation. However, no significant differences have been observed in the beta band (15-30 Hz). Furthermore, evidence suggests that the eyes-closed condition influences the behaviour of subjects. This study gives us greater insight into the motor cortex and could also be useful in the brain-computer interface (BCI) domain.

User's Self-Prediction of Performance in Motor Imagery Brain-Computer Interface

Performance variation is a critical issue in motor imagery brain–computer interface (MI-BCI), and various neurophysiological, psychological, and anatomical correlates have been reported in the literature. Although the main aim of such studies is to predict MI-BCI performance for the prescreening of poor performers, studies which focus on the user’s sense of the motor imagery process and directly estimate MI-BCI performance through the user’s self-prediction are lacking. In this study, we first test each user’s self-prediction idea regarding motor imagery experimental datasets. Fifty-two subjects participated in a classical, two-class motor imagery experiment and were asked to evaluate their easiness with motor imagery and to predict their own MI-BCI performance. During the motor imagery experiment, an electroencephalogram (EEG) was recorded; however, no feedback on motor imagery was given to subjects. From EEG recordings, the offline classification accuracy was estimated and compared with several questionnaire scores of subjects, as well as with each subject’s self-prediction of MI-BCI performance. The subjects’ performance predictions during motor imagery task showed a high positive correlation (r = 0.64, p < 0.01). Interestingly, it was observed that the self-prediction became more accurate as the subjects conducted more motor imagery tasks in the Correlation coefficient (pre-task to 2nd run: r = 0.02 to r = 0.54, p < 0.01) and root mean square error (pre-task to 3rd run: 17.7% to 10%, p < 0.01). We demonstrated that subjects may accurately predict their MI-BCI performance even without feedback information. This implies that the human brain is an active learning system and, by self-experiencing the endogenous motor imagery process, it can sense and adopt the quality of the process. Thus, it is believed that users may be able to predict MI-BCI performance and results may contribute to a better understanding of low performance and advancing BCI.