Multi-scale self-attention approach for analysing motor imagery signals in brain-computer interfaces

BACKGROUND

Motor imagery-based electroencephalogram (EEG) brain-computer interface (BCI) technology has seen tremendous advancements in the past several years. Deep learning has outperformed more traditional approaches, such next-gen neuro-technologies, in terms of productivity. It is still challenging to develop and train an end-to-end network that can sufficiently extract the possible characteristics from EEG data used in motor imaging. Brain-computer interface research is largely reliant on the fundamental problem of accurately classifying EEG data. There are still many challenges in the field of MI classification even after researchers have proposed a variety of methods, such as deep learning and machine learning techniques.

METHODOLOGY

We provide a model for four-class categorization of motor imagery EEG signals using attention mechanisms: left hand, right hand, foot, and tongue/rest. The model is built on multi-scale spatiotemporal self-attention networks. To determine the most effective channels, self-attention networks are implemented spatially to assign greater weight to channels associated with motion and lesser weight to channels unrelated to motion. To eliminate noise in the temporal domain, parallel multi-scale Temporal Convolutional Network (TCN) layers are utilized to extract temporal domain features at various scales.

RESULT

On the IV-2b dataset from the BCI Competition, the suggested model achieved an accuracy of 85.09 %; on the IV-2a and IV-2b datasets from the HGD datasets, it was 96.26 %.

COMPARISON WITH EXISTING METHODS

In single-subject classification, this approach demonstrates superior accuracy when compared to existing methods.

CONCLUSION

The findings suggest that this approach exhibits commendable performance, resilience, and capacity for transfer learning.

Interactive Virtual Ankle Movement Controlled by Wrist sEMG Improves Motor Imagery: An Exploratory Study

Virtual reality (VR) techniques can significantly enhance motor imagery training by creating a strong illusion of action for central sensory stimulation. In this article, we establish a precedent by using surface electromyography (sEMG) of contralateral wrist movement to trigger virtual ankle movement through an improved data-driven approach with a continuous sEMG signal for fast and accurate intention recognition. Our developed VR interactive system can provide feedback training for stroke patients in the early stages, even if there is no active ankle movement. Our objectives are to evaluate: 1) the effects of VR immersion mode on body illusion, kinesthetic illusion, and motor imagery performance in stroke patients; 2) the effects of motivation and attention when utilizing wrist sEMG as a trigger signal for virtual ankle motion; 3) the acute effects on motor function in stroke patients. Through a series of well-designed experiments, we have found that, compared to the 2D condition, VR significantly increases the degree of kinesthetic illusion and body ownership of the patients, and improves their motor imagery performance and motor memory. When compared to conditions without feedback, using contralateral wrist sEMG signals as trigger signals for virtual ankle movement enhances patients' sustained attention and motivation during repetitive tasks. Furthermore, the combination of VR and feedback has an acute impact on motor function. Our exploratory study suggests that the sEMG-based immersive virtual interactive feedback provides an effective option for active rehabilitation training for severe hemiplegia patients in the early stages, with great potential for clinical application.

Virtual Reality Action Observation and Motor Imagery to Enhance Neuroplastic Capacity in the Human Motor Cortex: A Pilot Double-blind, Randomized Cross-over Trial

Neuroplasticity is important for learning, development and recovery from injury. Therapies that can upregulate neuroplasticity are therefore of interest across a range of fields. We developed a novel virtual reality action observation and motor imagery (VR-AOMI) intervention and evaluated whether it could enhance the efficacy of mechanisms of neuroplasticity in the human motor cortex of healthy adults. A secondary question was to explore predictors of the change in neuroplasticity following VR-AOMI. A pre-registered, pilot randomized controlled cross-over trial was performed. Twenty right-handed adults (13 females; mean age: 23.0 ± 4.53 years) completed two experimental conditions in separate sessions; VR-AOMI and control. We used intermittent theta burst stimulation (iTBS) to induce long term potentiation-like plasticity in the motor cortex and recorded motor evoked potentials at multiple timepoints as a measure of corticospinal excitability. The VR-AOMI task did not significantly increase the change in MEP amplitude following iTBS when compared to the control task (Group × Timepoint interaction p = 0.17). However, regression analysis identified the change in iTBS response following VR-AOMI was significantly predicted by the baseline iTBS response in the control task. Specifically, participants that did not exhibit the expected increase in MEP amplitude following iTBS in the control condition appear to have greater excitability following iTBS in the VR-AOMI condition (r = -0.72, p < 0.001). Engaging in VR-AOMI might enhance capacity for neuroplasticity in some people who typically do not respond to iTBS. VR-AOMI may prime the brain for enhanced neuroplasticity in this sub-group.

Enhancing Brain-Computer Interface Performance by Incorporating Brain-to-Brain Coupling

Human cooperation relies on key features of social interaction in order to reach desirable outcomes. Similarly, human-robot interaction may benefit from integration with human-human interaction factors. In this paper, we aim to investigate brain-to-brain coupling during motor imagery (MI)-based brain-computer interface (BCI) training using eye-contact and hand-touch interaction. Twelve pairs of friends (experimental group) and 10 pairs of strangers (control group) were recruited for MI-based BCI tests concurrent with electroencephalography (EEG) hyperscanning. Event-related desynchronization (ERD) was estimated to measure cortical activation, and interbrain functional connectivity was assessed using multilevel statistical analysis. Furthermore, we compared BCI classification performance under different social interaction conditions. In the experimental group, greater ERD was found around the contralateral sensorimotor cortex under social interaction conditions compared with MI without any social interaction. Notably, EEG channels with decreased power were mainly distributed around the frontal, central, and occipital regions. A significant increase in interbrain coupling was also found under social interaction conditions. BCI decoding accuracies were significantly improved in the eye contact condition and eye and hand contact condition compared with the no-interaction condition. However, for the strangers' group, no positive effects were observed in comparisons of cortical activations between interaction and no-interaction conditions. These findings indicate that social interaction can improve the neural synchronization between familiar partners with enhanced brain activations and brain-to-brain coupling. This study may provide a novel method for enhancing MI-based BCI performance in conjunction with neural synchronization between users.

Effects of Action Observation Plus Motor Imagery Administered by Immersive Virtual Reality on Hand Dexterity in Healthy Subjects

Action observation and motor imagery (AOMI) are commonly delivered through a laptop screen. Immersive virtual reality (VR) may enhance the observer's embodiment, a factor that may boost AOMI effects. The study aimed to investigate the effects on manual dexterity of AOMI delivered through immersive VR compared to AOMI administered through a laptop. To evaluate whether VR can enhance the effects of AOMI, forty-five young volunteers were enrolled and randomly assigned to the VR-AOMI group, who underwent AOMI through immersive VR, the AOMI group, who underwent AOMI through a laptop screen, or the control group, who observed landscape video clips. All participants underwent a 5-day treatment, consisting of 12 min per day. We investigated between and within-group differences after treatments relative to functional manual dexterity tasks using the Purdue Pegboard Test (PPT). This test included right hand (R), left hand (L), both hands (B), R + L + B, and assembly tasks. Additionally, we analyzed kinematics parameters including total and sub-phase duration, peak and mean velocity, and normalized jerk, during the Nine-Hole Peg Test to examine whether changes in functional scores may also occur through specific kinematic patterns. Participants were assessed at baseline (T0), after the first training session (T1), and at the end of training (T2). A significant time by group interaction and time effects were found for PPT, where both VR-AOMI and AOMI groups improved at the end of training. Larger PPT-L task improvements were found in the VR-AOMI group (d: 0.84, CI95: 0.09-1.58) compared to the AOMI group from T0 to T1. Immersive VR used for the delivery of AOMI speeded up hand dexterity improvements.

The individual difference of motor imagery ability evoked by visual stimulus and its personality manifestation

Motor imagery has been commonly studied as a means of motor rehabilitation but, the individual differences limit its practical application. Visually evoked motor imagery has been widely highlighted by researchers because of its vivid stimulus. However, this modality is still not applicable to all persons. In this study, we studied the different performances of the visually evoked motor imagery between subjects and tried to explore the personality manifestation which can result in this performance. We found that conscientiousness and openness have negative connections with the performance of visually evoked motor imagery. To compare with spontaneous motor imagery, the visually evoked motor imagery reflects less personality difference between subjects with good and bad performances on motor imagery. This indicate that visually stimulus may increase the pervasive application of motor imagery. This study may provide benefits to predict the rehabilitation effect and to rapidly select the suitable motor rehabilitation methods.

Event-Related EEG Desynchronization Reveals Enhanced Motor Imagery From the Third Person Perspective by Manipulating Sense of Body Ownership With Virtual Reality for Stroke Patients

Virtual reality (VR)-based rehabilitation training holds great potential for post-stroke motor recovery. Existing VR-based motor imagery (MI) paradigms mostly focus on the first-person perspective, and the benefit of the third-person perspective (3PP) remains to be further exploited. The 3PP is advantageous for movements involving the back or those with a large range because of its field coverage. Some movements are easier to imagine from the 3PP. However, the 3PP training efficiency may be unsatisfactory, which may be attributed to the difficulty encountered when generating a strong sense of ownership (SOO). In this work, we attempt to enhance a visual-guided 3PP MI in stroke patients by eliciting the SOO over a virtual avatar with VR. We propose to achieve this by inducing the so-called out-of-body experience (OBE), which is a full-body illusion (FBI) that people misperceive a 3PP virtual body as his/her own (i.e., generating the SOO to the virtual body). Electroencephalography signals of 13 stroke patients are recorded while MI of the affected upper limb is being performed. The proposed paradigm is evaluated by comparing event-related desynchronization (ERD) with a control paradigm without FBI induction. The results show that the proposed paradigm leads to a significantly larger ERD during MI, indicating a bilateral activation pattern consistent with that in previous studies. In conclusion, 3PP MI can be enhanced in stroke patients by eliciting the SOO through induction of the "OBE" FBI. This study offers more possibilities for virtual rehabilitation in stroke patients and can further facilitate VR application in rehabilitation.

BrainGridNet: A two-branch depthwise CNN for decoding EEG-based multi-class motor imagery

Brain-computer interfaces (BCIs) based on motor imagery (MI) enable the disabled to interact with the world through brain signals. To meet demands of real-time, stable, and diverse interactions, it is crucial to develop lightweight networks that can accurately and reliably decode multi-class MI tasks. In this paper, we introduce BrainGridNet, a convolutional neural network (CNN) framework that integrates two intersecting depthwise CNN branches with 3D electroencephalography (EEG) data to decode a five-class MI task. The BrainGridNet attains competitive results in both the time and frequency domains, with superior performance in the frequency domain. As a result, an accuracy of 80.26 percent and a kappa value of 0.753 are achieved by BrainGridNet, surpassing the state-of-the-art (SOTA) model. Additionally, BrainGridNet shows optimal computational efficiency, excels in decoding the most challenging subject, and maintains robust accuracy despite the random loss of 16 electrode signals. Finally, the visualizations demonstrate that BrainGridNet learns discriminative features and identifies critical brain regions and frequency bands corresponding to each MI class. The convergence of BrainGridNet's strong feature extraction capability, high decoding accuracy, steady decoding efficacy, and low computational costs renders it an appealing choice for facilitating the development of BCIs.

M-FANet: Multi-Feature Attention Convolutional Neural Network for Motor Imagery Decoding

Motor imagery (MI) decoding methods are pivotal in advancing rehabilitation and motor control research. Effective extraction of spectral-spatial-temporal features is crucial for MI decoding from limited and low signal-to-noise ratio electroencephalogram (EEG) signal samples based on brain-computer interface (BCI). In this paper, we propose a lightweight Multi-Feature Attention Neural Network (M-FANet) for feature extraction and selection of multi-feature data. M-FANet employs several unique attention modules to eliminate redundant information in the frequency domain, enhance local spatial feature extraction and calibrate feature maps. We introduce a training method called Regularized Dropout (R-Drop) to address training-inference inconsistency caused by dropout and improve the model's generalization capability. We conduct extensive experiments on the BCI Competition IV 2a (BCIC-IV-2a) dataset and the 2019 World robot conference contest-BCI Robot Contest MI (WBCIC-MI) dataset. M-FANet achieves superior performance compared to state-of-the-art MI decoding methods, with 79.28% 4-class classification accuracy (kappa: 0.7259) on the BCIC-IV-2a dataset and 77.86% 3-class classification accuracy (kappa: 0.6650) on the WBCIC-MI dataset. The application of multi-feature attention modules and R-Drop in our lightweight model significantly enhances its performance, validated through comprehensive ablation experiments and visualizations.

The third-person perspective full-body illusion induced by visual-tactile stimulation in virtual reality for stroke patients

This paper attempts to induce the third-person perspective full body illusion (3PP-FBI) with virtual reality (VR) in stroke patients. Nineteen individuals with stroke were recruited. The 3PP-FBI induction method, which was well-established in healthy individuals, using synchronous visual-tactile stimulation on one body part was used. Questionnaire scores and proprioceptive drift values were collected under different conditions for characterizing the induced 3PP-FBI. Results showed that synchronous visual-tactile stimulation of a single body part (back or upper limb) was sufficient to elicit 3PP-FBI in stroke patients, forming a sense of ownership (SOO) over the entire virtual body. Moreover, the intensity of 3PP-FBI was stronger when the back was stimulated, compared to stimulating the impaired upper limb. This study demonstrated the viability of visual-guided rehabilitation training while having a SOO to a virtual body from the third-person perspective, in anticipation of achieving better rehabilitation outcome for movements beyond the first-person perspective.

Environmental enrichment through virtual reality as multisensory stimulation to mitigate the negative effects of prolonged bed rest

Prolonged bed rest causes a multitude of deleterious physiological changes in the human body that require interventions even during immobilization to prevent or minimize these negative effects. In addition to other interventions such as physical and nutritional therapy, non-physical interventions such as cognitive training, motor imagery, and action observation have demonstrated efficacy in mitigating or improving not only cognitive but also motor outcomes in bedridden patients. Recent technological advances have opened new opportunities to implement such non-physical interventions in semi- or fully-immersive environments to enable the development of bed rest countermeasures. Extended Reality (XR), which covers augmented reality (AR), mixed reality (MR), and virtual reality (VR), can enhance the training process by further engaging the kinesthetic, visual, and auditory senses. XR-based enriched environments offer a promising research avenue to investigate the effects of multisensory stimulation on motor rehabilitation and to counteract dysfunctional brain mechanisms that occur during prolonged bed rest. This review discussed the use of enriched environment applications in bedridden patients as a promising tool to improve patient rehabilitation outcomes and suggested their integration into existing treatment protocols to improve patient care. Finally, the neurobiological mechanisms associated with the positive cognitive and motor effects of an enriched environment are highlighted.

Enhanced lower-limb motor imagery by kinesthetic illusion

Brain-computer interface (BCI) based on lower-limb motor imagery (LMI) enables hemiplegic patients to stand and walk independently. However, LMI ability is usually poor for BCI-illiterate (e.g., some stroke patients), limiting BCI performance. This study proposed a novel LMI-BCI paradigm with kinesthetic illusion(KI) induced by vibratory stimulation on Achilles tendon to enhance LMI ability. Sixteen healthy subjects were recruited to carry out two research contents: (1) To verify the feasibility of induced KI by vibrating Achilles tendon and analyze the EEG features produced by KI, research 1 compared the subjective feeling and brain activity of participants during rest task with and without vibratory stimulation (V-rest, rest). (2) Research 2 compared the LMI-BCI performance with and without KI (KI-LMI, no-LMI) to explore whether KI enhances LMI ability. The analysis methods of both experiments included classification accuracy (V-rest vs. rest, no-LMI vs. rest, KI-LMI vs. rest, KI-LMI vs. V-rest), time-domain features, oral questionnaire, statistic analysis and brain functional connectivity analysis. Research 1 verified that induced KI by vibrating Achilles tendon might be feasible, and provided a theoretical basis for applying KI to LMI-BCI paradigm, evidenced by oral questionnaire (Q1) and the independent effect of vibratory stimulation during rest task. The results of research 2 that KI enhanced mesial cortex activation and induced more intensive EEG features, evidenced by ERD power, topographical distribution, oral questionnaire (Q2 and Q3), and brain functional connectivity map. Additionally, the KI increased the offline accuracy of no-LMI/rest task by 6.88 to 82.19% (p < 0.001). The simulated online accuracy was also improved for most subjects (average accuracy for all subjects: 77.23% > 75.31%, and average F1_score for all subjects: 76.4% > 74.3%). The LMI-BCI paradigm of this study provides a novel approach to enhance LMI ability and accelerates the practical applications of the LMI-BCI system.

The effect of combining action observation in virtual reality with kinesthetic motor imagery on cortical activity

Introduction

In the past, various techniques have been used to improve motor imagery (MI), such as immersive virtual-reality (VR) and kinesthetic rehearsal. While electroencephalography (EEG) has been used to study the differences in brain activity between VR-based action observation and kinesthetic motor imagery (KMI), there has been no investigation into their combined effect. Prior research has demonstrated that VR-based action observation can enhance MI by providing both visual information and embodiment, which is the perception of oneself as part of the observed entity. Additionally, KMI has been found to produce similar brain activity to physically performing a task. Therefore, we hypothesized that utilizing VR to offer an immersive visual scenario for action observation while participants performed kinesthetic motor imagery would significantly improve cortical activity related to MI.

Methods

In this study, 15 participants (9 male, 6 female) performed kinesthetic motor imagery of three hand tasks (drinking, wrist flexion-extension, and grabbing) both with and without VR-based action observation.

Results

Our results indicate that combining VR-based action observation with KMI enhances brain rhythmic patterns and provides better task differentiation compared to KMI without action observation.

Discussion

These findings suggest that using VR-based action observation alongside kinesthetic motor imagery can improve motor imagery performance.

Survey of Movement Reproduction in Immersive Virtual Rehabilitation

Virtual reality (VR) has emerged as a powerful tool for rehabilitation. Many effective VR applications have been developed to support motor rehabilitation of people affected by motor issues. Movement reproduction, which transfers users' movements from the physical world to the virtual environment, is commonly used in VR rehabilitation applications. Three major components are required for movement reproduction in VR: (1) movement input, (2) movement representation, and (3) movement modulation. Until now, movement reproduction in virtual rehabilitation has not yet been systematically studied. This article aims to provide a state-of-the-art review on this subject by focusing on existing literature on immersive motor rehabilitation using VR. In this review, we provided in-depth discussions on the rehabilitation goals and outcomes, technology issues behind virtual rehabilitation, and user experience regarding movement reproduction. Similarly, we present good practices and highlight challenges and opportunities that can form constructive suggestions for the design and development of fit-for-purpose VR rehabilitation applications and can help frame future research directions for this emerging area that combines VR and health.

Tailoring brain-machine interface rehabilitation training based on neural reorganization: towards personalized treatment for stroke patients

Electroencephalogram (EEG)-based brain-machine interface (BMI) has the potential to enhance rehabilitation training efficiency, but it still remains elusive regarding how to design BMI training for heterogeneous stroke patients with varied neural reorganization. Here, we hypothesize that tailoring BMI training according to different patterns of neural reorganization can contribute to a personalized rehabilitation trajectory. Thirteen stroke patients were recruited in a 2-week personalized BMI training experiment. Clinical and behavioral measurements, as well as cortical and muscular activities, were assessed before and after training. Following treatment, significant improvements were found in motor function assessment. Three types of brain activation patterns were identified during BMI tasks, namely, bilateral widespread activation, ipsilesional focusing activation, and contralesional recruitment activation. Patients with either ipsilesional dominance or contralesional dominance can achieve recovery through personalized BMI training. Results indicate that personalized BMI training tends to connect the potentially reorganized brain areas with event-contingent proprioceptive feedback. It can also be inferred that personalization plays an important role in establishing the sensorimotor loop in BMI training. With further understanding of neural rehabilitation mechanisms, personalized treatment strategy is a promising way to improve the rehabilitation efficacy and promote the clinical use of rehabilitation robots and other neurotechnologies.

EEG changes during passive movements improve the motor imagery feature extraction in BCIs-based sensory feedback calibration

Objective.This work proposes a method for two calibration schemes based on sensory feedback to extract reliable motor imagery (MI) features, and provide classification outputs more correlated to the user's intention.Method.After filtering the raw electroencephalogram (EEG), a two-step method for spatial feature extraction by using the Riemannian covariance matrices (RCM) method and common spatial patterns is proposed here. It uses EEG data from trials providing feedback, in an intermediate step composed of bothkth nearest neighbors and probability analyses, to find periods of time in which the user probably performed well the MI task without feedback. These periods are then used to extract features with better separability, and train a classifier for MI recognition. For evaluation, an in-house dataset with eight healthy volunteers and two post-stroke patients that performed lower-limb MI, and consequently received passive movements as feedback was used. Other popular public EEG datasets (such as BCI Competition IV dataset IIb, among others) from healthy subjects that executed upper-and lower-limbs MI tasks under continuous visual sensory feedback were further used.Results.The proposed system based on the Riemannian geometry method in two-steps (RCM-RCM) outperformed significantly baseline methods, reaching average accuracy up to 82.29%. These findings show that EEG data on periods providing passive movement can be used to contribute greatly during MI feature extraction.Significance.Unconscious brain responses elicited over the sensorimotor areas may be avoided or greatly reduced by applying our approach in MI-based brain-computer interfaces (BCIs). Therefore, BCI's outputs more correlated to the user's intention can be obtained.

180-degree immersive VR motion visualization in the treatment of haemophilic ankle arthropathy

BACKGROUND

Patients with haemophilic arthropathy suffer chronic pain that affects and restricts their quality of life. Visualization of movement through immersive virtual reality is used for pain management.

AIM

To evaluate the efficacy of 180-degree immersive VR motion visualization therapy in patients with haemophilic ankle arthropathy.

METHODS

Prospective, multicentre pilot study. Fifteen adult patients with bilateral haemophilic ankle arthropathy were recruited (mean age: 42.73 ± 12.36 years). The intervention lasted 4 weeks, with daily home sessions of 180-degree immersive motion visualization. The patients were given virtual reality glasses to use with their smartphones. From the YouTube mobile app® they accessed the recorded video with access from the He-Mirror App®. The study variables were joint state (Haemophilia Joint Health Score), pressure pain threshold (pressure algometer), muscle strength (dynamometry) and range of motion (goniometry). Three evaluations were performed: at baseline (T0), after the intervention (T1) and at the end of a 16-week follow-up period (T2).

RESULTS

No patient developed ankle hemarthrosis during the experimental phase. In the repeated measures analysis we found statistically significant differences in joint state (F = 51.38; η² _p = .63), pressure pain threshold of the lateral malleolus (F = 12.34; η² _p = .29) and range of motion (F = 11.7; η² _p = .28).

CONCLUSIONS

Therapy using immersive motion visualization does not cause hemarthrosis. This intervention can improve joint condition, pressure pain threshold and range of motion in patients with ankle arthropathy. Changes greater than the MDC were reported in more than 40% of patients for the variables pressure pain threshold, anterior tibialis strength and range of motion, which were considered clinically relevant.

Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery

Objective.Among the existing active brain-computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy.Approach.This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition.Main results.The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects' scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm.Significance.The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.

Approach to Knee Arthropathy through 180-Degree Immersive VR Movement Visualization in Adult Patients with Severe Hemophilia: A Pilot Study

(1) Background: Hemarthrosis is a typical clinical manifestation in patients with hemophilia. Its recurrence causes hemophilic arthropathy, characterized by chronic joint pain. Watching movement recorded from a first-person perspective and immersively can be effective in the management of chronic pain. The objective of this study was to evaluate the effectiveness of an immersive virtual reality intervention in improving the pain intensity, joint condition, muscle strength and range of motion in patients with hemophilic knee arthropathy. (2) Methods: Thirteen patients with hemophilic knee arthropathy were recruited. The patients wore virtual reality glasses and watched a flexion-extension movement of the knee on an immersive 180° video, recorded from a first-person perspective over a 28-day period. The primary variable was the pain intensity (visual analog scale). The secondary variables were the joint status (Hemophilia Joint Health Score), quadriceps and hamstring strength (dynamometry), and range of motion (goniometry). (3) Results: After the intervention period, statistically significant differences were observed in the intensity of the joint pain (Standard error [SE] = 19.31; 95% interval confidence [95%CI] = -1.05; -0.26), joint condition (SE = 18.68; 95%CI = -1.16; -0.52) and quadriceps strength (SE = 35.00; 95%CI = 2.53; 17.47). We found that 38.46% and 23.07% of the patients exhibited an improvement in their quadriceps muscle strength and joint condition above the minimum detectable change for both variables (8.21% and 1.79%, respectively). (4) Conclusions: One hundred and eighty degree immersive VR motion visualization can improve the intensity of joint pain in patients with hemophilic knee arthropathy. An intervention using immersive virtual reality can be an effective complementary approach to improve the joint condition and quadriceps strength in these patients.

Exploring Biomimetic Stiffness Modulation and Wearable Finger Haptics for Improving Myoelectric Control of Virtual Hand

The embodiment of virtual hand (VH) by the user is generally deemed to be important for virtual reality (VR) based hand rehabilitation applications, which may help to engage the user and promote motor skill relearning. In particular, it requires that the VH should produce task-dependent interaction behaviors from rigid to soft. While such a capability is inherent to humans via hand stiffness regulation and haptic interactions, yet it have not been successfully imitated by VH in existing studies. In this paper, we present a work which integrates biomimetic stiffness regulation and wearable finger force feedback in VR scenarios involving myoelectric control of VH. On one hand, the biomimetic stiffness modulation intuitively enables VH to imitate the stiffness profile of the user's hand in real time. On the other hand, the wearable finger force-feedback device elicits a natural and realistic sensation of external force on the fingertip, which provides the user a proper understanding of the environment for enhancing his/her stiffness regulation. The benefits of the proposed integrated system were evaluated with eight healthy subjects that performed two tasks with opposite stiffness requirements. The achieved performance is compared with reduced versions of the integrated system, where either biomimetic impedance control or wearable force feedback is excluded. The results suggest that the proposed integrated system enables the stiffness of VH to be adaptively regulated by the user through the perception of interaction torques and vision, resulting in task-dependent behaviors from rigid to soft for VH.

Effects of a Virtual Reality-Based Mirror Therapy Program on Improving Sensorimotor Function of Hands in Chronic Stroke Patients: A Randomized Controlled Trial

Background. Embedding mirror therapy within a virtual reality (VR) system may have a superior effect on motor remediation for chronic stroke patients. Objective. The objective is to investigate the differences in the effects of using conventional occupational therapy (COT), mirror therapy (MT), and VR-based MT (VR-MT) training on the sensorimotor function of the upper limb in chronic stroke patients. Methods. This was a single-blinded randomized controlled trial. A total of 54 participants, including chronic stroke patients, were randomized into a COT, MT, or VR-MT group. In addition to 20-minute sessions of task-specific training, patients received programs of 30 minutes of VR-MT, 30 minutes of MT, and 30 minutes of COT, respectively, in the VR-MT, MT, and COT groups twice a week for 9 weeks. The Fugl-Meyer motor assessment for the upper extremities (FM-UE; primary outcome), Semmes-Weinstein monofilament, motor activity log, modified Ashworth scale, and the box and block test were recorded at pre-treatment, post-intervention, and 12-week follow-up. Results. Fifty-two participants completed the study. There was no statistically significant group-by-time interaction effects on the FM-UE score (generalized estimating equations, (GEE), P = .075). Meanwhile, there were statistically significant group-by-time interaction effects on the wrist sub-score of the FM-UE (GEE, P = .012) and the result of box and block test (GEE, P = .044). Conclusions. VR-MT seemed to have potential effects on restoring the upper extremity motor function for chronic stroke patients. However, further confirmatory studies are warranted for the rather weak evidence of adding VR to MT on improving primary outcome of this study. Clinical trial registration: NCT03329417.

[Evaluation the effectiveness of immersive VR-assisted rehabilitation in a child with chemotherapy-induced neurological complication in acute lymphoblastic leukemia]

Rehabilitation therapy is considered as an actual and complex system of knowledge, in which the main task is the development and implementation of new methods of rehabilitation. In present time, the most perspective rehabilitation program is utilizing virtual reality. A report was made with the utilization of rehabilitation therapy with virtual reality in a child with chemotherapy-induced neurological disorders in acute lymphoblastic leukemia. The child performed a set of exercises using fully immersive virtual reality. Over the course of rehabilitation positive dynamics was observed, namely increased muscle strength in the injured limb from 3 to 5 scores according to the Medical Research Council Weakness Scale. There was improved balance on the Berg Balance Scale from 35 to 42. In addition, there were increased range of active movements, partly restored biomechanics of gait with increased velocity by 2 times. According to the results of testing the psycho-emotional state using the Luscher color test and the graphic technique «Cactus» by M.A. Panfilova, self-esteem, the desire to succeed and independence were improved, the level of auto-aggression was decreased. The results show that rehabilitation using fully immersive virtual reality is probably a perspective tool in addition to traditional rehabilitation. It improves the neurological and psycho-emotional state, raises motivation of patients, which, in turn, helps to increase the effectiveness of rehabilitation therapy and speeds up the rehabilitation process.

A comparison of directed functional connectivity among fist-related brain activities during movement imagery, movement execution, and movement observation

Brain-computer interface (BCI) has been widely used in sports training and rehabilitation training. It is primarily based on action simulation, including movement imagery (MI) and movement observation (MO). However, the development of BCI technology is limited due to the challenge of getting an in-depth understanding of brain networks involved in MI, MO, and movement execution (ME). To better understand the brain activity changes and the communications across various brain regions under MO, ME, and MI, this study conducted the fist experiment under MO, ME, and MI. We recorded 64-channel electroencephalography (EEG) from 39 healthy subjects (25 males, 14 females, all right-handed) during fist tasks, obtained intensities and locations of sources using EEG source imaging (ESI), computed source activation modes, and finally investigated the brain networks using spectral Granger causality (GC). The brain regions involved in the three motor conditions are similar, but the degree of participation of each brain region and the network connections among the brain regions are different. MO, ME, and MI did not recruit shared brain connectivity networks. In addition, both source activation modes and brain network connectivity had lateralization advantages.

5 Hz rTMS improves motor-imagery based BCI classification performance

Brain-computer interface (BCI) based rehabilitation has been proven a promising method facilitating motor recovery. Recognizing motor intention is crucial for realizing BCI rehabilitation training. Event-related desynchronization (ERD) is a kind of electroencephalogram (EEG) inherent characteristics associated with motor intention. However, due to brain deficits poststroke, some patients are not able to generate ERD, which discourages them to be involved in BCI rehabilitation training. To boost ERD during motor imagery (MI), this paper investigates the effects of high-frequency repetitive transcranial magnetic stimulation (rTMS) on BCI classification performance. Eleven subjects participated in this study. The experiment consisted of two conditions: rTMS + MI versus sham rTMS + MI, which were arranged on different days. MI tests with 64-channel EEG recording were arranged immediately before and after rTMS and sham rTMS. Time-frequency analysis were utilized to measure ERD changes. Common spatial pattern was used to extract features and linear discriminant analysis was used to calculate offline classification accuracies. Paired-sample t-test and Wilcoxon signed rank tests with post-hoc analysis were used to compare performance before and after stimulation. Statistically stronger ERD (-13.93±12.99%) was found after real rTMS compared with ERD (-5.71±21.25%) before real rTMS (p<0.05). Classification accuracy after real rTMS (70.71±10.32%) tended to be higher than that before real rTMS (66.50±8.48%) (p<0.1). However, no statistical differences were found after sham stimulation. This research provides an effective method in improving BCI performance by utilizing neural modulation.Clinical Relevance- This study offers a promising treatment for patients who cannot be recruited in BCI rehabilitation training due to poor BCI classification performance.

One Session Effects of Knee Motion Visualization Using Immersive Virtual Reality in Patients with Hemophilic Arthropathy

(1) Background: Hemophilic knee arthropathy is characterized by a loss of muscle mass and decreased strength of the quadriceps muscle. The visualization of movement aims to favor the recruitment of the motor system in the same premotor and parietal areas, as would happen with the active execution of the observed action. The aim was to evaluate changes in quadriceps activation in patients with hemophilic knee arthropathy following immersive VR visualization of knee extension movements. (2) Methods: We recruited 13 patients with severe hemophilia A and knee arthropathy. Patients underwent a 15 min session of immersive VR visualization of knee extension movements. The quadriceps muscle activation was evaluated by surface electromyography. (3) Results: After the intervention, there were no changes in the muscle activation of vastus medialis, vastus lateralis, or rectus femoris muscles. There was a large effect size of changes in rectus femoris muscle activation. Age and knee joint damage did not correlate with changes in quadriceps activation. Dominance, inhibitor development, and type of treatment were not related with post-intervention muscle activation. (4) Conclusions: A session of immersive VR visualization of knee extension movement does not modify quadriceps muscle activation. A specific protocol for patients with hemophilic knee arthropathy may be effective in improving the activation of the rectus femoris muscle.

Using Medical Device Standards for Design and Risk Management of Immersive Virtual Reality for At-Home Therapy and Remote Patient Monitoring

Numerous virtual reality (VR) systems have received regulatory clearance as therapeutic medical devices for in-clinic and at-home use. These systems enable remote patient monitoring of clinician-prescribed rehabilitation exercises, although most of these systems are nonimmersive. With the expanding availability of affordable and easy-to-use head-mounted display (HMD)-based VR, there is growing interest in immersive VR therapies. However, HMD-based VR presents unique risks. Following standards for medical device development, the objective of this paper is to demonstrate a risk management process for a generic immersive VR system for remote patient monitoring of at-home therapy. Regulations, standards, and guidance documents applicable to therapeutic VR design are reviewed to provide necessary background. Generic requirements for an immersive VR system for home use and remote patient monitoring are identified using predicate analysis and specified for both patients and clinicians using user stories. To analyze risk, failure modes and effects analysis, adapted for medical device risk management, is performed on the generic user stories and a set of risk control measures is proposed. Many therapeutic applications of VR would be regulated as a medical device if they were to be commercially marketed. Understanding relevant standards for design and risk management early in the development process can help expedite the availability of innovative VR therapies that are safe and effective.

Development of a Guidance System for Motor Imagery Enhancement Using the Virtual Hand Illusion

Motor imagery (MI) is widely used to produce input signals for brain-computer interfaces (BCI) due to the similarities between MI-BCI and the planning-execution cycle. Despite its usefulness, MI tasks can be ambiguous to users and MI produces weaker cortical signals than motor execution. Existing MI guidance systems, which have been reported to provide visual guidance for MI and enhance MI, still have limitations: insufficient immersion for MI or poor expandability to MI for another body parts. We propose a guidance system for MI enhancement that can immerse users in MI and will be easy to extend to other body parts and target motions with few physical constraints. To make easily extendable MI guidance system, the virtual hand illusion is applied to the MI guidance system with a motion tracking sensor. MI enhancement was evaluated in 11 healthy people by comparison with another guidance system and conventional motor commands for BCI. The results showed that the proposed MI guidance system produced an amplified cortical signal compared to pure MI (p < 0.017), and a similar cortical signal as those produced by both actual execution (p > 0.534) and an MI guidance system with the rubber hand illusion (p > 0.722) in the contralateral region. Therefore, we believe that the proposed MI guidance system with the virtual hand illusion is a viable alternative to existing MI guidance systems in various applications with MI-BCI.

Speech-imagery-based brain-computer interface system using ear-EEG

OBJECTIVE

This study investigates the efficacy of electroencephalography (EEG) centered around the user's ears (ear-EEG) for a speech-imagery-based brain-computer interface (BCI) system.

APPROACH

A wearable ear-EEG acquisition tool was developed and its performance was directly compared to that of a conventional 32-channel scalp-EEG setup in a multi-class speech imagery classification task. Riemannian tangent space projections of EEG covariance matrices were used as input features to a multi-layer extreme learning machine classifier. Ten subjects participated in an experiment consisting of six sessions spanning three days. The experiment involves imagining four speech commands ('Left,' 'Right,' 'Forward,' and 'Go back') and staying in a rest condition.

MAIN RESULTS

The classification accuracy of our system is significantly above the chance level (20%). The classification result averaged across all ten subjects is 38.2% and 43.1% with a maximum (max) of 43.8% and 55.0% for ear-EEG and scalp-EEG, respectively. According to an analysis of variance, seven out of ten subjects show no significant difference between the performance of ear-EEG and scalp-EEG.

SIGNIFICANCE

To our knowledge, this is the first study that investigates the performance of ear-EEG in a speech-imagery-based BCI. The results indicate that ear-EEG has great potential as an alternative to the scalp-EEG acquisition method for speech-imagery monitoring. We believe that the merits and feasibility of both speech imagery and ear-EEG acquisition in the proposed system will accelerate the development of the BCI system for daily-life use.

Improving performance in motor imagery BCI-based control applications via virtually embodied feedback

OBJECTIVE

Brain-computer interfaces (BCIs) based on motor imagery (MI) are commonly used for control applications. However, these applications require strong and discriminant neural patterns for which extensive experience in MI may be necessary. Inspired by the field of rehabilitation where embodiment is a key element for improving cortical activity, our study proposes a novel control scheme in which virtually embodiable feedback is provided during control to enhance performance.

METHODS

Subjects underwent two immersive virtual reality control scenarios in which they controlled the two-dimensional movement of a device using electroencephalography (EEG). The two scenarios only differ on whether embodiable feedback, which mirrors the movement of the classified intention, is provided. After undergoing each scenario, subjects also answered a questionnaire in which they rated how immersive the scenario and embodiable the feedback were.

RESULTS

Subjects exhibited higher control performance, greater discriminability in brain activity patterns, and enhanced cortical activation when using our control scheme compared to the standard control scheme in which embodiable feedback is absent. Moreover, the self-rated embodiment and presence scores showed significantly positive linear relationships with performance.

SIGNIFICANCE

The findings in our study provide evidence that providing embodiable feedback as guidance on how intention is classified may be effective for control applications by inducing enhanced neural activity and patterns with greater discriminability. By applying embodiable feedback to immersive virtual reality, our study also serves as another instance in which virtual reality is shown to be a promising tool for improving MI.