"Mine Works Better": Examining the Influence of Embodiment in Virtual Reality on the Sense of Agency During a Binary Motor Imagery Task With a Brain-Computer Interface

Enhancing motor imagery in the third-person perspective by manipulating sense of body ownership with virtual reality

Virtual reality (VR)-guided motor imagery (MI) is a widely used approach for motor rehabilitation, especially for patients with severe motor impairments. Most approaches provide visual guidance from the first-person perspective (1PP). MI training with visual guidance from the third-person perspective (3PP) remains largely unexplored. We argue that 3PP MI training has its own advantages and can supplement 1PP MI. For some movements beyond the view of 1PP, such as shoulder shrugging and other axial movements, MI are suitable performed under 3PP. However, the efficiency of existing paradigms for 3PP MI is unsatisfactory. We speculate that the absence of sense of body ownership (SOO) from 3PP could be one possible factor and hypothesize that 3PP MI could be enhanced by eliciting SOO over a 3PP avatar. Based on our hypothesis, a novel paradigm was proposed to enhance 3PP MI by inducing full-body illusion (FBI) from 3PP, which is similar to the so-called out-of-body experience (OBE), using synchronous visuo-tactile stimulus with VR. The event-related Electroencephalograph (EEG) desynchronization (ERD) at motor-related regions from 31 healthy participants were calculated and compared with a control paradigm without "OBE" FBI induction. This study attempts to enhance 3PP MI with FBI induction. It offers an opportunity to perform MI guided by action observation from 3PP with elicited SOO to the observed avatar. We believe that 3PP MI could provide more possibilities for effective rehabilitation training, when SOO could be elicited to a virtual avatar and the present work demonstrates its viability and effectiveness.

Analyzing and computing humans by means of the brain using Brain-Computer Interfaces - understanding the user - previous evidence, self-relevance and the user's self-concept as potential superordinate human factors of relevance

Brain-computer interfaces (BCIs) are well-known instances of how technology can convert a user's brain activity taken from non-invasive electroencephalography (EEG) into computer commands for the purpose of computer-assisted communication and interaction. However, not all users are attaining the accuracy required to use a BCI consistently, despite advancements in technology. Accordingly, previous research suggests that human factors could be responsible for the variance in BCI performance among users. Therefore, the user's internal mental states and traits including motivation, affect or cognition, personality traits, or the user's satisfaction, beliefs or trust in the technology have been investigated. Going a step further, this manuscript aims to discuss which human factors could be potential superordinate factors that influence BCI performance, implicitly, explicitly as well as inter- and intraindividually. Based on the results of previous studies that used comparable protocols to examine the motivational, affective, cognitive state or personality traits of healthy and vulnerable EEG-BCI users within and across well-investigated BCIs (P300-BCIs or SMR-BCIs, respectively), it is proposed that the self-relevance of tasks and stimuli and the user's self-concept provide a huge potential for BCI applications. As potential key human factors self-relevance and the user's self-concept (self-referential knowledge and beliefs about one's self) guide information processing and modulate the user's motivation, attention, or feelings of ownership, agency, and autonomy. Changes in the self-relevance of tasks and stimuli as well as self-referential processing related to one's self (self-concept) trigger changes in neurophysiological activity in specific brain networks relevant to BCI. Accordingly, concrete examples will be provided to discuss how past and future research could incorporate self-relevance and the user's self-concept in the BCI setting - including paradigms, user instructions, and training sessions.

Continual Learning of a Transformer-Based Deep Learning Classifier Using an Initial Model from Action Observation EEG Data to Online Motor Imagery Classification

The motor imagery (MI)-based brain computer interface (BCI) is an intuitive interface that enables users to communicate with external environments through their minds. However, current MI-BCI systems ask naïve subjects to perform unfamiliar MI tasks with simple textual instruction or a visual/auditory cue. The unclear instruction for MI execution not only results in large inter-subject variability in the measured EEG patterns but also causes the difficulty of grouping cross-subject data for big-data training. In this study, we designed an BCI training method in a virtual reality (VR) environment. Subjects wore a head-mounted device (HMD) and executed action observation (AO) concurrently with MI (i.e., AO + MI) in VR environments. EEG signals recorded in AO + MI task were used to train an initial model, and the initial model was continually improved by the provision of EEG data in the following BCI training sessions. We recruited five healthy subjects, and each subject was requested to participate in three kinds of tasks, including an AO + MI task, an MI task, and the task of MI with visual feedback (MI-FB) three times. This study adopted a transformer- based spatial-temporal network (TSTN) to decode the user's MI intentions. In contrast to other convolutional neural network (CNN) or recurrent neural network (RNN) approaches, the TSTN extracts spatial and temporal features, and applies attention mechanisms along spatial and temporal dimensions to perceive the global dependencies. The mean detection accuracies of TSTN were 0.63, 0.68, 0.75, and 0.77 in the MI, first MI-FB, second MI-FB, and third MI-FB sessions, respectively. This study demonstrated the AO + MI gave an easier way for subjects to conform their imagery actions, and the BCI performance was improved with the continual learning of the MI-FB training process.

Implementing Performance Accommodation Mechanisms in Online BCI for Stroke Rehabilitation: A Study on Perceived Control and Frustration

Brain-computer interfaces (BCIs) are successfully used for stroke rehabilitation, but the training is repetitive and patients can lose the motivation to train. Moreover, controlling the BCI may be difficult, which causes frustration and leads to even worse control. Patients might not adhere to the regimen due to frustration and lack of motivation/engagement. The aim of this study was to implement three performance accommodation mechanisms (PAMs) in an online motor imagery-based BCI to aid people and evaluate their perceived control and frustration. Nineteen healthy participants controlled a fishing game with a BCI in four conditions: (1) no help, (2) augmented success (augmented successful BCI-attempt), (3) mitigated failure (turn unsuccessful BCI-attempt into neutral output), and (4) override input (turn unsuccessful BCI-attempt into successful output). Each condition was followed-up and assessed with Likert-scale questionnaires and a post-experiment interview. Perceived control and frustration were best predicted by the amount of positive feedback the participant received. PAM-help increased perceived control for poor BCI-users but decreased it for good BCI-users. The input override PAM frustrated the users the most, and they differed in how they wanted to be helped. By using PAMs, developers have more freedom to create engaging stroke rehabilitation games.