Development of a Guidance System for Motor Imagery Enhancement Using the Virtual Hand Illusion

Enhancing motor imagery in the third-person perspective by manipulating sense of body ownership with virtual reality

Virtual reality (VR)-guided motor imagery (MI) is a widely used approach for motor rehabilitation, especially for patients with severe motor impairments. Most approaches provide visual guidance from the first-person perspective (1PP). MI training with visual guidance from the third-person perspective (3PP) remains largely unexplored. We argue that 3PP MI training has its own advantages and can supplement 1PP MI. For some movements beyond the view of 1PP, such as shoulder shrugging and other axial movements, MI are suitable performed under 3PP. However, the efficiency of existing paradigms for 3PP MI is unsatisfactory. We speculate that the absence of sense of body ownership (SOO) from 3PP could be one possible factor and hypothesize that 3PP MI could be enhanced by eliciting SOO over a 3PP avatar. Based on our hypothesis, a novel paradigm was proposed to enhance 3PP MI by inducing full-body illusion (FBI) from 3PP, which is similar to the so-called out-of-body experience (OBE), using synchronous visuo-tactile stimulus with VR. The event-related Electroencephalograph (EEG) desynchronization (ERD) at motor-related regions from 31 healthy participants were calculated and compared with a control paradigm without "OBE" FBI induction. This study attempts to enhance 3PP MI with FBI induction. It offers an opportunity to perform MI guided by action observation from 3PP with elicited SOO to the observed avatar. We believe that 3PP MI could provide more possibilities for effective rehabilitation training, when SOO could be elicited to a virtual avatar and the present work demonstrates its viability and effectiveness.

Event-Related EEG Desynchronization Reveals Enhanced Motor Imagery From the Third Person Perspective by Manipulating Sense of Body Ownership With Virtual Reality for Stroke Patients

Virtual reality (VR)-based rehabilitation training holds great potential for post-stroke motor recovery. Existing VR-based motor imagery (MI) paradigms mostly focus on the first-person perspective, and the benefit of the third-person perspective (3PP) remains to be further exploited. The 3PP is advantageous for movements involving the back or those with a large range because of its field coverage. Some movements are easier to imagine from the 3PP. However, the 3PP training efficiency may be unsatisfactory, which may be attributed to the difficulty encountered when generating a strong sense of ownership (SOO). In this work, we attempt to enhance a visual-guided 3PP MI in stroke patients by eliciting the SOO over a virtual avatar with VR. We propose to achieve this by inducing the so-called out-of-body experience (OBE), which is a full-body illusion (FBI) that people misperceive a 3PP virtual body as his/her own (i.e., generating the SOO to the virtual body). Electroencephalography signals of 13 stroke patients are recorded while MI of the affected upper limb is being performed. The proposed paradigm is evaluated by comparing event-related desynchronization (ERD) with a control paradigm without FBI induction. The results show that the proposed paradigm leads to a significantly larger ERD during MI, indicating a bilateral activation pattern consistent with that in previous studies. In conclusion, 3PP MI can be enhanced in stroke patients by eliciting the SOO through induction of the "OBE" FBI. This study offers more possibilities for virtual rehabilitation in stroke patients and can further facilitate VR application in rehabilitation.

"Tricking the Brain" Using Immersive Virtual Reality: Modifying the Self-Perception Over Embodied Avatar Influences Motor Cortical Excitability and Action Initiation

To offer engaging neurorehabilitation training to neurologic patients, motor tasks are often visualized in virtual reality (VR). Recently introduced head-mounted displays (HMDs) allow to realistically mimic the body of the user from a first-person perspective (i.e., avatar) in a highly immersive VR environment. In this immersive environment, users may embody avatars with different body characteristics. Importantly, body characteristics impact how people perform actions. Therefore, alternating body perceptions using immersive VR may be a powerful tool to promote motor activity in neurologic patients. However, the ability of the brain to adapt motor commands based on a perceived modified reality has not yet been fully explored. To fill this gap, we "tricked the brain" using immersive VR and investigated if multisensory feedback modulating the physical properties of an embodied avatar influences motor brain networks and control. Ten healthy participants were immersed in a virtual environment using an HMD, where they saw an avatar from first-person perspective. We slowly transformed the surface of the avatar (i.e., the "skin material") from human to stone. We enforced this visual change by repetitively touching the real arm of the participant and the arm of the avatar with a (virtual) hammer, while progressively replacing the sound of the hammer against skin with stone hitting sound via loudspeaker. We applied single-pulse transcranial magnetic simulation (TMS) to evaluate changes in motor cortical excitability associated with the illusion. Further, to investigate if the "stone illusion" affected motor control, participants performed a reaching task with the human and stone avatar. Questionnaires assessed the subjectively reported strength of embodiment and illusion. Our results show that participants experienced the "stone arm illusion." Particularly, they rated their arm as heavier, colder, stiffer, and more insensitive when immersed with the stone than human avatar, without the illusion affecting their experienced feeling of body ownership. Further, the reported illusion strength was associated with enhanced motor cortical excitability and faster movement initiations, indicating that participants may have physically mirrored and compensated for the embodied body characteristics of the stone avatar. Together, immersive VR has the potential to influence motor brain networks by subtly modifying the perception of reality, opening new perspectives for the motor recovery of patients.

Proof-of-Concept of a Sensor-Based Evaluation Method for Better Sensitivity of Upper-Extremity Motor Function Assessment

In rehabilitation, the Fugl-Meyer assessment (FMA) is a typical clinical instrument to assess upper-extremity motor function of stroke patients, but it cannot measure fine changes of motor function (both in recovery and deterioration) due to its limited sensitivity. This paper introduces a sensor-based automated FMA system that addresses this limitation with a continuous rating algorithm. The system consists of a depth sensor (Kinect V2) and an algorithm to rate the continuous FM scale based on fuzzy inference. Using a binary logic based classification method developed from a linguistic scoring guideline of FMA, we designed fuzzy input/output variables, fuzzy rules, membership functions, and a defuzzification method for several representative FMA tests. A pilot trial with nine stroke patients was performed to test the feasibility of the proposed approach. The continuous FM scale from the proposed algorithm exhibited a high correlation with the clinician rated scores and the results showed the possibility of more sensitive upper-extremity motor function assessment.