Electroencephalographic evidence for the involvement of mirror-neuron and error-monitoring related processes in virtual body ownership

Enhancing motor imagery in the third-person perspective by manipulating sense of body ownership with virtual reality

Virtual reality (VR)-guided motor imagery (MI) is a widely used approach for motor rehabilitation, especially for patients with severe motor impairments. Most approaches provide visual guidance from the first-person perspective (1PP). MI training with visual guidance from the third-person perspective (3PP) remains largely unexplored. We argue that 3PP MI training has its own advantages and can supplement 1PP MI. For some movements beyond the view of 1PP, such as shoulder shrugging and other axial movements, MI are suitable performed under 3PP. However, the efficiency of existing paradigms for 3PP MI is unsatisfactory. We speculate that the absence of sense of body ownership (SOO) from 3PP could be one possible factor and hypothesize that 3PP MI could be enhanced by eliciting SOO over a 3PP avatar. Based on our hypothesis, a novel paradigm was proposed to enhance 3PP MI by inducing full-body illusion (FBI) from 3PP, which is similar to the so-called out-of-body experience (OBE), using synchronous visuo-tactile stimulus with VR. The event-related Electroencephalograph (EEG) desynchronization (ERD) at motor-related regions from 31 healthy participants were calculated and compared with a control paradigm without "OBE" FBI induction. This study attempts to enhance 3PP MI with FBI induction. It offers an opportunity to perform MI guided by action observation from 3PP with elicited SOO to the observed avatar. We believe that 3PP MI could provide more possibilities for effective rehabilitation training, when SOO could be elicited to a virtual avatar and the present work demonstrates its viability and effectiveness.

Measurement of Empathy in Virtual Reality with Head-Mounted Displays: A Systematic Review

We present a systematic review of 111 papers that measure the impact of virtual experiences created through head-mounted displays (HMDs) on empathy. Our goal was to analyze the conditions and the extent to which virtual reality (VR) enhances empathy. To achieve this, we categorized the relevant literature according to measurement methods, correlated human factors, viewing experiences, topics, and participants. Meta-analysis was performed based on categorized themes, and under specified conditions, we found that VR can improve empathy. Emotional empathy increased temporarily after the VR experience and returned to its original level over time, whereas cognitive empathy remained enhanced. Furthermore, while VR did not surpass 2D video in improving emotional empathy, it did enhance cognitive empathy, which is associated with embodiment. Our results are consistent with existing research suggesting differentiation between cognitive empathy (influenced by environmental factors and learnable) and emotional empathy (highly heritable and less variable). Interactivity, target of empathy, and point of view were not found to significantly affect empathy, but participants' age and nationality were found to influence empathy levels. It can be concluded that VR enhances cognitive empathy by immersing individuals in the perspective of others and that storytelling and personal characteristics are more important than the composition of the VR scene. Our findings provide guiding information for creating empathy content in VR and designing experiments to measure empathy.

Customizing the human-avatar mapping based on EEG error related potentials

Objective.A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption.Approach.We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion.Main results.Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling 'Plug-and-Play' ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports.Significance.We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.

Event-Related EEG Desynchronization Reveals Enhanced Motor Imagery From the Third Person Perspective by Manipulating Sense of Body Ownership With Virtual Reality for Stroke Patients

Virtual reality (VR)-based rehabilitation training holds great potential for post-stroke motor recovery. Existing VR-based motor imagery (MI) paradigms mostly focus on the first-person perspective, and the benefit of the third-person perspective (3PP) remains to be further exploited. The 3PP is advantageous for movements involving the back or those with a large range because of its field coverage. Some movements are easier to imagine from the 3PP. However, the 3PP training efficiency may be unsatisfactory, which may be attributed to the difficulty encountered when generating a strong sense of ownership (SOO). In this work, we attempt to enhance a visual-guided 3PP MI in stroke patients by eliciting the SOO over a virtual avatar with VR. We propose to achieve this by inducing the so-called out-of-body experience (OBE), which is a full-body illusion (FBI) that people misperceive a 3PP virtual body as his/her own (i.e., generating the SOO to the virtual body). Electroencephalography signals of 13 stroke patients are recorded while MI of the affected upper limb is being performed. The proposed paradigm is evaluated by comparing event-related desynchronization (ERD) with a control paradigm without FBI induction. The results show that the proposed paradigm leads to a significantly larger ERD during MI, indicating a bilateral activation pattern consistent with that in previous studies. In conclusion, 3PP MI can be enhanced in stroke patients by eliciting the SOO through induction of the "OBE" FBI. This study offers more possibilities for virtual rehabilitation in stroke patients and can further facilitate VR application in rehabilitation.

Mu Rhythm Desynchronization while Observing Rubber Hand Movement in the Mirror: The Interaction of Body Representation with Visuo-Tactile Stimulation

During rubber hand illusion (RHI), participants feel that a rubber (fake) hand is their own (i.e., embodiment of the rubber hand) if the unseen real hand and seen rubber hand are stroked synchronously (i.e., visuo-tactile stimuli). The RHI is also evoked if the real and rubber hands are placed in the same position (i.e., visual-proprioceptive congruency), which can be performed using a mirror setting. Using electroencephalography (EEG) and mirror settings, we compared μ rhythm (8-13 Hz) event-related desynchronization (ERD; an index of sensorimotor activation) while watching the movements of embodied or non-embodied rubber hands, which was preceded by an observation of the rubber hand with or without synchronous visuo-tactile stimuli. The illusory ownership of the fake hand was manipulated using visual continuity with (RHI) and without (non-RHI) a fake forearm. Resultantly, an ownership-dependent μ rhythm ERD was found when delivering visuo-tactile stimuli; a greater and more persistent μ rhythm ERD during the rubber hand movement was identified in the RHI in comparison to the non-RHI condition. However, no difference was observed between the two when observing the fake hand alone. These findings suggest the possibility that a self-related multisensory interaction between body representation (top-down processing) and visuo-tactile inputs (bottom-up processing) before a fake hand movement produces ownership-dependent sensorimotor activations during subsequent movement observations.

Evaluating the Alterations Induced by Virtual Reality in Cerebral Small-World Networks Using Graph Theory Analysis with Electroencephalography

Virtual reality (VR), a rapidly evolving technology that simulates three-dimensional virtual environments for users, has been proven to activate brain functions. However, the continuous alteration pattern of the functional small-world network in response to comprehensive three-dimensional stimulation rather than realistic two-dimensional media stimuli requires further exploration. Here, we aimed to validate the effect of VR on the pathways and network parameters of a small-world organization and interpret its mechanism of action. Fourteen healthy volunteers were selected to complete missions in an immersive VR game. The changes in the functional network in six different frequency categories were analyzed using graph theory with electroencephalography data measured during the pre-, VR, and post-VR stages. The mutual information matrix revealed that interactions between the frontal and posterior areas and those within the frontal and occipital lobes were strengthened. Subsequently, the betweenness centrality (BC) analysis indicated more robust and extensive pathways among hubs. Furthermore, a specific lateralized channel (O1 or O2) increment in the BC was observed. Moreover, the network parameters improved simultaneously in local segregation, global segregation, and global integration. The overall topological improvements of small-world organizations were in high-frequency bands and exhibited some degree of sustainability.

Virtual/Augmented Reality for Rehabilitation Applications Using Electromyography as Control/Biofeedback: Systematic Literature Review

Virtual reality (VR) and augmented reality (AR) are engaging interfaces that can be of benefit for rehabilitation therapy. However, they are still not widely used, and the use of surface electromyography (sEMG) signals is not established for them. Our goal is to explore whether there is a standardized protocol towards therapeutic applications since there are not many methodological reviews that focus on sEMG control/feedback. A systematic literature review using the PRISMA (preferred reporting items for systematic reviews and meta-analyses) methodology is conducted. A Boolean search in databases was performed applying inclusion/exclusion criteria; articles older than 5 years and repeated were excluded. A total of 393 articles were selected for screening, of which 66.15% were excluded, 131 records were eligible, 69.46% use neither VR/AR interfaces nor sEMG control; 40 articles remained. Categories are, application: neurological motor rehabilitation (70%), prosthesis training (30%); processing algorithm: artificial intelligence (40%), direct control (20%); hardware: Myo Armband (22.5%), Delsys (10%), proprietary (17.5%); VR/AR interface: training scene model (25%), videogame (47.5%), first-person (20%). Finally, applications are focused on motor neurorehabilitation after stroke/amputation; however, there is no consensus regarding signal processing or classification criteria. Future work should deal with proposing guidelines to standardize these technologies for their adoption in clinical practice.

"Tricking the Brain" Using Immersive Virtual Reality: Modifying the Self-Perception Over Embodied Avatar Influences Motor Cortical Excitability and Action Initiation

To offer engaging neurorehabilitation training to neurologic patients, motor tasks are often visualized in virtual reality (VR). Recently introduced head-mounted displays (HMDs) allow to realistically mimic the body of the user from a first-person perspective (i.e., avatar) in a highly immersive VR environment. In this immersive environment, users may embody avatars with different body characteristics. Importantly, body characteristics impact how people perform actions. Therefore, alternating body perceptions using immersive VR may be a powerful tool to promote motor activity in neurologic patients. However, the ability of the brain to adapt motor commands based on a perceived modified reality has not yet been fully explored. To fill this gap, we "tricked the brain" using immersive VR and investigated if multisensory feedback modulating the physical properties of an embodied avatar influences motor brain networks and control. Ten healthy participants were immersed in a virtual environment using an HMD, where they saw an avatar from first-person perspective. We slowly transformed the surface of the avatar (i.e., the "skin material") from human to stone. We enforced this visual change by repetitively touching the real arm of the participant and the arm of the avatar with a (virtual) hammer, while progressively replacing the sound of the hammer against skin with stone hitting sound via loudspeaker. We applied single-pulse transcranial magnetic simulation (TMS) to evaluate changes in motor cortical excitability associated with the illusion. Further, to investigate if the "stone illusion" affected motor control, participants performed a reaching task with the human and stone avatar. Questionnaires assessed the subjectively reported strength of embodiment and illusion. Our results show that participants experienced the "stone arm illusion." Particularly, they rated their arm as heavier, colder, stiffer, and more insensitive when immersed with the stone than human avatar, without the illusion affecting their experienced feeling of body ownership. Further, the reported illusion strength was associated with enhanced motor cortical excitability and faster movement initiations, indicating that participants may have physically mirrored and compensated for the embodied body characteristics of the stone avatar. Together, immersive VR has the potential to influence motor brain networks by subtly modifying the perception of reality, opening new perspectives for the motor recovery of patients.

Distinct modulation of mu and beta rhythm desynchronization during observation of embodied fake hand rotation

The rubber hand illusion (RHI) is a phenomenon whereby participants recognize a fake hand as their own. Studies have examined the effects of observing fake hand movements after the RHI on brain sensorimotor activity, although results remain controversial. To address these discrepancies, we investigated the effects of observation of fake hand rotation after the RHI on sensorimotor mu (μ: 8-13 Hz) and beta (β: 15-25 Hz) rhythm event-related desynchronization (ERD) using electroencephalography (EEG). Questionnaire results and proprioceptive drift revealed that the RHI occurred in participants when their invisible hand and fake visible hand were stroked synchronously but not during asynchronous stroking. Independent component (IC) clustering from EEG data during movement observation identified three IC clusters, including the right sensorimotor, left sensorimotor, and left occipital cluster. In the right sensorimotor cluster, we observed distinct modulation of μ and β ERD during fake hand rotation. Illusory ownership over the fake hand enhanced μ ERD but inversely attenuated β ERD. Further, the extent of μ ERD correlated with proprioceptive drift, but not with questionnaire ratings, whereas the converse results were obtained for β ERD. No ownership-dependent ERD modulation was detected in the left sensorimotor cluster. Alpha (α: 8-13 Hz) rhythm ERD of the left occipital cluster was smaller in the synchronous condition than in the asynchronous condition, but α ERD was not correlated with questionnaire rating or drift. These findings suggest that observing embodied fake hand rotation induces distinct cortical processing in sensorimotor brain areas.

The Challenges and Perspectives of the Integration Between Virtual and Augmented Reality and Manual Therapies

Virtual reality (VR) and augmented reality (AR) have been combined with physical rehabilitation and psychological treatments to improve patients' emotional reactions, body image, and physical function. Nonetheless, no detailed investigation assessed the relationship between VR or AR manual therapies (MTs), which are touch-based approaches that involve the manipulation of tissues for relieving pain and improving balance, postural stability and well-being in several pathological conditions. The present review attempts to explore whether and how VR and AR might be integrated with MTs to improve patient care, with particular attention to balance and to fields like chronic pain that need an approach that engages both mind and body. MTs rely essentially on touch to induce tactile, proprioceptive, and interoceptive stimulations, whereas VR and AR rely mainly on visual, auditory, and proprioceptive stimulations. MTs might increase patients' overall immersion in the virtual experience by inducing parasympathetic tone and relaxing the mind, thus enhancing VR and AR effects. VR and AR could help manual therapists overcome patients' negative beliefs about pain, address pain-related emotional issues, and educate them about functional posture and movements. VR and AR could also engage and change the sensorimotor neural maps that the brain uses to cope with environmental stressors. Hence, combining MTs with VR and AR could define a whole mind-body intervention that uses psychological, interoceptive, and exteroceptive stimulations for rebalancing sensorimotor integration, distorted perceptions, including visual, and body images. Regarding the technology needed to integrate VR and AR with MTs, head-mounted displays could be the most suitable devices due to being low-cost, also allowing patients to follow VR therapy at home. There is enough evidence to argue that integrating MTs with VR and AR could help manual therapists offer patients better and comprehensive treatments. However, therapists need valid tools to identify which patients would benefit from VR and AR to avoid potential adverse effects, and both therapists and patients have to be involved in the development of VR and AR applications to define truly patient-centered therapies. Furthermore, future studies should assess whether the integration between MTs and VR or AR is practically feasible, safe, and clinically useful.

Virtual reality training enhances gait poststroke: a systematic review and meta-analysis

Virtual reality (VR)-based interventions are gaining widespread attention for managing neurological disorders such as stroke. A metastatistical consensus regarding the intervention is strongly warranted. In this study, we attempt to address this gap in the literature and provide the current state of evidence for the effects of VR on gait performance. We conducted both between- and within-group meta-analyses to provide a state of evidence for VR. Moreover, we conducted a search adhering to PRISMA guidelines on nine databases. Out of 1866 records, 32 studies involving a total of 809 individuals were included in this review. Considering all included studies, significant enhancements in gait parameters were observed with VR-based interventions compared with conventional therapy. A between-group meta-analysis reported beneficial significant medium effects of VR training on cadence (Hedge's g = 0.55), stride length ((STrL; Hedge's g = 0.46), and gait speed (Hedge's g = 0.30). Similarly, a within-group meta-analysis further revealed positive medium effects of VR on cadence (Hedge's g = 0.76), STrL (Hedge's g = 0.61), and gait speed (Hedge's g = 0.69). Additional subgroup analyses revealed beneficial effects of joint application of VR and robot-assisted gait training on gait speed (Hedge's g = 0.50). Collectively, findings from this review provide evidence for the effectiveness of VR-based gait training for stroke survivors.