Predicting motor behavior: an efficient EEG signal processing pipeline to detect brain states with potential therapeutic relevance for VR-based neurorehabilitation

A review of combined functional neuroimaging and motion capture for motor rehabilitation

BACKGROUND

Technological advancements in functional neuroimaging and motion capture have led to the development of novel methods that facilitate the diagnosis and rehabilitation of motor deficits. These advancements allow for the synchronous acquisition and analysis of complex signal streams of neurophysiological data (e.g., EEG, fNIRS) and behavioral data (e.g., motion capture). The fusion of those data streams has the potential to provide new insights into cortical mechanisms during movement, guide the development of rehabilitation practices, and become a tool for assessment and therapy in neurorehabilitation.

RESEARCH OBJECTIVE

This paper aims to review the existing literature on the combined use of motion capture and functional neuroimaging in motor rehabilitation. The objective is to understand the diversity and maturity of technological solutions employed and explore the clinical advantages of this multimodal approach.

METHODS

This paper reviews literature related to the combined use of functional neuroimaging and motion capture for motor rehabilitation following the PRISMA guidelines. Besides study and participant characteristics, technological aspects of the used systems, signal processing methods, and the nature of multimodal feature synchronization and fusion were extracted.

RESULTS

Out of 908 publications, 19 were included in the final review. Basic or translation studies were mainly represented and based predominantly on healthy participants or stroke patients. EEG and mechanical motion capture technologies were most used for biomechanical data acquisition, and their subsequent processing is based mainly on traditional methods. The system synchronization techniques at large were underreported. The fusion of multimodal features mainly supported the identification of movement-related cortical activity, and statistical methods were occasionally employed to examine cortico-kinematic relationships.

CONCLUSION

The fusion of motion capture and functional neuroimaging might offer advantages for motor rehabilitation in the future. Besides facilitating the assessment of cognitive processes in real-world settings, it could also improve rehabilitative devices' usability in clinical environments. Further, by better understanding cortico-peripheral coupling, new neuro-rehabilitation methods can be developed, such as personalized proprioceptive training. However, further research is needed to advance our knowledge of cortical-peripheral coupling, evaluate the validity and reliability of multimodal parameters, and enhance user-friendly technologies for clinical adaptation.

Exposure to Green, Blue and Historic Environments and Mental Well-Being: A Comparison between Virtual Reality Head-Mounted Display and Flat Screen Exposure

Improving the mental health of urban residents is a global public health priority. This study builds on existing work that demonstrates the ability of virtual exposure to restorative environments to improve population mental health. It compares the restorative effects of green, blue and historic environments delivered by both flat screen and immersive virtual reality technology, and triangulates data from psychological, physiological and qualitative sources. Results from the subjective measure analyses showed that exposures to all the experimental videos were associated with self-reported reduced anxiety and improved mood, although the historic environment was associated with a smaller reduction of anxiety (p < 0.01). These results were supported by the qualitative accounts. For two of the electroencephalography (EEG) frequency bands, higher levels of activity were observed for historic environments. In relation to the mode of delivery, the subjective measures did not suggest any effect, while for the EEG analyses there was evidence of a significant effect of technology across three out of four frequency bands. In conclusion, this study adds to the evidence that the benefits of restorative environments can be delivered through virtual exposure and suggests that virtual reality may provide greater levels of immersion than flat screen viewing.

Effects of Restorative Environment and Presence on Anxiety and Depression Based on Interactive Virtual Reality Scenarios

Anxiety and depression have been growing global mental health problems. The following studies explored the effect of interactive VR scenarios to find a low-cost and high-efficiency solution. Study 1 designed a 2 (anxiety and depression state) × 4 (interactive VR scenarios) experiment, the results of 20 participants showed that the designed scenarios had good restoration and presence, assisting to improve depression mood for people with mild to moderate anxiety and depression. Study 2 further investigated the intervention effects of two environment types (urban and park) and four interactive activities (automatic viewing, free-roaming, fishing, and watering plants in the park environment), based on data from a 10-minute experiment conducted by 195 participants with mild to moderate anxiety and depression. The subjective scales, EEG and EMG, and scenario experience were analyzed and the results showed that: (1) the restorative and present VR scenarios were beneficial in alleviating state anxiety and depression; (2) the restorative environment and presence were significantly and positively related to the reduction of anxiety and depression respectively, moreover, presence mediated the restorative environment on the recovery from anxiety and depression; (3) the environmental settings, the complexity of interaction, human factors, and maturity of VR devices and technology were also key factors that influenced the effects of interactive VR scenario experience and intervention. These studies revealed VR psychological intervention scenarios could be designed with comprehensive factors. Moreover, they might help pave the way for future study in exploring the physiology and psychology mode in virtual and real spaces, enhancing intervention effectiveness.

Artifacts in EEG-Based BCI Therapies: Friend or Foe?

EEG-based brain-computer interfaces (BCI) have promising therapeutic potential beyond traditional neurofeedback training, such as enabling personalized and optimized virtual reality (VR) neurorehabilitation paradigms where the timing and parameters of the visual experience is synchronized with specific brain states. While BCI algorithms are often designed to focus on whichever portion of a signal is most informative, in these brain-state-synchronized applications, it is of critical importance that the resulting decoder is sensitive to physiological brain activity representative of various mental states, and not to artifacts, such as those arising from naturalistic movements. In this study, we compare the relative classification accuracy with which different motor tasks can be decoded from both extracted brain activity and artifacts contained in the EEG signal. EEG data were collected from 17 chronic stroke patients while performing six different head, hand, and arm movements in a realistic VR-based neurorehabilitation paradigm. Results show that the artifactual component of the EEG signal is significantly more informative than brain activity with respect to classification accuracy. This finding is consistent across different feature extraction methods and classification pipelines. While informative brain signals can be recovered with suitable cleaning procedures, we recommend that features should not be designed solely to maximize classification accuracy, as this could select for remaining artifactual components. We also propose the use of machine learning approaches that are interpretable to verify that classification is driven by physiological brain states. In summary, whereas informative artifacts are a helpful friend in BCI-based communication applications, they can be a problematic foe in the estimation of physiological brain states.