Combined Action Observation and Motor Imagery Neurofeedback for Modulation of Brain Activity

Brain oscillations in reflecting motor status and recovery induced by action observation-driven robotic hand intervention in chronic stroke

Hand rehabilitation in chronic stroke remains challenging, and finding markers that could reflect motor function would help to understand and evaluate the therapy and recovery. The present study explored whether brain oscillations in different electroencephalogram (EEG) bands could indicate the motor status and recovery induced by action observation-driven brain-computer interface (AO-BCI) robotic therapy in chronic stroke. The neurophysiological data of 16 chronic stroke patients who received 20-session BCI hand training is the basis of the study presented here. Resting-state EEG was recorded during the observation of non-biological movements, while task-stage EEG was recorded during the observation of biological movements in training. The motor performance was evaluated using the Action Research Arm Test (ARAT) and upper extremity Fugl-Meyer Assessment (FMA), and significant improvements (p < 0.05) on both scales were found in patients after the intervention. Averaged EEG band power in the affected hemisphere presented negative correlations with scales pre-training; however, no significant correlations (p > 0.01) were found both in the pre-training and post-training stages. After comparing the variation of oscillations over training, we found patients with good and poor recovery presented different trends in delta, low-beta, and high-beta variations, and only patients with good recovery presented significant changes in EEG band power after training (delta band, p < 0.01). Importantly, motor improvements in ARAT correlate significantly with task EEG power changes (low-beta, c.c = 0.71, p = 0.005; high-beta, c.c = 0.71, p = 0.004) and task/rest EEG power ratio changes (delta, c.c = -0.738, p = 0.003; low-beta, c.c = 0.67, p = 0.009; high-beta, c.c = 0.839, p = 0.000). These results suggest that, in chronic stroke, EEG band power may not be a good indicator of motor status. However, ipsilesional oscillation changes in the delta and beta bands provide potential biomarkers related to the therapeutic-induced improvement of motor function in effective BCI intervention, which may be useful in understanding the brain plasticity changes and contribute to evaluating therapy and recovery in chronic-stage motor rehabilitation.

A review on EMG/EEG based control scheme of upper limb rehabilitation robots for stroke patients

Stroke is a common worldwide health problem and a crucial contributor to gained disability. The abilities of people, who are subjected to stroke, to live independently are significantly affected since affected upper limbs' functions are essential for our daily life. This review article focuses on emerging trends in BCI-controlled rehabilitation techniques based on EMG, EEG, or EGM + EEG signals in the last few years. Working on developing rehabilitation robotics, is considered a wealthy scientific area for researchers in the last period. There is a significant advantage that the human acquires from the interaction between the machine and his body, rehabilitation for a patient's limb is very important to get the body limb recovery, and this is what is provided mostly by applying robotic devices.

Motor-Related Mu/Beta Rhythm in Older Adults: A Comprehensive Review

Mu rhythm, also known as the mu wave, occurs on sensorimotor cortex activity at rest, and the frequency range is defined as 8-13Hz, the same frequency as the alpha band. Mu rhythm is a cortical oscillation that can be recorded from the scalp over the primary sensorimotor cortex by electroencephalogram (EEG) and magnetoencephalography (MEG). The subjects of previous mu/beta rhythm studies ranged widely from infants to young and older adults. Furthermore, these subjects were not only healthy people but also patients with various neurological and psychiatric diseases. However, very few studies have referred to the effect of mu/beta rhythm with aging, and there was no literature review about this theme. It is important to review the details of the characteristics of mu/beta rhythm activity in older adults compared with young adults, focusing on age-related mu rhythm changes. By comprehensive review, we found that, compared with young adults, older adults showed mu/beta activity change in four characteristics during voluntary movement, increased event-related desynchronization (ERD), earlier beginning and later end, symmetric pattern of ERD and increased recruitment of cortical areas, and substantially reduced beta event-related desynchronization (ERS). It was also found that mu/beta rhythm patterns of action observation were changing with aging. Future work is needed in order to investigate not only the localization but also the network of mu/beta rhythm in older adults.

Enhancement of motor imagery training efficiency by an online adaptive training paradigm integrated with error related potential

Objective. Motor imagery (MI) is a process of autonomously modulating the motor area to rehearse action mentally without actual execution. Based on the neuroplasticity of the cerebral cortex, MI can promote the functional rehabilitation of the injured cerebral cortex motor area. However, it usually takes several days to a few months to train individuals to acquire the necessary MI ability to control rehabilitation equipment in current studies, which greatly limits the clinical application of rehabilitation training systems based on the MI brain-computer interface (BCI).Approach. A novel MI training paradigm combined with the error related potential (ErrP) is proposed, and online adaptive training of the MI classifier was performed using ErrP. ErrP is used to correct the output of the MI classification to obtain a higher accuracy of kinesthetic feedback based on the imagination intention of subjects while generating simulated labels for MI online adaptive training. In this way, we improved the MI training efficiency. Thirteen subjects were randomly divided into an experimental group using the proposed paradigm and a control group using the traditional MI training paradigm to participate in six MI training experiments.Main results. The proposed paradigm enabled the experimental group to obtain a higher event-related desynchronization modulation level in the contralateral brain region compared with the control group and 69.76% online classification accuracy of MI after three MI training experiments. The online classification accuracy reached 72.76% and the whole system recognized the MI intention of the subjects with an online accuracy of 82.61% after six experiments.Significance. Compared with the conventional unimodal MI training strategy, the proposed approach enables subjects to use the MI-BCI based system directly and achieve a better performance after only three training experiments with training left and right hands simultaneously. This greatly improves the usability of the MI-BCI-based rehabilitation system and makes it more convenient for clinical use.

The Effect of Action Observation Combined with Motor Imagery Training on Upper Extremity Function and Corticospinal Excitability in Stroke Patients: A Randomized Controlled Trial

This study aimed to investigate the effect of motor imagery (MI) intervention with action observation (AO) on upper extremity function and corticospinal activation in stroke patients. MI and AO are two forms of motion simulation that activate the motor system without physical activity. There are many studies that show the effect of AO and MI alone, but there are few studies in parallel. This study was conducted on 45 patients who were divided into an experimental group (n = 22) and a control group (n = 23) using a computer randomization program. The experimental group conducted AO with MI, and the control group only AO. All participants received interventions for twenty-five minutes per session, five times a week, for eight weeks. For the pre- and post-evaluation of all participants, motor evoked potential (MEP) amplitude was measured to compare corticospinal activation, and Fugl-Meyer Assessment Upper Extremity (FMA UE), Wolf Motor Function Test (WMFT) and Motor Activity Log (MAL) were evaluated for changes in upper extremity function. In comparing the amount of change before and after the intervention, a significant change was observed in the experimental group's MEP amplitude. In the comparison between groups after the intervention and the amount of change before and after the intervention, the experimental group showed significant changes in FMA UE and MAL Amount of Use (MAL AOU) items. As a result of this study, AO with MI is effective in enhancing upper extremity function and increasing cortical spinal cord activation in patients with severe stroke with limited movement.

Functional Localizers for Motor Areas of the Brain Using fMRI

Neuroimaging researchers increasingly take advantage of the known functional properties of brain regions to localize motor regions in the brain and investigate changes in their activity under various conditions. Using this noninvasive functional MRI (fMRI) method makes it possible to identify and localize brain activation. There are many localizers that can be used to identify brain areas, namely, motor areas such as functional localizer, anatomical localizer, or Atlas mask. Eighteen right-handed participants were recruited for this research to test the reliability of five localizers for primary motor cortex (M1), supplementary motor area (SMA), premotor cortex (PMC), motor cerebellum, and motor thalamus. Motor execution task, namely, hand clenching was used to activate M1, SMA, and motor cerebellum. A combined action observation and motor imagery (AOMI) task was used to functionally activate PMC. Finally, a mask based on Talairach coordinates Atlas was created and used to identify the motor thalamus. Our results show that all localizers were successfully activated in the desired regions of interest. Motor execution successfully activated M1, SMA, and motor cerebellum. A novel localizer based on AOMI was successfully activated in PMC, and the motor thalamus mask obtained from the thalamus mask was successfully implemented on each participant. In conclusion, all five localizers tested in this research were reliable and can be used for rt-fMRI neurofeedback research to define the regions of interest.

Neural Suppression Elicited During Motor Imagery Following the Observation of Biological Motion From Point-Light Walker Stimuli

Introduction: Advantageous effects of biological motion (BM) detection, a low-perceptual mechanism that allows the rapid recognition and understanding of spatiotemporal characteristics of movement via salient kinematics information, can be amplified when combined with motor imagery (MI), i.e., the mental simulation of motor acts. According to Jeannerod's neurostimulation theory, asynchronous firing and reduction of mu and beta rhythm oscillations, referred to as suppression over the sensorimotor area, are sensitive to both MI and action observation (AO) of BM. Yet, not many studies investigated the use of BM stimuli using combined AO-MI tasks. In this study, we assessed the neural response in the form of event-related synchronization and desynchronization (ERD/S) patterns following the observation of point-light-walkers and concordant MI, as compared to MI alone. Methods: Twenty right-handed healthy participants accomplished the experimental task by observing BM stimuli and subsequently performing the same movement using kinesthetic MI (walking, cycling, and jumping conditions). We recorded an electroencephalogram (EEG) with 32 channels and performed time-frequency analysis on alpha (8-13 Hz) and beta (18-24 Hz) frequency bands during the MI task. A two-way repeated-measures ANOVA was performed to test statistical significance among conditions and electrodes of interest. Results: The results revealed significant ERD/S patterns in the alpha frequency band between conditions and electrode positions. Post hoc comparisons showed significant differences between condition 1 (walking) and condition 3 (jumping) over the left primary motor cortex. For the beta band, a significantly less difference in ERD patterns (p < 0.01) was detected only between condition 3 (jumping) and condition 4 (reference). Discussion: Our results confirmed that the observation of BM combined with MI elicits a neural suppression, although just in the case of jumping. This is in line with previous findings of AO and MI (AOMI) eliciting a neural suppression for simulated whole-body movements. In the last years, increasing evidence started to support the integration of AOMI training as an adjuvant neurorehabilitation tool in Parkinson's disease (PD). Conclusion: We concluded that using BM stimuli in AOMI training could be promising, as it promotes attention to kinematic features and imitative motor learning.

Noninvasive Electroencephalography Equipment for Assistive, Adaptive, and Rehabilitative Brain-Computer Interfaces: A Systematic Literature Review

Humans interact with computers through various devices. Such interactions may not require any physical movement, thus aiding people with severe motor disabilities in communicating with external devices. The brain-computer interface (BCI) has turned into a field involving new elements for assistive and rehabilitative technologies. This systematic literature review (SLR) aims to help BCI investigator and investors to decide which devices to select or which studies to support based on the current market examination. This examination of noninvasive EEG devices is based on published BCI studies in different research areas. In this SLR, the research area of noninvasive BCIs using electroencephalography (EEG) was analyzed by examining the types of equipment used for assistive, adaptive, and rehabilitative BCIs. For this SLR, candidate studies were selected from the IEEE digital library, PubMed, Scopus, and ScienceDirect. The inclusion criteria (IC) were limited to studies focusing on applications and devices of the BCI technology. The data used herein were selected using IC and exclusion criteria to ensure quality assessment. The selected articles were divided into four main research areas: education, engineering, entertainment, and medicine. Overall, 238 papers were selected based on IC. Moreover, 28 companies were identified that developed wired and wireless equipment as means of BCI assistive technology. The findings of this review indicate that the implications of using BCIs for assistive, adaptive, and rehabilitative technologies are encouraging for people with severe motor disabilities and healthy people. With an increasing number of healthy people using BCIs, other research areas, such as the motivation of players when participating in games or the security of soldiers when observing certain areas, can be studied and collaborated using the BCI technology. However, such BCI systems must be simple (wearable), convenient (sensor fabrics and self-adjusting abilities), and inexpensive.

An Introductory Tutorial on Brain–Computer Interfaces and Their Applications

The prospect and potentiality of interfacing minds with machines has long captured human imagination. Recent advances in biomedical engineering, computer science, and neuroscience are making brain–computer interfaces a reality, paving the way to restoring and potentially augmenting human physical and mental capabilities. Applications of brain–computer interfaces are being explored in applications as diverse as security, lie detection, alertness monitoring, gaming, education, art, and human cognition augmentation. The present tutorial aims to survey the principal features and challenges of brain–computer interfaces (such as reliable acquisition of brain signals, filtering and processing of the acquired brainwaves, ethical and legal issues related to brain–computer interface (BCI), data privacy, and performance assessment) with special emphasis to biomedical engineering and automation engineering applications. The content of this paper is aimed at students, researchers, and practitioners to glimpse the multifaceted world of brain–computer interfacing.

Brain-computer interface robotics for hand rehabilitation after stroke: a systematic review

BACKGROUND

Hand rehabilitation is core to helping stroke survivors regain activities of daily living. Recent studies have suggested that the use of electroencephalography-based brain-computer interfaces (BCI) can promote this process. Here, we report the first systematic examination of the literature on the use of BCI-robot systems for the rehabilitation of fine motor skills associated with hand movement and profile these systems from a technical and clinical perspective.

METHODS

A search for January 2010-October 2019 articles using Ovid MEDLINE, Embase, PEDro, PsycINFO, IEEE Xplore and Cochrane Library databases was performed. The selection criteria included BCI-hand robotic systems for rehabilitation at different stages of development involving tests on healthy participants or people who have had a stroke. Data fields include those related to study design, participant characteristics, technical specifications of the system, and clinical outcome measures.

RESULTS

30 studies were identified as eligible for qualitative review and among these, 11 studies involved testing a BCI-hand robot on chronic and subacute stroke patients. Statistically significant improvements in motor assessment scores relative to controls were observed for three BCI-hand robot interventions. The degree of robot control for the majority of studies was limited to triggering the device to perform grasping or pinching movements using motor imagery. Most employed a combination of kinaesthetic and visual response via the robotic device and display screen, respectively, to match feedback to motor imagery.

CONCLUSION

19 out of 30 studies on BCI-robotic systems for hand rehabilitation report systems at prototype or pre-clinical stages of development. We identified large heterogeneity in reporting and emphasise the need to develop a standard protocol for assessing technical and clinical outcomes so that the necessary evidence base on efficiency and efficacy can be developed.

On the (in)efficacy of motor imagery training without feedback and event-related desynchronizations considerations

Motor imagery (MI) constitutes a recurrent strategy for signals generation in brain-computer interfaces (BCIs) - systems that aim to control external devices by directly associating brain responses to distinct commands. Although great improvement has been achieved in MI-BCIs performance over recent years, they still suffer from inter- and intra-subject variability issues. As an attempt to cope with this, some studies have suggested that MI training should aid users to appropriately modulate their response for BCI usage: generally, this training is performed based on the sensorimotor rhythms' modulation over the primary sensorimotor cortex (PMC), with the signal being feedbacked to the user. Nonetheless, recent studies have revisited the actual involvement of the PMC into MI, and little to no attention has been devoted to understanding the participation of other cortical areas into training protocols. Therefore, in this work, our aim was to analyze the response induced by hands MI of 10 healthy subjects in the form of event-related desynchronizations (ERDs) and to assess whether features from beyond the PMC might be useful for hands MI classification. We investigated how this response occurs for distinct frequency intervals between 7-30 Hz, and ex0plored changes in their evocation pattern across 12 MI training sessions without feedback. Overall, we found that ERD patterns occur differently for the frequencies encompassed by the μ and β bands, with its evocation being favored for the first band. Over time, the no-feedback approach was inefficient to aid in enhancing ERD evocation (EO). Moreover, to some extent, EO tends to decrease over blocks within a given run, and runs within an MI session, but remains stable within an MI block. We also found that the C3/C4 pair is not necessarily optimal for data classification, and both spectral and spatial subjects' specificities should be considered when designing training protocols.

A study of EEG mu neurofeedback during action observation

The mirror system is a brain network that gets activated during action performance and observation. Brain mu waves have been used as a mirror system activity index; however, mu rhythm is prone to contamination by occipital alpha wave activity, thus raising a concern regarding its reliability as an index of the mirror system activity. In this study, we investigated whether mu suppression can be used as an index of neurofeedback training, which influences mirror system activities. Participants observed videos of hand movement under three different conditions: central mu feedback (muFB), occipital alpha feedback (aFB), and simple observation without any feedback (OBS). Results showed that at the 4-5 min mark, mu wave was most significantly suppressed in the central site at muFB. We thus demonstrated the possibility of increasing mu wave suppression in feedback training using a specific stimulus such as motion observation.

A visual-haptic neurofeedback training improves sensorimotor cortical activations and BCI performance

Neurofeedback training (NFT) could provide a novel way to investigate or restore the impaired brain function and neuroplasticity. However, it remains unclear how much the different feedback modes can contribute to NFT training. Specifically, whether they can enhance the cortical activations for motor training. To this end, our study proposed a brain-computer interface (BCI) based visual-haptic NFT incorporating synchronous visual scene and proprioceptive electrical stimulation feedback. By comparison between previous and posterior control sessions, the cortical activations measured by multi-band (i.e. alpha_1: 8-10Hz, alpha_2: 11-13Hz, beta_1: 15-20Hz and beta_2: 22-28Hz) lateralized relative event-related desynchronization (lrERD) patterns were significantly enhanced after NFT. And the classification performance was also significantly improved, achieving a ~9% improvement and reaching ~85% in mean classification accuracy from a relatively low MI-BCI performance. These findings validate the feasibility of our proposed visual- haptic NFT approach to improve sensorimotor cortical activations and BCI performance during motor training.

Enhancement of motor-imagery ability via combined action observation and motor-imagery training with proprioceptive neurofeedback

Varied individual ability to control the sensory-motor rhythms may limit the potential use of motor-imagery (MI) in neurorehabilitation and neuroprosthetics. We employed neurofeedback training of MI under action observation (AO: AOMI) with proprioceptive feedback and examined whether it could enhance MI-induced event-related desynchronization (ERD). Twenty-eight healthy young adults participated in the neurofeedback training. They performed MI while watching a video of hand-squeezing motion from a first-person perspective. Eleven participants received correct proprioceptive feedback of the same hand motion with the video, via an exoskeleton robot attached to their hand, upon their successful generation of ERD. Another nine participants received random feedback. The training lasted for approximately 20 min per day and continued for 6 days within an interval of 2 weeks. MI-ERD power was evaluated separately, without AO, on each experimental day. The MI-ERD power of the participants receiving correct feedback, as opposed to random feedback, was significantly increased after training. An additional experiment in which the remaining eight participants were trained with auditory instead of proprioceptive feedback failed to show statistically significant increase in MI-ERD power. The significant training effect obtained in shorter training time relative to previously proposed methods suggests the superiority of AOMI training and physiologically-congruent proprioceptive feedback to enhance the MI-ERD power. The proposed neurofeedback training could help patients with motor deficits to attain better use of brain-machine interfaces for rehabilitation and/or prosthesis.

Body position and motor imagery strategy effects on imagining gait in healthy adults: Results from a cross-sectional study

Background

Assessment of changes in higher levels of gait control with aging is important to better understand age-related gait instability, with the perspective to improve the screening of individuals at risk for falls. The comparison between actual Timed Up and Go test (aTUG) and its imagined version (iTUG) is a simple clinical way to assess age-related changes in gait control. The modulations of iTUG performances by body positions and motor imagery (MI) strategies with normal aging have not been evaluated yet. This study aims 1) to compare the aTUG time with the iTUG time under different body positions (i.e., sitting, standing or supine) in healthy young and middle age, and older adults, and 2) to examine the associations of body positions and MI strategies (i.e., egocentric versus allocentric) with the time needed to complete the iTUG and the delta TUG time (i.e., relative difference between aTUG and iTUG) while taking into consideration clinical characteristics of participants.

Methods

A total of 60 healthy individuals (30 young and middle age participants 26.6±7.4 years, and 30 old participants 75.0±4.4 years) were recruited in this cross-sectional study. The iTUG was performed while sitting, standing and in supine position. Times of the aTUG, the iTUG under the three body positions, the TUG delta time and the strategies of MI (i.e., ego representation, defined as representation of the location of objects in space relative to the body axes of the self, versus allocentric representation defined as encoding information about body movement with respect to other object, the location of body being defined relative to the location of other objects) were used as outcomes. Age, sex, height, weight, number of drugs taken daily, level of physical activity and prevalence of closed eyes while performing iTUG were recorded.

Results

The aTUG time is significantly greater than iTUG while sitting and standing (P<0.001), except when older participants are standing. A significant difference is reported between iTUG while sitting or standing and iTUG while supine (P≤0.002), higher time being reported in supine position. The multiple linear regressions confirm that the supine position is associated with significant increased iTUG (P≤0.04) and decreased TUG delta time (P≤0.010), regardless of the adjustment. Older participants use the allocentric MI while imagining TUG more frequently than young and middle age participants, regardless of body positions (P≤0.001). Allocentric MI strategy is associated with a significant decrease in iTUG (P = 0.037) only while adjusting for age. A significant increase of iTUG time is associated with age (P≤0.026).

Conclusions

Supine position while imagining TUG represents a more accurate position of actual performance of TUG. Age has a limited effect on iTUG performance but is associated with a change in MI from ego to allocentric representation that decreases the iTUG performances, and thus increases the discrepancy with aTUG.

Hand Rehabilitation Robotics on Poststroke Motor Recovery

The recovery of hand function is one of the most challenging topics in stroke rehabilitation. Although the robot-assisted therapy has got some good results in the latest decades, the development of hand rehabilitation robotics is left behind. Existing reviews of hand rehabilitation robotics focus either on the mechanical design on designers' view or on the training paradigms on the clinicians' view, while these two parts are interconnected and both important for designers and clinicians. In this review, we explore the current literature surrounding hand rehabilitation robots, to help designers make better choices among varied components and thus promoting the application of hand rehabilitation robots. An overview of hand rehabilitation robotics is provided in this paper firstly, to give a general view of the relationship between subjects, rehabilitation theories, hand rehabilitation robots, and its evaluation. Secondly, the state of the art hand rehabilitation robotics is introduced in detail according to the classification of the hardware system and the training paradigm. As a result, the discussion gives available arguments behind the classification and comprehensive overview of hand rehabilitation robotics.

Retest reliability of individual alpha ERD topography assessed by human electroencephalography

Background

Despite the immense literature related to diverse human electroencephalographic (EEG) parameters, very few studies have focused on the reliability of these measures. Some of the most studied components (i.e., P3 or MMN) have received more attention regarding the stability of their main parameters, such as latency, amplitude or topography. However, spectral modulations have not been as extensively evaluated considering that different analysis methods are available.

The main aim of the present study is to assess the reliability of the latency, amplitude and topography of event-related desynchronization (ERD) for the alpha band (10–14 Hz) observed in a cognitive task (visual oddball). Topography reliability was analysed at different levels (for the group, within-subjects individually and between-subjects individually).

Results

The latency for alpha ERD showed stable behaviour between two sessions, and the amplitude exhibited an increment (more negative) in the second session.

Alpha ERD topography exhibited a high correlation score between sessions at the group level (r = 0.903, p<0.001). The mean value for within-subject correlations was 0.750 (with a range from 0.391 to 0.954). Regarding between-subject topography comparisons, some subjects showed a highly specific topography, whereas other subjects showed topographies that were more similar to those of other subjects.

Conclusion

ERD was mainly stable between the two sessions with the exception of amplitude, which exhibited an increment in the second session. Topography exhibits excellent reliability at the group level; however, it exhibits highly heterogeneous behaviour at the individual level. Considering that the P3 was previously evaluated for this group of subjects, a direct comparison of the correlation scores was possible, and it showed that the ERD component is less reliable in individual topography than in the ERP component (P3).

Virtual and Actual Humanoid Robot Control with Four-Class Motor-Imagery-Based Optical Brain-Computer Interface

Motor-imagery tasks are a popular input method for controlling brain-computer interfaces (BCIs), partially due to their similarities to naturally produced motor signals. The use of functional near-infrared spectroscopy (fNIRS) in BCIs is still emerging and has shown potential as a supplement or replacement for electroencephalography. However, studies often use only two or three motor-imagery tasks, limiting the number of available commands. In this work, we present the results of the first four-class motor-imagery-based online fNIRS-BCI for robot control. Thirteen participants utilized upper- and lower-limb motor-imagery tasks (left hand, right hand, left foot, and right foot) that were mapped to four high-level commands (turn left, turn right, move forward, and move backward) to control the navigation of a simulated or real robot. A significant improvement in classification accuracy was found between the virtual-robot-based BCI (control of a virtual robot) and the physical-robot BCI (control of the DARwIn-OP humanoid robot). Differences were also found in the oxygenated hemoglobin activation patterns of the four tasks between the first and second BCI. These results corroborate previous findings that motor imagery can be improved with feedback and imply that a four-class motor-imagery-based fNIRS-BCI could be feasible with sufficient subject training.