Bridging the gap between motor imagery and motor execution with a brain–robot interface

A comprehensive guide to BCI-based stroke neurorehabilitation interventions

Brain-Computer Interfaces (BCIs) offer the potential to facilitate neurorehabilitation in stroke patients by decoding user intentions from the central nervous system, thereby enabling control over external devices. Despite their promise, the diverse range of intervention parameters and technical challenges in clinical settings have hindered the accumulation of substantial evidence supporting the efficacy and effectiveness of BCIs in stroke rehabilitation. This article introduces a practical guide designed to navigate through these challenges in conducting BCI interventions for stroke rehabilitation. Applicable regardless of infrastructure and study design limitations, this guide acts as a comprehensive reference for executing BCI-based stroke interventions. Furthermore, it encapsulates insights gleaned from administering hundreds of BCI rehabilitation sessions to stroke patients.•Presents a comprehensive methodology for implementing BCI-based upper extremity therapy in stroke patients.•Provides detailed guidance on the number of sessions, trials, as well as the necessary hardware and software for effective intervention.

Hand-worn devices for assessment and rehabilitation of motor function and their potential use in BCI protocols: a review

Introduction

Various neurological conditions can impair hand function. Affected individuals cannot fully participate in activities of daily living due to the lack of fine motor control. Neurorehabilitation emphasizes repetitive movement and subjective clinical assessments that require clinical experience to administer.

Methods

Here, we perform a review of literature focused on the use of hand-worn devices for rehabilitation and assessment of hand function. We paid particular attention to protocols that involve brain-computer interfaces (BCIs) since BCIs are gaining ground as a means for detecting volitional signals as the basis for interactive motor training protocols to augment recovery. All devices reviewed either monitor, assist, stimulate, or support hand and finger movement.

Results

A majority of studies reviewed here test or validate devices through clinical trials, especially for stroke. Even though sensor gloves are the most commonly employed type of device in this domain, they have certain limitations. Many such gloves use bend or inertial sensors to monitor the movement of individual digits, but few monitor both movement and applied pressure. The use of such devices in BCI protocols is also uncommon.

Discussion

We conclude that hand-worn devices that monitor both flexion and grip will benefit both clinical diagnostic assessment of function during treatment and closed-loop BCI protocols aimed at rehabilitation.

Resolving equivocal gain modulation of corticospinal excitability

The ratio between the input and output of neuronal populations, usually referred to as gain modulation, is rhythmically modulated along the oscillatory cycle. Previous research on spinal neurons, however, revealed contradictory findings: both uni- and bimodal patterns of increased responsiveness for synaptic input have been proposed for the oscillatory beta rhythm. In this study, we compared previous approaches of phase estimation directly on simulated data and empirically tested the corresponding predictions in healthy males and females. We applied single-pulse transcranial magnetic stimulation over the primary motor cortex at rest, and assessed the spinal output generated by this input. Specifically, the peak-to-peak amplitude of the motor evoked potential in the contralateral forearm was estimated as a function of the EMG phase at which the stimulus was applied. The findings indicated that human spinal neurons adhere to a unimodal pattern of increased responsiveness, and suggest that the rising phase of the upper beta band maximizes gain modulation. Importantly, a bimodal pattern of increased responsiveness was shown to result in an artifact during data analysis and filtering. This observation of invalid preprocessing could be generalized to other frequency bands (i.e., delta, theta, alpha, and gamma), different task conditions (i.e., voluntary muscle contraction), and EEG-based phase estimations. Appropriate analysis algorithms, such as broad-band filtering, enable us to accurately determine gain modulation of neuronal populations and to avoid erroneous phase estimations. This may facilitate novel phase-specific interventions for targeted neuromodulation.

Applying correlation analysis to electrode optimization in source domain

In brain computer interface-based neurorehabilitation system, a large number of electrodes may increase the difficulty of signal acquisition and the time consumption of decoding algorithm for motor imagery EEG (MI-EEG). The traditional electrode optimization methods were limited by the low spatial resolution of scalp EEG. EEG source imaging (ESI) was further applied to reduce the number of electrodes, in which either the electrodes covering activated cortical areas were selected, or the reconstructed electrodes of EEGs with higher Fisher scores were retained. However, the activated dipoles do not all contribute equally to decoding, and the Fisher score cannot represent the correlations between electrodes and dipoles. In this paper, based on ESI and correlation analysis, a novel electrode optimization method, denoted ECCEO, was developed. The scalp MI-EEG was mapped to cortical regions by ESI, and the dipoles with larger amplitudes were chosen to designate a region of interest (ROI). Then, Pearson correlation coefficients between each dipole of the ROI and the corresponding electrode were calculated, averaged, and ranked to obtain two average correlation coefficient sequences. A small but important group of electrodes for each class were alternately added to the predetermined basic electrode set to form a candidate electrode set. Their features were extracted and evaluated to determine the optimal electrode set. Experiments were conducted on two public datasets, the average decoding accuracies achieved 95.99% and 88.30%, and the reduction of computational cost were 65% and 56%, respectively; statistical significance was examined as well.

Oscillatory neurofeedback networks and poststroke rehabilitative potential in severely impaired stroke patients

Motor restoration after severe stroke is often limited. However, some of the severely impaired stroke patients may still have a rehabilitative potential. Biomarkers that identify these patients are sparse. Eighteen severely impaired chronic stroke patients with a lack of volitional finger extension participated in an EEG study. During sixty-six trials of kinesthetic motor imagery, a brain-machine interface turned event-related beta-band desynchronization of the ipsilesional sensorimotor cortex into opening of the paralyzed hand by a robotic orthosis. A subgroup of eight patients participated in a subsequent four-week rehabilitation training. Changes of the movement extent were captured with sensors which objectively quantified even discrete improvements of wrist movement. Albeit with the same motor impairment level, patients could be differentiated into two groups, i.e., with and without task-related increase of bilateral cortico-cortical phase synchronization between frontal/premotor and parietal areas. This fronto-parietal integration (FPI) was associated with a significantly higher volitional beta modulation range in the ipsilesional sensorimotor cortex. Following the four-week training, patients with FPI showed significantly higher improvement in wrist movement than those without FPI. Moreover, only the former group improved significantly in the upper extremity Fugl-Meyer-Assessment score. Neurofeedback-related long-range oscillatory coherence may differentiate severely impaired stroke patients with regard to their rehabilitative potential, a finding that needs to be confirmed in larger patient cohorts.

Sources of independent mu components reveal different brain areas involved in motor imagery, motor execution, and movement observation

To answer the question of whether the same brain circuit(s) facilitates motor imagery (MI), motor execution (ME), and movement observation (MO), we conducted electroencephalography (EEG) experiment combining the three motor conditions in the same experimental runs. The EEG data were analyzed using two different independent component analysis (ICA) decomposition approaches: a single ICA decomposition on all EEG data combined and separate ICA decomposition on the EEG data obtained from the separate conditions. The results indicated that the separate ICA approach may provide a better fit to the EEG data obtained from the separate conditions to deliver specific independent right mu components with distinct topographies for each of the motor conditions. The topography of the MI condition covered the brain regions posterior to the central sulcus (P4 EEG channel); the ME condition covered the brain regions anterior to the central sulcus (C4 EEG channel), and the MO condition had broader coverage with the main activation in the premotor region (CP4 EEG channel). The source localization results also exhibited significant differences among the motor conditions. In addition, the result of single ICA decomposition resembled the result of separate ICA decomposition on the EEG data of ME with similar topographies and closely located EEG sources. This finding may further indicate that the result of single ICA decomposition may be dominated by the ME motor condition because it manifests higher data variance than the other two motor conditions.

Cervical Spinal Lesion, Completeness of Injury, Stress, and Depression Reduce the Efficiency of Mental Imagery in People With Spinal Cord Injury

OBJECTIVES

The aims of this study were to assess the relationships of (1) clinical variables (age, level of injury, time since injury [TSI], and completeness of injury) and (2) psychological variables (stress and depression) with mental imagery ability in individuals with spinal cord injury.

STUDY DESIGN

This was a cross-sectional study. Participants with spinal cord injury (N = 130) were requested to fill the Kinesthetic and Visual Imagery Questionnaire and Vividness of Motor Imagery Questionnaire. They also completed the Perceived Stress Scale and Patient Health Questionnaire 9 for the assessment of stress and depression, respectively.

RESULTS

Mental imagery scores were found to be significantly low in cervical injuries (P < 0.001) as compared with thoracic injuries (P < 0.001). Furthermore, higher levels of spinal injuries resulted in lower mental imagery scores. Completeness of injury (according to Asia Impairment Scale) also had a significant relationship (P < 0.001) with the mental imagery ability among spinal cord injury participants. Presence of stress (P < 0.001) and depression (P < 0.001) also associated with reduced efficiency of mental imagery in these individuals.

CONCLUSIONS

Injury type and psychological factors were associated with mental imagery in SCI patients. Imagery-based interventions should be designed after consideration of identified factors yielding effect on their outcomes.

TO CLAIM CME CREDITS

Complete the self-assessment activity and evaluation online at http://www.physiatry.org/JournalCME.

CME OBJECTIVES

Upon completion of this article, the reader should be able to: (1) Determine the impact of clinical variables such as level of injury, completeness and chronicity of injury on mental imagery ability in spinal cord injury; (2) Discuss the role of stress and depression on mental imagery ability in spinal cord injury; and (3) Describe the various dimensions of mental imagery ability and its variability among individuals who have spinal cord injury.

LEVEL

Advanced.

ACCREDITATION

The Association of Academic Physiatrists is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.The Association of Academic Physiatrists designates this Journal-based CME activity for a maximum of 1.0 AMA PRA Category 1 Credit(s)™. Physicians should only claim credit commensurate with the extent of their participation in the activity.

Paired associative stimulations: Novel tools for interacting with sensory and motor cortical plasticity

In the early 2000s, a novel non-invasive brain stimulation protocol, the paired associative stimulation (PAS), was introduced, allowing to induce and investigate Hebbian associative plasticity within the humans' motor system, with patterns resembling spike-timing-dependent plasticity properties found in cellular models. Since this evidence, PAS efficacy has been proved in healthy, and to a lesser extent, in clinical populations. Recently, novel 'modified' protocols targeting sensorimotor and crossmodal networks appeared in the literature. In the present work, we have reviewed recent advances using these 'modified' PAS protocols targeting sensory and motor cortical networks. To better categorize them, we propose a novel classification according to the nature of the peripheral and cortical stimulations (i.e., within-system, cross-systems, and cortico-cortical PAS). For each protocol of the categories mentioned above, we describe and discuss their main features, how they have been used to study and promote brain plasticity, and their advantages and disadvantages. Overall, current evidence suggests that these novel non-invasive brain stimulation protocols represent very promising tools to study the plastic properties of humans' sensorimotor and crossmodal networks, both in the healthy and in the damaged central nervous system.

On the characterization of cognitive tasks using activity-specific short-lived synchronization between electroencephalography channels

Accurate characterization of brain activity during a cognitive task is challenging due to the dynamically changing and the complex nature of the brain. The majority of the proposed approaches assume stationarity in brain activity and disregard the systematic timing organization among brain regions during cognitive tasks. In this study, we propose a novel cognitive activity recognition method that captures the activity-specific timing parameters from training data that elicits maximal average short-lived pairwise synchronization between electroencephalography signals. We evaluated the characterization power of the activity-specific timing parameter triplets in a motor imagery activity recognition framework. The activity-specific timing parameter triplets consist of latency of the maximally synchronized signal segments from activity onset Δt, the time lag between maximally synchronized signal segments τ, and the duration of the maximally synchronized signal segments w. We used cosine-based similarity, wavelet bi-coherence, phase-locking value, phase coherence value, linearized mutual information, and cross-correntropy to calculate the channel synchronizations at the specific timing parameters. Recognition performances as well as statistical analyses on both BCI Competition-III dataset IVa and PhysioNet Motor Movement/Imagery dataset, indicate that the inter-channel short-lived synchronization calculated using activity-specific timing parameter triplets elicit significantly distinct synchronization profiles for different motor imagery tasks and can thus reliably be used for cognitive task recognition purposes.

Alpha and high gamma phase amplitude coupling during motor imagery and weighted cross-frequency coupling to extract discriminative cross-frequency patterns

Motor imagery modulates specific neural oscillations like actual movement does. Representatively, suppression of the alpha power (e.g., event-related desynchronization [ERD]) is the typical pattern of motor imagery in the motor cortex. However, in addition to this amplitude-based feature, the coupling across frequencies includes important information about the brain functions and the existence of such complex information has been reported in various invasive studies. Yet, the interaction across multiple frequencies during motor imagery processing is still unclear and has not been widely studied, particularly concerning the non-invasive signals. In this study, we provide empirical evidence of the comodulation between the phase of alpha rhythm and the amplitude of high gamma rhythm during the motor imagery process. We used electroencephalography (EEG) in our investigation during the imagination of left- or right-hand movement recorded from 52 healthy subjects, and quantified the ERD of alpha and phase-amplitude coupling (PAC) which is a relative change of modulation index to the base line period (before the cue). As a result, we found that the coupling between the phase of alpha (8-12 Hz) and the amplitude of high gamma (70-120 Hz) and this PAC decreases during motor imagery and then rebounds to the baseline like alpha ERD (r = 0.29 to 0.42). This correlation between PAC and ERD was particularly stronger in the ipsilateral area. In addition, trials that demonstrated higher alpha power during the ready period (before the cue) showed a larger ERD during motor imagery and similarly, trials with higher modulation index during the ready period yielded a greater decrease in PAC during imagery. In the classification analysis, we found that the effective phase frequency that showed better decoding accuracy in left and right-hand imagery, varied across subjects. Motivated by result, we proposed a weighted cross-frequency coupling (WCFC) method that extracts the maximal discriminative feature by combining band power and CFC. In the evaluation, WCFC with only two electrodes yielded a performance comparable to the conventional algorithm with 64 electrodes in classifying left and right-hand motor imagery. These results indicate that the phase-amplitude frequency plays an important role in motor imagery, and that optimizing this frequency ranges is crucial for extracting information features to decode the motor imagery types.

Muscle-specific movement-phase-dependent modulation of corticospinal excitability during upper-limb motor execution and motor imagery combined with virtual action observation

Corticospinal excitability in humans can be facilitated during imagination and/or observation of upper-limb motor tasks. However, it remains unclear to what extent facilitation levels may differ from those elicited during execution of the same tasks. Twelve able-bodied individuals were recruited in this study. Motor evoked potentials (MEPs) in extensor carpi radialis (ECR) and flexor carpi radialis (FCR) muscles were elicited through transcranial magnetic stimulation of the primary motor cortex during: (i) rest; (ii) wrist extension; and (iii) wrist flexion. Responses were compared between: (1) motor imagery combined with virtual action observation (MI + AO; first-person virtual wrist movements shown on a computer display, while participants remained at rest and imagined these movements); and (2) motor execution (ME; participants extended or flexed their wrist). During MI + AO, ECR MEPs were facilitated during the extension phase but not the flexion phase, while FCR MEPs were facilitated during the flexion phase but not extension phase, compared to rest. During the ME condition, same, but greater, modulations were shown as those during MI + AO, while background muscle activities were similar in the rest phase as during extension and flexion phase in the MI + AO condition. Our results demonstrated that kinesthetic MI that included imagination and observation of virtual hands can elicit phase-dependent muscles-specific corticospinal facilitation of wrist muscles, consistent to those during actual hand extension and flexion. Moreover, we showed that MI + AO can contribute considerably to the overall corticospinal facilitation (∼20 % of ME) even without muscle contractions. These findings support utility of computer graphics-based motor imagery, which may have implications for rehabilitation and development of brain-computer interfaces.

Brain State-dependent Gain Modulation of Corticospinal Output in the Active Motor System

The communication through coherence hypothesis suggests that only coherently oscillating neuronal groups can interact effectively and predicts an intrinsic response modulation along the oscillatory rhythm. For the motor cortex (MC) at rest, the oscillatory cycle has been shown to determine the brain's responsiveness to external stimuli. For the active MC, however, the demonstration of such a phase-specific modulation of corticospinal excitability (CSE) along the rhythm cycle is still missing. Motor evoked potentials in response to transcranial magnetic stimulation (TMS) over the MC were used to probe the effect of cortical oscillations on CSE during several motor conditions. A brain-machine interface (BMI) with a robotic hand orthosis allowed investigating effects of cortical activity on CSE without the confounding effects of voluntary muscle activation. Only this BMI approach (and not active or passive hand opening alone) revealed a frequency- and phase-specific cortical modulation of CSE by sensorimotor beta-band activity that peaked once per oscillatory cycle and was independent of muscle activity. The active MC follows an intrinsic response modulation in accordance with the communication through coherence hypothesis. Furthermore, the BMI approach may facilitate and strengthen effective corticospinal communication in a therapeutic context, for example, when voluntary hand opening is no longer possible after stroke.

Enhancement of Event-Related Desynchronization in Motor Imagery Based on Transcranial Electrical Stimulation

Due to the individual differences controlling brain-computer interfaces (BCIs), the applicability and accuracy of BCIs based on motor imagery (MI-BCIs) are limited. To improve the performance of BCIs, this article examined the effect of transcranial electrical stimulation (tES) on brain activity during MI. This article designed an experimental paradigm that combines tES and MI and examined the effects of tES based on the measurements of electroencephalogram (EEG) features in MI processing, including the power spectral density (PSD) and dynamic event-related desynchronization (ERD). Finally, we investigated the effect of tES on the accuracy of MI classification using linear discriminant analysis (LDA). The results showed that the ERD of the μ and β rhythms in the left-hand MI task was enhanced after electrical stimulation with a significant effect in the tDCS group. The average classification accuracy of the transcranial alternating current stimulation (tACS) group and transcranial direct current stimulation (tDCS) group (88.19% and 89.93% respectively) were improved significantly compared to the pre-and pseudo stimulation groups. These findings indicated that tES can improve the performance and applicability of BCI and that tDCS was a potential approach in regulating brain activity and enhancing valid features during noninvasive MI-BCI processing.

Influence of iTBS on the Acute Neuroplastic Change After BCI Training

Objective: Brain-computer interface (BCI) training is becoming increasingly popular in neurorehabilitation. However, around one third subjects have difficulties in controlling BCI devices effectively, which limits the application of BCI training. Furthermore, the effectiveness of BCI training is not satisfactory in stroke rehabilitation. Intermittent theta burst stimulation (iTBS) is a powerful neural modulatory approach with strong facilitatory effects. Here, we investigated whether iTBS would improve BCI accuracy and boost the neuroplastic changes induced by BCI training. Methods: Eight right-handed healthy subjects (four males, age: 20-24) participated in this two-session study (BCI-only session and iTBS+BCI session in random order). Neuroplastic changes were measured by functional near-infrared spectroscopy (fNIRS) and single-pulse transcranial magnetic stimulation (TMS). In BCI-only session, fNIRS was measured at baseline and immediately after BCI training. In iTBS+BCI session, BCI training was followed by iTBS delivered on the right primary motor cortex (M1). Single-pulse TMS was measured at baseline and immediately after iTBS. fNIRS was measured at baseline, immediately after iTBS, and immediately after BCI training. Paired-sample t-tests were used to compare amplitudes of motor-evoked potentials, cortical silent period duration, oxygenated hemoglobin (HbO2) concentration and functional connectivity across time points, and BCI accuracy between sessions. Results: No significant difference in BCI accuracy was detected between sessions (p > 0.05). In BCI-only session, functional connectivity matrices between motor cortex and prefrontal cortex were significantly increased after BCI training (p's < 0.05). In iTBS+BCI session, amplitudes of motor-evoked potentials were significantly increased after iTBS (p's < 0.05), but no change in HbO2 concentration or functional connectivity was observed throughout the whole session (p's > 0.05). Conclusions: To our knowledge, this is the first study that investigated how iTBS targeted on M1 influences BCI accuracy and the acute neuroplastic changes after BCI training. Our results revealed that iTBS targeted on M1 did not influence BCI accuracy or facilitate the neuroplastic changes after BCI training. Therefore, M1 might not be an effective stimulation target of iTBS for the purpose of improving BCI accuracy or facilitate its effectiveness; other brain regions (i.e., prefrontal cortex) are needed to be further investigated as potentially effective stimulation targets.

Brain Patterns During Single- and Dual-Task Leg Movements

The brain is able to engage in dual tasks such as motor imagery (MI) and action observation (AO) or motor execution (ME) with action observation. In this study, we have quantitatively compared event-related desynchronization (ERD) patterns during tasks of pure MI, MI with AO (O-MI), ME, and ME with AO (O-ME) of the leg to investigate the underlying neuronal mechanisms using EEG. Subjects were instructed to imagine or perform rhythmical actions while watching a video of leg movements during O-MI and O-ME tasks; In contrast, subjects imagined and performed the leg movements without observing any video during pure MI and ME tasks. We noticed that the amplitude of ERDs from MI, O-MI, ME and O-ME sequentially increases in central regions of the brain. These quantified ERD patterns in EEG were used to study the differences of brain oscillatory changes among the four tasks. We found that ERDs in motor area were more distinct in O-MI, compared with pure MI. These results suggest that O-MI produced stronger motor activations than MI. Plus, O-ME showed significantly greater activations than ME in the beta band. O-ME has produced stronger neurophysiological effects than MI, and stronger behavioral effects than ME. These empirical results do provide convincing evidence of the dual tasks such combined MI or ME with action observation on brain pattern changes. The video of the goal-directed leg movements is most likely able to improve the ability of performing or imagining movements. O-MI and O-ME may get better and closer therapeutic effects in leg rehabilitation and motor skill training. Furthermore, the extent analysis of ERD may provide the basis for evaluating the ability of O-MI and O-ME in leg rehabilitation and motor skill training.

Brain-computer interface robotics for hand rehabilitation after stroke: a systematic review

BACKGROUND

Hand rehabilitation is core to helping stroke survivors regain activities of daily living. Recent studies have suggested that the use of electroencephalography-based brain-computer interfaces (BCI) can promote this process. Here, we report the first systematic examination of the literature on the use of BCI-robot systems for the rehabilitation of fine motor skills associated with hand movement and profile these systems from a technical and clinical perspective.

METHODS

A search for January 2010-October 2019 articles using Ovid MEDLINE, Embase, PEDro, PsycINFO, IEEE Xplore and Cochrane Library databases was performed. The selection criteria included BCI-hand robotic systems for rehabilitation at different stages of development involving tests on healthy participants or people who have had a stroke. Data fields include those related to study design, participant characteristics, technical specifications of the system, and clinical outcome measures.

RESULTS

30 studies were identified as eligible for qualitative review and among these, 11 studies involved testing a BCI-hand robot on chronic and subacute stroke patients. Statistically significant improvements in motor assessment scores relative to controls were observed for three BCI-hand robot interventions. The degree of robot control for the majority of studies was limited to triggering the device to perform grasping or pinching movements using motor imagery. Most employed a combination of kinaesthetic and visual response via the robotic device and display screen, respectively, to match feedback to motor imagery.

CONCLUSION

19 out of 30 studies on BCI-robotic systems for hand rehabilitation report systems at prototype or pre-clinical stages of development. We identified large heterogeneity in reporting and emphasise the need to develop a standard protocol for assessing technical and clinical outcomes so that the necessary evidence base on efficiency and efficacy can be developed.

Sensorimotor Functional Connectivity: A Neurophysiological Factor Related to BCI Performance

Brain-Computer Interfaces (BCIs) are systems that allow users to control devices using brain activity alone. However, the ability of participants to command BCIs varies from subject to subject. About 20% of potential users of sensorimotor BCIs do not gain reliable control of the system. The inefficiency to decode user's intentions requires the identification of neurophysiological factors determining “good” and “poor” BCI performers. One of the important neurophysiological aspects in BCI research is that the neuronal oscillations, used to control these systems, show a rich repertoire of spatial sensorimotor interactions. Considering this, we hypothesized that neuronal connectivity in sensorimotor areas would define BCI performance. Analyses for this study were performed on a large dataset of 80 inexperienced participants. They took part in a calibration and an online feedback session recorded on the same day. Undirected functional connectivity was computed over sensorimotor areas by means of the imaginary part of coherency. The results show that post- as well as pre-stimulus connectivity in the calibration recording is significantly correlated to online feedback performance in μ and feedback frequency bands. Importantly, the significance of the correlation between connectivity and BCI feedback accuracy was not due to the signal-to-noise ratio of the oscillations in the corresponding post and pre-stimulus intervals. Thus, this study demonstrates that BCI performance is not only dependent on the amplitude of sensorimotor oscillations as shown previously, but that it also relates to sensorimotor connectivity measured during the preceding training session. The presence of such connectivity between motor and somatosensory systems is likely to facilitate motor imagery, which in turn is associated with the generation of a more pronounced modulation of sensorimotor oscillations (manifested in ERD/ERS) required for the adequate BCI performance. We also discuss strategies for the up-regulation of such connectivity in order to enhance BCI performance.

State-Dependent Gain Modulation of Spinal Motor Output

Afferent somatosensory information plays a crucial role in modulating efferent motor output. A better understanding of this sensorimotor interplay may inform the design of neurorehabilitation interfaces. Current neurotechnological approaches that address motor restoration after trauma or stroke combine motor imagery (MI) and contingent somatosensory feedback, e.g., via peripheral stimulation, to induce corticospinal reorganization. These interventions may, however, change the motor output already at the spinal level dependent on alterations of the afferent input. Neuromuscular electrical stimulation (NMES) was combined with measurements of wrist deflection using a kinematic glove during either MI or rest. We investigated 360 NMES bursts to the right forearm of 12 healthy subjects at two frequencies (30 and 100 Hz) in random order. For each frequency, stimulation was assessed at nine intensities. Measuring the induced wrist deflection across different intensities allowed us to estimate the input-output curve (IOC) of the spinal motor output. MI decreased the slope of the IOC independent of the stimulation frequency. NMES with 100 Hz vs. 30 Hz decreased the threshold of the IOC. Human-machine interfaces for neurorehabilitation that combine MI and NMES need to consider bidirectional communication and may utilize the gain modulation of spinal circuitries by applying low-intensity, high-frequency stimulation.

Task-independent Electrophysiological Correlates of Motor Imagery Ability from Kinaesthetic and Visual Perspectives

Motor imagery (MI) ability is highly subjective, as indicated by the individual scores of the MIQ-3 questionnaire, and poor imagers compensate for the difficulty in performing MI with larger cerebral activations, as demonstrated by MI studies involving hands/limbs. In order to identify general, task-independent MI ability correlates, 16 volunteers were stratified with MIQ-3. The scores in the kinaesthetic (K) and 1st-person visual (V) perspectives were associated with EEG patterns obtained during K-MI and V-MI of the same complex MIQ-3 movements during these MI tasks (Spearman's correlation, significance at <0.05, SnPM corrected). EEG measures were relative to rest (relaxation, closed eyes), and based on six electrode clusters both for band spectral content and connectivity (Granger causality). Lower K-MI ability was associated with greater theta decreases during tasks in fronto-central clusters and greater inward information flow to prefrontal clusters for theta, high alpha and beta bands. On the other hand, power band relative decreases were associated with V-MI ability in fronto-central clusters for low alpha and left fronto-central and both centro-parietal clusters for beta bands. The results thus suggest different computational mechanisms for MI-V and MI-K. The association between low alpha/beta desynchronization and V-MIQ scores and between theta changes and K-MIQ scores suggest a cognitive effort with greater cerebral activation in participants with lower V-MI ability. The association between information flow to prefrontal hub and K-MI ability suggest the need for a continuous update of information to support MI-related executive functions in subjects with poor K-MI ability.

Neurofeedback of scalp bi-hemispheric EEG sensorimotor rhythm guides hemispheric activation of sensorimotor cortex in the targeted hemisphere

Oscillatory electroencephalographic (EEG) activity is associated with the excitability of cortical regions. Visual feedback of EEG-oscillations may promote sensorimotor cortical activation, but its spatial specificity is not truly guaranteed due to signal interaction among interhemispheric brain regions. Guiding spatially specific activation is important for facilitating neural rehabilitation processes. Here, we tested whether users could explicitly guide sensorimotor cortical activity to the contralateral or ipsilateral hemisphere using a spatially bivariate EEG-based neurofeedback that monitors bi-hemispheric sensorimotor cortical activities for healthy participants. Two different motor imageries (shoulder and hand MIs) were selected to see how differences in intrinsic corticomuscular projection patterns might influence activity lateralization. We showed sensorimotor cortical activities during shoulder, but not hand MI, can be brought under ipsilateral control with guided EEG-based neurofeedback. These results are compatible with neuroanatomy; shoulder muscles are innervated bihemispherically, whereas hand muscles are mostly innervated contralaterally. We demonstrate the neuroanatomically-inspired approach enables us to investigate potent neural remodeling functions that underlie EEG-based neurofeedback via a BCI.

Brain-Machine Neurofeedback: Robotics or Electrical Stimulation?

Neurotechnology such as brain-machine interfaces (BMI) are currently being investigated as training devices for neurorehabilitation, when active movements are no longer possible. When the hand is paralyzed following a stroke for example, a robotic orthosis, functional electrical stimulation (FES) or their combination may provide movement assistance; i.e., the corresponding sensory and proprioceptive neurofeedback is given contingent to the movement intention or imagination, thereby closing the sensorimotor loop. Controlling these devices may be challenging or even frustrating. Direct comparisons between these two feedback modalities (robotics vs. FES) with regard to the workload they pose for the user are, however, missing. Twenty healthy subjects controlled a BMI by kinesthetic motor imagery of finger extension. Motor imagery-related sensorimotor desynchronization in the EEG beta frequency-band (17–21 Hz) was turned into passive opening of the contralateral hand by a robotic orthosis or FES in a randomized, cross-over block design. Mental demand, physical demand, temporal demand, performance, effort, and frustration level were captured with the NASA Task Load Index (NASA-TLX) questionnaire by comparing these workload components to each other (weights), evaluating them individually (ratings), and estimating the respective combinations (adjusted workload ratings). The findings were compared to the task-related aspects of active hand movement with EMG feedback. Furthermore, both feedback modalities were compared with regard to their BMI performance. Robotic and FES feedback had similar workloads when weighting and rating the different components. For both robotics and FES, mental demand was the most relevant component, and higher than during active movement with EMG feedback. The FES task led to significantly more physical (p = 0.0368) and less temporal demand (p = 0.0403) than the robotic task in the adjusted workload ratings. Notably, the FES task showed a physical demand 2.67 times closer to the EMG task, but a mental demand 6.79 times closer to the robotic task. On average, significantly more onsets were reached during the robotic as compared to the FES task (17.22 onsets, SD = 3.02 vs. 16.46, SD = 2.94 out of 20 opportunities; p = 0.016), even though there were no significant differences between the BMI classification accuracies of the conditions (p = 0.806; CI = −0.027 to −0.034). These findings may inform the design of neurorehabilitation interfaces toward human-centered hardware for a more natural bidirectional interaction and acceptance by the user.

Scaling Analysis of Phase Fluctuations of Brain Networks in Dynamic Constrained Object Manipulation

In this study, we investigated the dynamic properties of oscillatory activities in the scalp electro-encephalographs (EEGs) of 20 participants involved in a novel dynamic manipulating task using a physical interface and a virtual feedback. The complexity of such a task a rises from the unexpected relationship between the magnitude of the motion and the feedback. The characterization of complex patterns arising from EEG is an important problem in identifying different mental intentions. We proposed a scaling analysis of phase fluctuation in the scalp EEG to discriminate the network states related to different EEG patterns, which correspond to manipulating the task with right or left movement intention. These intentions are generated while the participant is engaged in such a complex task. The phase characterization method was used to calculate the instantaneous phase from the operational EEG. Then, functional brain networks (FBNs) of 20 subjects based on the task-related EEG were constructed by phase synchronization. The degree features representing the structures and scaling components of brain networks are sensitive to the EEG patterns with left or right motor intention. The correlation between features and mental intentions was investigated by discriminant analysis. For 20 subjects, the average accuracy of state detection is [Formula: see text], and the average mean-squared error (MSE) is [Formula: see text]. The brain state depicted by the results is related to high awareness, the phase characterization is of the effectiveness in EEG processing and FBN construction and the difference of control intentions can be explored by the phase characterization method. This finding may be relevant to understanding some neuronal mechanisms underlying the attention and some applications of closed-loop control for the safety operation of tools.

G-Causality Brain Connectivity Differences of Finger Movements between Motor Execution and Motor Imagery

Motor imagery is one of the classical paradigms which have been used in brain-computer interface and motor function recovery. Finger movement-based motor execution is a complex biomechanical architecture and a crucial task for establishing most complicated and natural activities in daily life. Some patients may suffer from alternating hemiplegia after brain stroke and lose their ability of motor execution. Fortunately, the ability of motor imagery might be preserved independently and worked as a backdoor for motor function recovery. The efficacy of motor imagery for achieving significant recovery for the motor cortex after brain stroke is still an open question. In this study, we designed a new paradigm to investigate the neural mechanism of thirty finger movements in two scenarios: motor execution and motor imagery. Eleven healthy participants performed or imagined thirty hand gestures twice based on left and right finger movements. The electroencephalogram (EEG) signal for each subject during sixty trials left and right finger motor execution and imagery were recorded during our proposed experimental paradigm. The Granger causality (G-causality) analysis method was employed to analyze the brain connectivity and its strength between contralateral premotor, motor, and sensorimotor areas. Highest numbers for G-causality trials of 37 ± 7.3, 35.5 ± 8.8, 36.3 ± 10.3, and 39.2 ± 9.0 and lowest Granger causality coefficients of 9.1 ± 3.2, 10.9 ± 3.7, 13.2 ± 0.6, and 13.4 ± 0.6 were achieved from the premotor to motor area during execution/imagination tasks of right and left finger movements, respectively. These results provided a new insight into motor execution and motor imagery based on hand gestures, which might be useful to build a new biomarker of finger motor recovery for partially or even completely plegic patients. Furthermore, a significant difference of the G-causality trial number was observed during left finger execution/imagery and right finger imagery, but it was not observed during the right finger execution phase. Significant difference of the G-causality coefficient was observed during left finger execution and imagery, but it was not observed during right finger execution and imagery phases. These results suggested that different MI-based brain motor function recovery strategies should be taken for right-hand and left-hand patients after brain stroke.

The Sensitivity of Single-Trial Mu-Suppression Detection for Motor Imagery Performance as Compared to Motor Execution and Motor Observation Performance

Motor imagery (MI) has been widely used to operate brain-computer interface (BCI) systems for rehabilitation and some life assistive devices. However, the current performance of an MI-based BCI cannot fully meet the needs of its in-field applications. Most of the BCIs utilizing a generalized feature for all participants have been found to greatly hamper the efficacy of the BCI system. Hence, some attempts have made on the exploration of subject-dependent parameters, but it remains challenging to enhance BCI performance as expected. To this end, in this study, we used the independent component analysis (ICA), which has been proved capable of isolating the pure motor-related component from non-motor-related brain processes and artifacts and extracting the common motor-related component across MI, motor execution (ME), and motor observation (MO) conditions. Then, a sliding window approach was used to detect significant mu-suppression from the baseline using the electroencephalographic (EEG) alpha power time course and, thus, the success rate of the mu-suppression detection could be assessed on a single-trial basis. By comparing the success rates using different parameters, we further quantified the extent of the improvement in each motor condition to evaluate the effectiveness of both generalized and individualized parameters. The results showed that in ME condition, the success rate under individualized latency and that under generalized latency was 90.0% and 77.75%, respectively; in MI condition, the success rate was 74.14% for individual latency and 58.47% for generalized latency, and in MO condition, the success rate was 67.89% and 61.26% for individual and generalized latency, respectively. As can be seen, the success rate in each motor condition was significantly improved by utilizing an individualized latency compared to that using the generalized latency. Moreover, the comparison of the individualized window latencies for the mu-suppression detection across different runs of the same participant as well as across different participants showed that the window latency was significantly more consistent in the intra-subject than in the inter-subject settings. As a result, we proposed that individualizing the latency for detecting the mu-suppression feature for each participant might be a promising attempt to improve the MI-based BCI performance.

Does hypnotic assessment predict the functional equivalence between motor imagery and action?

Motor imagery is influenced by individual and contextual factors. We investigated whether the psychophysiological trait of hypnotisability modulates its subjective experience and cortical correlates similarly to what was previously shown for head postures mental images. EEG was acquired in 18 high (highs) and 15 low (lows) hypnotizable subjects (Stanford Hypnotic Susceptibility Scale, A). The experimental conditions were: baseline, a complex arm/hand movement, visual (1st person) and kinesthetic imagery of the movement. After each imagery condition, participants scored the vividness and easeness of their performance and their ability to mantain the requested modality of imagery. Subjective reports, chronometric visual/kinesthetic indices, absolute beta and fronto-central midline alpha powers were analyzed. Findings confirmed earlier reports of better kinestetic imagery ability in highs than in lows and better visual than kinesthetic imagery in lows, as well as smaller restructuring of the cortical activity in highs than in lows, during all tasks. Also, they show that hypnotisability accounts for most of the correlations between brain regions for both alpha and beta changes. Thus, imagined and actual movements were less demanding processes in highs at subjective and cortical levels. Finally, hypnotic assessment assists to plan personalized mental training for neuro-rehabilitation and sports and predict their efficacy.

Different oscillatory entrainment of cortical networks during motor imagery and neurofeedback in right and left handers

Volitional modulation and neurofeedback of sensorimotor oscillatory activity is currently being evaluated as a strategy to facilitate motor restoration following stroke. Knowledge on the interplay between this regional brain self-regulation, distributed network entrainment and handedness is, however, limited. In a randomized cross-over design, twenty-one healthy subjects (twelve right-handers [RH], nine left-handers [LH]) performed kinesthetic motor imagery of left (48 trials) and right finger extension (48 trials). A brain-machine interface turned event-related desynchronization in the beta frequency-band (16-22 Hz) during motor imagery into passive hand opening by a robotic orthosis. Thereby, every participant subsequently activated either the dominant (DH) or non-dominant hemisphere (NDH) to control contralateral hand opening. The task-related cortical networks were studied with electroencephalography. The magnitude of the induced oscillatory modulation range in the sensorimotor cortex was independent of both handedness (RH, LH) and hemispheric specialization (DH, NDH). However, the regional beta-band modulation was associated with different alpha-band networks in RH and LH: RH presented a stronger inter-hemispheric connectivity, while LH revealed a stronger intra-hemispheric interaction. Notably, these distinct network entrainments were independent of hemispheric specialization. In healthy subjects, sensorimotor beta-band activity can be robustly modulated by motor imagery and proprioceptive feedback in both hemispheres independent of handedness. However, right and left handers show different oscillatory entrainment of cortical alpha-band networks during neurofeedback. This finding may inform neurofeedback interventions in future to align them more precisely with the underlying physiology.

Combined endogenous and exogenous disinhibition of intracortical circuits augments plasticity induction in the human motor cortex

BACKGROUND

Motor imagery (MI) engages cortical areas in the human brain similar to motor practice. Corticospinal excitability (CSE) is facilitated during but not after MI practice. We hypothesized that lasting CSE changes could be achieved by associatively pairing this endogenous modulation with exogenous stimulation of the same intracortical circuits.

METHODS

We combined MI with a disinhibition protocol (DIS) targeting intracortical circuits by paired-pulse repetitive transcranial magnetic stimulation in one main and three subsequent experiments. The follow-up experiments were applied to increase effects, e.g., by individualizing inter-stimulus intervals, adding neuromuscular stimulation and expanding the intervention period. CSE was captured during (online) and after (offline) the interventions via input-output changes and cortical maps of motor evoked potentials. A total of 35 healthy subjects (mean age 26.1 ± 2.6 years, 20 females) participated in this study.

RESULTS

A short intervention (48 stimuli within ∼90s) increased CSE. This plasticity developed rapidly, was associative (with MI_on, but not MI_off or REST) and persisted beyond the intervention period. Follow-up experiments revealed the relevance of individualizing inter-stimulus intervals and of consistent inter-burst periods for online and offline effects, respectively. Expanding this combined MI/DIS intervention to 480 stimuli amplified the sustainability of CSE changes. When concurrent neuromuscular electrical stimulation was applied, the plasticity induction was cancelled.

CONCLUSIONS

This novel associative stimulation protocol augmented plasticity induction in the human motor cortex within a remarkably short period of time and in the absence of active movements. The combination of endogenous and exogenous disinhibition of intracortical circuits may provide a therapeutic backdoor when active movements are no longer possible, e.g., for hand paralysis after stroke.

Reaction Time Predicts Brain-Computer Interface Aptitude

There is evidence that 15-30% of the general population cannot effectively operate brain-computer interfaces (BCIs). Thus the BCI performance predictors are critically required to pre-screen participants. Current neurophysiological and psychological tests either require complicated equipment or suffer from subjectivity. Thus, a simple and objective BCI performance predictor is desirable. Neurofeedback (NFB) training involves performing a cognitive task (motor imagery) instructed via sensory stimuli and re-adjusted through ongoing real-time feedback. A simple reaction time (SRT) test reflects the time required for a subject to respond to a defined stimulus. Thus, we postulated that individuals with shorter reaction times operate a BCI with rapidly updated feedback better than individuals with longer reaction times. Furthermore, we investigated how changing the feedback update interval (FUI), i.e., modification of the feedback provision frequency, affects the correlation between the SRT and BCI performance. Ten participants attended four NFB sessions with FUIs of 16, 24, 48, and 96 ms in a randomized order. We found that: 1) SRT is correlated with the BCI performance with FUIs of 16 and 96 ms; 2) good and poor performers elicit stronger ERDs and control BCIs more effectively (i.e., produced larger information transfer rates) with 16 and 96 ms FUIs, respectively. Our findings suggest that SRT may be used as a simple and objective surrogate for BCI aptitude with FUIs of 16 and 96 ms. It also implies that the FUI customization according to participants SRT measure may enhance the BCI performance.

Flow and Immersion in Video Games: The Aftermath of a Conceptual Challenge

One of the most pleasurable aspects of video games is their ability to induce immersive experiences. However, there appears to be a tentative conceptualization of what an immersive experience is. In this short review, we specifically focus on the terms of flow and immersion, as they are the most widely used and applied definitions in the video game literature, whilst their differences remain disputable. We critically review the concepts separately and proceed with a comparison on their proposed differences. We conclude that immersion and flow do not substantially differ in current studies and that more evidence is needed to justify their separation.

Muscle-selective disinhibition of corticomotor representations using a motor imagery-based brain-computer interface

Bridging between brain activity and machine control, brain-computer interface (BCI) can be employed to activate distributed neural circuits implicated in a specific aspect of motor control. Using a motor imagery-based BCI paradigm, we previously found a disinhibition within the primary motor cortex contralateral to the imagined movement, as evidenced by event-related desynchronization (ERD) of oscillatory cortical activity. Yet it is unclear whether this BCI approach does selectively facilitate corticomotor representations targeted by the imagery. To address this question, we used brain state-dependent transcranial magnetic stimulation while participants performed kinesthetic motor imagery of wrist movements with their right hand and received online visual feedback of the ERD. Single and paired-pulse magnetic stimulation were given to the left primary motor cortex at a low or high level of ERD to assess intracortical excitability. While intracortical facilitation showed no modulation by ERD, short-latency intracortical inhibition was reduced the higher the ERD. Intracortical disinhibition was only found in the agonist muscle targeted by motor imagery at high ERD level, but not in the antagonist muscle. Single pulse motor-evoked potential was also increased the higher the ERD. However, at high ERD level, this facilitatory effect on overall corticospinal excitability was not selective to the agonist muscle. Analogous results were found in two independent experiments, in which participants either performed kinesthetic motor imagery of wrist extension or flexion. Our results showed that motor imagery-based BCI can selectively disinhibit the corticomotor output to the agonist muscle, enabling effector-specific training in patients with motor paralysis.

Active Physical Practice Followed by Mental Practice Using BCI-Driven Hand Exoskeleton: A Pilot Trial for Clinical Effectiveness and Usability

Appropriately combining mental practice (MP) and physical practice (PP) in a poststroke rehabilitation is critical for ensuring a substantially positive rehabilitation outcome. Here, we present a rehabilitation protocol incorporating a separate active PP stage followed by MP stage, using a hand exoskeleton and brain-computer interface (BCI). The PP stage was mediated by a force sensor feedback-based assist-as-needed control strategy, whereas the MP stage provided BCI-based multimodal neurofeedback combining anthropomorphic visual feedback and proprioceptive feedback of the impaired hand extension attempt. A six week long clinical trial was conducted on four hemiparetic stroke patients (screened out of 16) with a left-hand disability. The primary outcome, motor functional recovery, was measured in terms of changes in grip-strength (GS) and action research arm test (ARAT) scores; whereas the secondary outcome, usability of the system was measured in terms of changes in mood, fatigue, and motivation on a visual-analog-scale. A positive rehabilitative outcome was found as the group mean changes from the baseline in the GS and ARAT were +6.38 kg and +5.66 accordingly. The VAS scale measurements also showed betterment in mood ( 1.38), increased motivation (+2.10) and reduced fatigue (0.98) as compared to the baseline. Thus, the proposed neurorehabilitation protocol is found to be promising both in terms of clinical effectiveness and usability.

Novel tDCS montage favors lower limb motor imagery detection

This work studies a novel transcranial direct current stimulation (tDCS) montage to improve a brain-machine interface (BMI) lower limb motor imagery detection. The tDCS montage is composed by two anodes and one cathode. One anode is located over the motor cortex and the other one over the cerebellum. Ten healthy subjects participated in this experiment. They were randomly separated into two groups: sham, which received a fake stimulation, and active tDCS, which received a real stimulation. Each subject was experimented on five consecutive days. Results pointed out that there was a significant difference $(p < 0 .05)$ in the classification accuracy between the sham and the active tDCS group. On each of the five days of the experiment the active tDCS group achieved better accuracy results than the sham group: 4%, 10%, 10%, 9% and 7% higher respectively.

Recruitment of Additional Corticospinal Pathways in the Human Brain with State-Dependent Paired Associative Stimulation

Standard brain stimulation protocols modify human motor cortex excitability by modulating the gain of the activated corticospinal pathways. However, the restoration of motor function following lesions of the corticospinal tract requires also the recruitment of additional neurons to increase the net corticospinal output. For this purpose, we investigated a novel protocol based on brain state-dependent paired associative stimulation.Motor imagery (MI)-related electroencephalography was recorded in healthy males and females for brain state-dependent control of both cortical and peripheral stimulation in a brain-machine interface environment. State-dependency was investigated with concurrent, delayed, and independent stimulation relative to the MI task. Specifically, sensorimotor event-related desynchronization (ERD) in the β-band (16-22 Hz) triggered peripheral stimulation through passive hand opening by a robotic orthosis and transcranial magnetic stimulation to the respective cortical motor representation, either synchronously or subsequently. These MI-related paradigms were compared with paired cortical and peripheral input applied independent of the brain state. Cortical stimulation resulted in a significant increase in corticospinal excitability only when applied brain state-dependently and synchronously to peripheral input. These gains were resistant to a depotentiation task, revealed a nonlinear evolution of plasticity, and were mediated via the recruitment of additional corticospinal neurons rather than via synchronization of neuronal firing. Recruitment of additional corticospinal pathways may be achieved when cortical and peripheral inputs are applied concurrently, and during β-ERD. These findings resemble a gating mechanism and are potentially important for developing closed-loop brain stimulation for the treatment of hand paralysis following lesions of the corticospinal tract.SIGNIFICANCE STATEMENT The activity state of the motor system influences the excitability of corticospinal pathways to external input. State-dependent interventions harness this property to increase the connectivity between motor cortex and muscles. These stimulation protocols modulate the gain of the activated pathways, but not the overall corticospinal recruitment. In this study, a brain-machine interface paired peripheral stimulation through passive hand opening with transcranial magnetic stimulation to the respective cortical motor representation during volitional β-band desynchronization. Cortical stimulation resulted in the recruitment of additional corticospinal pathways, but only when applied brain state-dependently and synchronously to peripheral input. These effects resemble a gating mechanism and may be important for the restoration of motor function following lesions of the corticospinal tract.

Motor imagery learning across a sequence of trials in stroke patients

PURPOSE

In brain-computer interfaces (BCIs), electrical brain signals during motor imagery are utilized as commands connecting the brain to a computer. To use BCI in patients with stroke, unique brain signal changes should be characterized during motor imagery process. This study aimed to examine the trial-dependent motor-imagery-related activities in stroke patients.

METHODS

During the recording of electroencephalography (EEG) signals, 12 chronic stroke patients and 11 age-matched healthy controls performed motor imagery finger tapping at 1.3 sec intervals. Trial-dependent brain signal changes were assessed by analysis of the mu and beta bands.

RESULTS

Neuronal activity in healthy controls was observed over bilateral hemispheres at the mu and beta bands regardless of changes in the trials, whereas neuronal activity in stroke patients was mainly seen over the ipsilesional hemisphere at the beta band. With progression to repeated trials, healthy controls displayed a decrease in cortical activity in the contralateral hemisphere at the mu band and in bilateral hemispheres at the beta band. In contrast, stroke patients showed a decreasing trend in cortical activity only over the ipsilesional hemisphere at the beta band.

CONCLUSIONS

Trial-dependent changes during motor imagery learning presented in a different manner in stroke patients. Understanding motor imagery learning in stroke patients is crucial for enhancing the effectiveness of motor-imagery-based BCIs.

Virtual and Actual Humanoid Robot Control with Four-Class Motor-Imagery-Based Optical Brain-Computer Interface

Motor-imagery tasks are a popular input method for controlling brain-computer interfaces (BCIs), partially due to their similarities to naturally produced motor signals. The use of functional near-infrared spectroscopy (fNIRS) in BCIs is still emerging and has shown potential as a supplement or replacement for electroencephalography. However, studies often use only two or three motor-imagery tasks, limiting the number of available commands. In this work, we present the results of the first four-class motor-imagery-based online fNIRS-BCI for robot control. Thirteen participants utilized upper- and lower-limb motor-imagery tasks (left hand, right hand, left foot, and right foot) that were mapped to four high-level commands (turn left, turn right, move forward, and move backward) to control the navigation of a simulated or real robot. A significant improvement in classification accuracy was found between the virtual-robot-based BCI (control of a virtual robot) and the physical-robot BCI (control of the DARwIn-OP humanoid robot). Differences were also found in the oxygenated hemoglobin activation patterns of the four tasks between the first and second BCI. These results corroborate previous findings that motor imagery can be improved with feedback and imply that a four-class motor-imagery-based fNIRS-BCI could be feasible with sufficient subject training.

Comparison of Brain Activation during Motor Imagery and Motor Movement Using fNIRS

Motor-activity-related mental tasks are widely adopted for brain-computer interfaces (BCIs) as they are a natural extension of movement intention, requiring no training to evoke brain activity. The ideal BCI aims to eliminate neuromuscular movement, making motor imagery tasks, or imagined actions with no muscle movement, good candidates. This study explores cortical activation differences between motor imagery and motor execution for both upper and lower limbs using functional near-infrared spectroscopy (fNIRS). Four simple finger- or toe-tapping tasks (left hand, right hand, left foot, and right foot) were performed with both motor imagery and motor execution and compared to resting state. Significant activation was found during all four motor imagery tasks, indicating that they can be detected via fNIRS. Motor execution produced higher activation levels, a faster response, and a different spatial distribution compared to motor imagery, which should be taken into account when designing an imagery-based BCI. When comparing left versus right, upper limb tasks are the most clearly distinguishable, particularly during motor execution. Left and right lower limb activation patterns were found to be highly similar during both imagery and execution, indicating that higher resolution imaging, advanced signal processing, or improved subject training may be required to reliably distinguish them.

Constraints and Adaptation of Closed-Loop Neuroprosthetics for Functional Restoration

Closed-loop neuroprosthetics aim to compensate for lost function, e.g., by controlling external devices such as prostheses or wheelchairs. Such assistive approaches seek to maximize speed and classification accuracy for high-dimensional control. More recent approaches use similar technology, but aim to restore lost motor function in the long term. To achieve this goal, restorative neuroprosthetics attempt to facilitate motor re-learning and to strengthen damaged and/or alternative neural connections on the basis of neurofeedback training within rehabilitative environments. Such a restorative approach requires reinforcement learning of self-modulated brain activity which is considered to be beneficial for functional rehabilitation, e.g., improvement of β-power modulation over sensorimotor areas for post-stroke movement restoration. Patients with motor impairments, however, may also have a compromised ability for motor task-related regulation of the targeted brain activity. This would affect the estimation of feature weights and hence the classification accuracy of the feedback device. This, in turn, can frustrate the patients and compromise their motor learning. Furthermore, the feedback training may even become erroneous when unconstrained classifier adaptation-which is often used in assistive approaches-is also applied in this rehabilitation context. In conclusion, the conceptual switch from assistance toward restoration necessitates a methodological paradigm shift from classification accuracy toward instructional efficiency. Furthermore, a constrained feature space, a priori regularized feature weights, and difficulty adaptation present key elements of restorative brain interfaces. These factors need, therefore, to be addressed within a therapeutic framework to facilitate reinforcement learning of brain self-regulation for restorative purposes.

Proprioceptive Feedback Facilitates Motor Imagery-Related Operant Learning of Sensorimotor β-Band Modulation

Motor imagery (MI) activates the sensorimotor system independent of actual movements and might be facilitated by neurofeedback. Knowledge on the interaction between feedback modality and the involved frequency bands during MI-related brain self-regulation is still scarce. Previous studies compared the cortical activity during the MI task with concurrent feedback (MI with feedback condition) to cortical activity during the relaxation task where no feedback was provided (relaxation without feedback condition). The observed differences might, therefore, be related to either the task or the feedback. A proper comparison would necessitate studying a relaxation condition with feedback and a MI task condition without feedback as well. Right-handed healthy subjects performed two tasks, i.e., MI and relaxation, in alternating order. Each of the tasks (MI vs. relaxation) was studied with and without feedback. The respective event-driven oscillatory activity, i.e., sensorimotor desynchronization (during MI) or synchronization (during relaxation), was rewarded with contingent feedback. Importantly, feedback onset was delayed to study the task-related cortical activity in the absence of feedback provision during the delay period. The reward modality was alternated every 15 trials between proprioceptive and visual feedback. Proprioceptive input was superior to visual input to increase the range of task-related spectral perturbations in the α- and β-band, and was necessary to consistently achieve MI-related sensorimotor desynchronization (ERD) significantly below baseline. These effects occurred in task periods without feedback as well. The increased accuracy and duration of learned brain self-regulation achieved in the proprioceptive condition was specific to the β-band. MI-related operant learning of brain self-regulation is facilitated by proprioceptive feedback and mediated in the sensorimotor β-band.

EEG based zero-phase phase-locking value (PLV) and effects of spatial filtering during actual movement

Phase-locking value (PLV) is a well-known feature in sensorimotor rhythm (SMR) based BCI. Zero-phase PLV has not been explored because it is generally regarded as the result of volume conduction. Because spatial filters are often used to enhance the amplitude (square root of band power (BP)) feature and attenuate volume conduction, they are frequently applied as pre-processing methods when computing PLV. However, the effects of spatial filtering on PLV are ambiguous. Therefore, this article aims to explore whether zero-phase PLV is meaningful and how this is influenced by spatial filtering. Based on archival EEG data of left and right hand movement tasks for 32 subjects, we compared BP and PLV feature using data with and without pre-processing by a large Laplacian. Results showed that using ear-referenced data, zero-phase PLV provided unique information independent of BP for task prediction which was not explained by volume conduction and was significantly decreased when a large Laplacian was applied. In other words, the large Laplacian eliminated the useful information in zero-phase PLV for task prediction suggesting that it contains effects of both amplitude and phase. Therefore, zero-phase PLV may have functional significance beyond volume conduction. The interpretation of spatial filtering may be complicated by effects of phase.

Closed-Loop Task Difficulty Adaptation during Virtual Reality Reach-to-Grasp Training Assisted with an Exoskeleton for Stroke Rehabilitation

Stroke patients with severe motor deficits of the upper extremity may practice rehabilitation exercises with the assistance of a multi-joint exoskeleton. Although this technology enables intensive task-oriented training, it may also lead to slacking when the assistance is too supportive. Preserving the engagement of the patients while providing “assistance-as-needed” during the exercises, therefore remains an ongoing challenge. We applied a commercially available seven degree-of-freedom arm exoskeleton to provide passive gravity compensation during task-oriented training in a virtual environment. During this 4-week pilot study, five severely affected chronic stroke patients performed reach-to-grasp exercises resembling activities of daily living. The subjects received virtual reality feedback from their three-dimensional movements. The level of difficulty for the exercise was adjusted by a performance-dependent real-time adaptation algorithm. The goal of this algorithm was the automated improvement of the range of motion. In the course of 20 training and feedback sessions, this unsupervised adaptive training concept led to a progressive increase of the virtual training space (p < 0.001) in accordance with the subjects' abilities. This learning curve was paralleled by a concurrent improvement of real world kinematic parameters, i.e., range of motion (p = 0.008), accuracy of movement (p = 0.01), and movement velocity (p < 0.001). Notably, these kinematic gains were paralleled by motor improvements such as increased elbow movement (p = 0.001), grip force (p < 0.001), and upper extremity Fugl-Meyer-Assessment score from 14.3 ± 5 to 16.9 ± 6.1 (p = 0.026). Combining gravity-compensating assistance with adaptive closed-loop feedback in virtual reality provides customized rehabilitation environments for severely affected stroke patients. This approach may facilitate motor learning by progressively challenging the subject in accordance with the individual capacity for functional restoration. It might be necessary to apply concurrent restorative interventions to translate these improvements into relevant functional gains of severely motor impaired patients in activities of daily living.

What Turns Assistive into Restorative Brain-Machine Interfaces?

Brain-machine interfaces (BMI) may support motor impaired patients during activities of daily living by controlling external devices such as prostheses (assistive BMI). Moreover, BMIs are applied in conjunction with robotic orthoses for rehabilitation of lost motor function via neurofeedback training (restorative BMI). Using assistive BMI in a rehabilitation context does not automatically turn them into restorative devices. This perspective article suggests key features of restorative BMI and provides the supporting evidence: In summary, BMI may be referred to as restorative tools when demonstrating subsequently (i) operant learning and progressive evolution of specific brain states/dynamics, (ii) correlated modulations of functional networks related to the therapeutic goal, (iii) subsequent improvement in a specific task, and (iv) an explicit correlation between the modulated brain dynamics and the achieved behavioral gains. Such findings would provide the rationale for translating BMI-based interventions into clinical settings for reinforcement learning and motor rehabilitation following stroke.

Neuromuscular Plasticity: Disentangling Stable and Variable Motor Maps in the Human Sensorimotor Cortex

Motor maps acquired with transcranial magnetic stimulation (TMS) are evolving as a biomarker for monitoring disease progression or the effects of therapeutic interventions. High test-retest reliability of this technique for long observation periods is therefore required to differentiate daily or weekly fluctuations from stable plastic reorganization of corticospinal connectivity. In this study, a novel projection, interpolation, and coregistration technique, which considers the individual gyral anatomy, was applied in healthy subjects for biweekly acquired TMS motor maps over a period of twelve weeks. The intraclass correlation coefficient revealed long-term reliability of motor maps with relevant interhemispheric differences. The sensorimotor cortex and nonprimary motor areas of the dominant hemisphere showed more extended and more stable corticospinal connectivity. Long-term correlations of the MEP amplitudes at each stimulation site revealed mosaic-like clusters of consistent corticospinal excitability. The resting motor threshold, centre of gravity, and mean MEPs across all TMS sites, as highly reliable cortical map parameters, could be disentangled from more variable parameters such as MEP area and volume. Cortical TMS motor maps provide high test-retest reliability for long-term monitoring when analyzed with refined techniques. They may guide restorative interventions which target dormant corticospinal connectivity for neurorehabilitation.

Hybrid Neuroprosthesis for the Upper Limb: Combining Brain-Controlled Neuromuscular Stimulation with a Multi-Joint Arm Exoskeleton

Brain-machine interface-controlled (BMI) neurofeedback training aims to modulate cortical physiology and is applied during neurorehabilitation to increase the responsiveness of the brain to subsequent physiotherapy. In a parallel line of research, robotic exoskeletons are used in goal-oriented rehabilitation exercises for patients with severe motor impairment to extend their range of motion (ROM) and the intensity of training. Furthermore, neuromuscular electrical stimulation (NMES) is applied in neurologically impaired patients to restore muscle strength by closing the sensorimotor loop. In this proof-of-principle study, we explored an integrated approach for providing assistance as needed to amplify the task-related ROM and the movement-related brain modulation during rehabilitation exercises of severely impaired patients. For this purpose, we combined these three approaches (BMI, NMES, and exoskeleton) in an integrated neuroprosthesis and studied the feasibility of this device in seven severely affected chronic stroke patients who performed wrist flexion and extension exercises while receiving feedback via a virtual environment. They were assisted by a gravity-compensating, seven degree-of-freedom exoskeleton which was attached to the paretic arm. NMES was applied to the wrist extensor and flexor muscles during the exercises and was controlled by a hybrid BMI based on both sensorimotor cortical desynchronization (ERD) and electromyography (EMG) activity. The stimulation intensity was individualized for each targeted muscle and remained subthreshold, i.e., induced no overt support. The hybrid BMI controlled the stimulation significantly better than the offline analyzed ERD (p = 0.028) or EMG (p = 0.021) modality alone. Neuromuscular stimulation could be well integrated into the exoskeleton-based training and amplified both the task-related ROM (p = 0.009) and the movement-related brain modulation (p = 0.019). Combining a hybrid BMI with neuromuscular stimulation and antigravity assistance augments upper limb function and brain activity during rehabilitation exercises and may thus provide a novel restorative framework for severely affected stroke patients.

Do Motor Imagery Performances Depend on the Side of the Lesion at the Acute Stage of Stroke?

Motor imagery has been considered a substitute for overt motor execution to study post-stroke motor recovery. However, motor imagery abilities at the acute stage (<3 weeks) are poorly known. The aim of this study was to compare explicit and implicit motor imagery abilities in stroke patients and healthy subjects, correlate them with motor function, and investigate the role of right or left hemisphere lesions on performance. Twenty-four stroke patients at the acute stage and 24 age- and gender-matched healthy volunteers performed implicit (Hand Laterality Judgment Task) and explicit (number of imagined/executed hand movements) motor imagery tasks and a clinical motor assessment. Differences between healthy subjects and patients as well as the impact of lesion side on motor imagery were studied using ANOVA. We analyzed the relationship between motor executed and imagined movements (temporal congruence) using Pearson correlations. Our study shows that for implicit imagery, stroke patients had slower reaction times [RTs, t(46) = 1.7, p = 0.02] and higher error rates for the affected hand [t(46) = 3.7, p < 0.01] yet shared similar characteristics [angle effect: F(1,46) = 30.8, p ≤ 0.0001] with respect to healthy subjects. For the unaffected hand, right-sided stroke patients had a higher error rate and similar RTs whereas left sided stroke had higher RTs but similar error rate than healthy subjects. For explicit imagery, patients exhibited bilateral deficits compared to healthy subjects in the executed and imagined condition (p < 0.0001). Patients and healthy subjects exhibited a temporal congruence between executed and imagined movements (p ≤ 0.04) except for right-sided strokes who had no correlation for both hands. When using motor imagery as a tool for upper limb rehabilitation early after stroke, caution must be taken related to the side of the lesion.