Reinforcement learning for adaptive threshold control of restorative brain-computer interfaces: a Bayesian simulation

A reinforcement learning-based optimal control approach for managing an elective surgery backlog after pandemic disruption

Contagious disease pandemics, such as COVID-19, can cause hospitals around the world to delay nonemergent elective surgeries, which results in a large surgery backlog. To develop an operational solution for providing patients timely surgical care with limited health care resources, this study proposes a stochastic control process-based method that helps hospitals make operational recovery plans to clear their surgery backlog and restore surgical activity safely. The elective surgery backlog recovery process is modeled by a general discrete-time queueing network system, which is formulated by a Markov decision process. A scheduling optimization algorithm based on the piecewise decaying [Formula: see text]-greedy reinforcement learning algorithm is proposed to make dynamic daily surgery scheduling plans considering newly arrived patients, waiting time and clinical urgency. The proposed method is tested through a set of simulated dataset, and implemented on an elective surgery backlog that built up in one large general hospital in China after the outbreak of COVID-19. The results show that, compared with the current policy, the proposed method can effectively and rapidly clear the surgery backlog caused by a pandemic while ensuring that all patients receive timely surgical care. These results encourage the wider adoption of the proposed method to manage surgery scheduling during all phases of a public health crisis.

Oscillatory neurofeedback networks and poststroke rehabilitative potential in severely impaired stroke patients

Motor restoration after severe stroke is often limited. However, some of the severely impaired stroke patients may still have a rehabilitative potential. Biomarkers that identify these patients are sparse. Eighteen severely impaired chronic stroke patients with a lack of volitional finger extension participated in an EEG study. During sixty-six trials of kinesthetic motor imagery, a brain-machine interface turned event-related beta-band desynchronization of the ipsilesional sensorimotor cortex into opening of the paralyzed hand by a robotic orthosis. A subgroup of eight patients participated in a subsequent four-week rehabilitation training. Changes of the movement extent were captured with sensors which objectively quantified even discrete improvements of wrist movement. Albeit with the same motor impairment level, patients could be differentiated into two groups, i.e., with and without task-related increase of bilateral cortico-cortical phase synchronization between frontal/premotor and parietal areas. This fronto-parietal integration (FPI) was associated with a significantly higher volitional beta modulation range in the ipsilesional sensorimotor cortex. Following the four-week training, patients with FPI showed significantly higher improvement in wrist movement than those without FPI. Moreover, only the former group improved significantly in the upper extremity Fugl-Meyer-Assessment score. Neurofeedback-related long-range oscillatory coherence may differentiate severely impaired stroke patients with regard to their rehabilitative potential, a finding that needs to be confirmed in larger patient cohorts.

Modern views of machine learning for precision psychiatry

In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.

Rewiring Cortico-Muscular Control in the Healthy and Poststroke Human Brain with Proprioceptive β-Band Neurofeedback

In severely affected stroke survivors, cortico-muscular control is disturbed and volitional upper limb movements often absent. Mental rehearsal of the impaired movement in conjunction with sensory feedback provision are suggested as promising rehabilitation exercises. Knowledge about the underlying neural processes, however, remains vague. In male and female chronic stroke patients with hand paralysis, a brain-computer interface controlled a robotic orthosis and turned sensorimotor β-band desynchronization during motor imagery (MI) of finger extension into contingent hand opening. Healthy control subjects performed the same task and received the same proprioceptive feedback with a robotic orthosis or visual feedback only. Only when proprioceptive feedback was provided, cortico-muscular coherence (CMC) increased with a predominant information flow from the sensorimotor cortex to the finger extensors. This effect (1) was specific to the β frequency band, (2) transferred to a motor task (MT), (3) was proportional to subsequent corticospinal excitability (CSE) and correlated with behavioral changes in the (4) healthy and (5) poststroke condition; notably, MI-related enhancement of β-band CMC in the ipsilesional premotor cortex correlated with motor improvements after the intervention. In the healthy and injured human nervous system, synchronized activation of motor-related cortical and spinal neural pools facilitates, in accordance with the communication-through-coherence hypothesis, cortico-spinal communication and may, thereby, be therapeutically relevant for functional restoration after stroke, when voluntary movements are no longer possible.SIGNIFICANCE STATEMENT This study provides insights into the neural processes that transfer effects of brain-computer interface neurofeedback to subsequent motor behavior. Specifically, volitional control of cortical oscillations and proprioceptive feedback enhances both cortical activity and behaviorally relevant connectivity to the periphery in a topographically circumscribed and frequency-specific way. This enhanced cortico-muscular control can be induced in the healthy and poststroke brain. Thereby, activating the motor cortex with mental rehearsal of the impaired movement and closing the loop by robot-assisted feedback synchronizes ipsilesional premotor cortex and spinal neural pools in the β frequency band. This facilitates, in accordance with the communication-through-coherence hypothesis, cortico-spinal communication and may, thereby, be therapeutically relevant for functional restoration after stroke, when voluntary movements are no longer possible.

Adaptive Cognitive Training with Reinforcement Learning

Computer-assisted cognitive training can help patients affected by several illnesses alleviate their cognitive deficits or healthy people improve their mental performance. In most computer-based systems, training sessions consist of graded exercises, which should ideally be able to gradually improve the trainee’s cognitive functions. Indeed, adapting the difficulty of the exercises to how individuals perform in their execution is crucial to improve the effectiveness of cognitive training activities. In this article, we propose the use of reinforcement learning (RL) to learn how to automatically adapt the difficulty of computerized exercises for cognitive training. In our approach, trainees’ performance in performed exercises is used as a reward to learn a policy that changes over time the values of the parameters that determine exercise difficulty. We illustrate a method to be initially used to learn difficulty-variation policies tailored for specific categories of trainees, and then to refine these policies for single individuals. We present the results of two user studies that provide evidence for the effectiveness of our method: a first study, in which a student category policy obtained via RL was found to have better effects on the cognitive function than a standard baseline training that adopts a mechanism to vary the difficulty proposed by neuropsychologists, and a second study, demonstrating that adding an RL-based individual customization further improves the training process.

A Charge Balanced Neural Stimulator Silicon Chip for Human-Machine Interface

This paper proposes a neural stimulator silicon chip design with an improved charge balancing technology. The proposed neural stimulation integrated circuit (IC) uses two charge balancing modules including synchronous charge detection module and short-time pulse insertion module. The synchronous charge detection module is designed based on a current splitter with ultra-small output current and an integrator circuit for neural stimulation pulse width control, which greatly reduces the residual charge remained on the electrode-tissue interface. The short-time pulse insertion module is designed based on the electrode voltage detection and compensation current control, which further reduces the accumulated residual charge and keeps the electrode voltage within a safety range of ±25 mV during multiple stimulation cycles. Finally, this neural stimulator is implemented in TSMC 0.18-μm CMOS process technology, and the chip function is tested and verified in both experiments with the electrode-tissue RC model and the PBS saline solution environment. The measurement result shows the neural stimulator chip achieves improved charge balancing with the residual charge smaller than 0.95 nC, which is the lowest compared to the traditional neural stimulator chips.

Neurosurgery and artificial intelligence

Neurosurgeons receive extensive and lengthy training to equip themselves with various technical skills, and neurosurgery require a great deal of pre-, intra- and postoperative clinical data collection, decision making, care and recovery. The last decade has seen a significant increase in the importance of artificial intelligence (AI) in neurosurgery. AI can provide a great promise in neurosurgery by complementing neurosurgeons' skills to provide the best possible interventional and noninterventional care for patients by enhancing diagnostic and prognostic outcomes in clinical treatment and help neurosurgeons with decision making during surgical interventions to improve patient outcomes. Furthermore, AI is playing a pivotal role in the production, processing and storage of clinical and experimental data. AI usage in neurosurgery can also reduce the costs associated with surgical care and provide high-quality healthcare to a broader population. Additionally, AI and neurosurgery can build a symbiotic relationship where AI helps to push the boundaries of neurosurgery, and neurosurgery can help AI to develop better and more robust algorithms. This review explores the role of AI in interventional and noninterventional aspects of neurosurgery during pre-, intra- and postoperative care, such as diagnosis, clinical decision making, surgical operation, prognosis, data acquisition, and research within the neurosurgical arena.

Brain State-dependent Gain Modulation of Corticospinal Output in the Active Motor System

The communication through coherence hypothesis suggests that only coherently oscillating neuronal groups can interact effectively and predicts an intrinsic response modulation along the oscillatory rhythm. For the motor cortex (MC) at rest, the oscillatory cycle has been shown to determine the brain's responsiveness to external stimuli. For the active MC, however, the demonstration of such a phase-specific modulation of corticospinal excitability (CSE) along the rhythm cycle is still missing. Motor evoked potentials in response to transcranial magnetic stimulation (TMS) over the MC were used to probe the effect of cortical oscillations on CSE during several motor conditions. A brain-machine interface (BMI) with a robotic hand orthosis allowed investigating effects of cortical activity on CSE without the confounding effects of voluntary muscle activation. Only this BMI approach (and not active or passive hand opening alone) revealed a frequency- and phase-specific cortical modulation of CSE by sensorimotor beta-band activity that peaked once per oscillatory cycle and was independent of muscle activity. The active MC follows an intrinsic response modulation in accordance with the communication through coherence hypothesis. Furthermore, the BMI approach may facilitate and strengthen effective corticospinal communication in a therapeutic context, for example, when voluntary hand opening is no longer possible after stroke.

The value-complexity trade-off for reinforcement learning based brain-computer interfaces

OBJECTIVE

One of the recent developments in the field of brain-computer interfaces (BCI) is the reinforcement learning (RL) based BCI paradigm, which uses neural error responses as the reward feedback on the agent's action. While having several advantages over motor imagery based BCI, the reliability of RL-BCI is critically dependent on the decoding accuracy of noisy neural error signals. A principled method is needed to optimally handle this inherent noise under general conditions.

APPROACH

By determining a trade-off between the expected value and the informational cost of policies, the info-RL (IRL) algorithm provides optimal low-complexity policies, which are robust under noisy reward conditions and achieve the maximal obtainable value. In this work we utilize the IRL algorithm to characterize the maximal obtainable value under different noise levels, which in turn is used to extract the optimal robust policy for each noise level.

MAIN RESULTS

Our simulation results of a setting with Gaussian noise show that the complexity level of the optimal policy is dependent on the reward magnitude but not on the reward variance, whereas the variance determines whether a lower complexity solution is favorable or not. We show how this analysis can be utilized to select optimal robust policies for an RL-BCI and demonstrate its use on EEG data.

SIGNIFICANCE

We propose here a principled method to determine the optimal policy complexity of an RL problem with a noisy reward, which we argue is particularly useful for RL-based BCI paradigms. This framework may be used to minimize initial training time and allow for a more dynamic and robust shared control between the agent and the operator under different conditions.

Reinforcement learning for intelligent healthcare applications: A survey

Discovering new treatments and personalizing existing ones is one of the major goals of modern clinical research. In the last decade, Artificial Intelligence (AI) has enabled the realization of advanced intelligent systems able to learn about clinical treatments and discover new medical knowledge from the huge amount of data collected. Reinforcement Learning (RL), which is a branch of Machine Learning (ML), has received significant attention in the medical community since it has the potentiality to support the development of personalized treatments in accordance with the more general precision medicine vision. This report presents a review of the role of RL in healthcare by investigating past work, and highlighting any limitations and possible future contributions.

Brain-Machine Neurofeedback: Robotics or Electrical Stimulation?

Neurotechnology such as brain-machine interfaces (BMI) are currently being investigated as training devices for neurorehabilitation, when active movements are no longer possible. When the hand is paralyzed following a stroke for example, a robotic orthosis, functional electrical stimulation (FES) or their combination may provide movement assistance; i.e., the corresponding sensory and proprioceptive neurofeedback is given contingent to the movement intention or imagination, thereby closing the sensorimotor loop. Controlling these devices may be challenging or even frustrating. Direct comparisons between these two feedback modalities (robotics vs. FES) with regard to the workload they pose for the user are, however, missing. Twenty healthy subjects controlled a BMI by kinesthetic motor imagery of finger extension. Motor imagery-related sensorimotor desynchronization in the EEG beta frequency-band (17–21 Hz) was turned into passive opening of the contralateral hand by a robotic orthosis or FES in a randomized, cross-over block design. Mental demand, physical demand, temporal demand, performance, effort, and frustration level were captured with the NASA Task Load Index (NASA-TLX) questionnaire by comparing these workload components to each other (weights), evaluating them individually (ratings), and estimating the respective combinations (adjusted workload ratings). The findings were compared to the task-related aspects of active hand movement with EMG feedback. Furthermore, both feedback modalities were compared with regard to their BMI performance. Robotic and FES feedback had similar workloads when weighting and rating the different components. For both robotics and FES, mental demand was the most relevant component, and higher than during active movement with EMG feedback. The FES task led to significantly more physical (p = 0.0368) and less temporal demand (p = 0.0403) than the robotic task in the adjusted workload ratings. Notably, the FES task showed a physical demand 2.67 times closer to the EMG task, but a mental demand 6.79 times closer to the robotic task. On average, significantly more onsets were reached during the robotic as compared to the FES task (17.22 onsets, SD = 3.02 vs. 16.46, SD = 2.94 out of 20 opportunities; p = 0.016), even though there were no significant differences between the BMI classification accuracies of the conditions (p = 0.806; CI = −0.027 to −0.034). These findings may inform the design of neurorehabilitation interfaces toward human-centered hardware for a more natural bidirectional interaction and acceptance by the user.

The combination of brain-computer interfaces and artificial intelligence: applications and challenges

Brain-computer interfaces (BCIs) have shown great prospects as real-time bidirectional links between living brains and actuators. Artificial intelligence (AI), which can advance the analysis and decoding of neural activity, has turbocharged the field of BCIs. Over the past decade, a wide range of BCI applications with AI assistance have emerged. These "smart" BCIs including motor and sensory BCIs have shown notable clinical success, improved the quality of paralyzed patients' lives, expanded the athletic ability of common people and accelerated the evolution of robots and neurophysiological discoveries. However, despite technological improvements, challenges remain with regard to the long training periods, real-time feedback, and monitoring of BCIs. In this article, the authors review the current state of AI as applied to BCIs and describe advances in BCI applications, their challenges and where they could be headed in the future.

A Systematic Review of Closed-Loop Feedback Techniques in Sleep Studies-Related Issues and Future Directions

Advances in computer processing technology have enabled researchers to analyze real-time brain activity and build real-time closed-loop paradigms. In many fields, the effectiveness of these closed-loop protocols has proven to be better than that of the simple open-loop paradigms. Recently, sleep studies have attracted much attention as one possible application of closed-loop paradigms. To date, several studies that used closed-loop paradigms have been reported in the sleep-related literature and recommend a closed-loop feedback system to enhance specific brain activity during sleep, which leads to improvements in sleep's effects, such as memory consolidation. However, to the best of our knowledge, no report has reviewed and discussed the detailed technical issues that arise in designing sleep closed-loop paradigms. In this paper, we reviewed the most recent reports on sleep closed-loop paradigms and offered an in-depth discussion of some of their technical issues. We found 148 journal articles strongly related with 'sleep and stimulation' and reviewed 20 articles on closed-loop feedback sleep studies. We focused on human sleep studies conducting any modality of feedback stimulation. Then we introduced the main component of the closed-loop system and summarized several open-source libraries, which are widely used in closed-loop systems, with step-by-step guidelines for closed-loop system implementation for sleep. Further, we proposed future directions for sleep research with closed-loop feedback systems, which provide some insight into closed-loop feedback systems.

Individualized Neurofeedback Training May Help Achieve Long-Term Improvement of Working Memory in Children With ADHD

Background. Children with attention deficit hyperactivity disorder (ADHD) may suffer from working memory deficits, which can adversely affect their academic performance. Neurofeedback training may enhance working memory and provide a solution to this problem. Aim. To investigate the effect of frequency-neurofeedback on working memory in children with ADHD and to check if the effect is long-lasting. Method. Forty-eight children with ADHD (aged 6-12 years) were randomly assigned either to a neurofeedback with training parameters chosen to take into account each child's peak alpha frequency (PAF) or to a waiting list control group. Each trained child underwent 19-channel electroencephalography (EEG). All children had average intelligence and none were receiving treatment, such as medications, for ADHD. Prior to the training, MOXO and n-back tests were performed. Next, neurofeedback training sessions with frequency bands for theta and beta ranges determined using each child's PAF were carried out for 10 weeks. Training parameters were set to increase amplitudes in the low beta range and to decrease amplitudes in the theta and high beta frequency ranges. The n-back test was performed again right after the training and then a year later. Results. During the first n-back test, children from both groups responded correctly to more than 43% of the stimuli. During the second test, children from the waiting list responded correctly to an average of 49% of the stimuli, while children who underwent the neurofeedback training were correct, on average, 69% of the time (significant difference, P < .001). During the third n-back test a year later, children from the waiting list responded correctly to 53% of the stimuli, while those who underwent the neurofeedback training responded correctly to nearly 71%. Conclusion. This study found a statistically significant improvement in a measure of working memory in children who did 10 to 12 sessions of neurofeedback training with training frequency ranges for theta and beta defined according to each child's PAF. The beneficial effects were still present a year after training.

Different oscillatory entrainment of cortical networks during motor imagery and neurofeedback in right and left handers

Volitional modulation and neurofeedback of sensorimotor oscillatory activity is currently being evaluated as a strategy to facilitate motor restoration following stroke. Knowledge on the interplay between this regional brain self-regulation, distributed network entrainment and handedness is, however, limited. In a randomized cross-over design, twenty-one healthy subjects (twelve right-handers [RH], nine left-handers [LH]) performed kinesthetic motor imagery of left (48 trials) and right finger extension (48 trials). A brain-machine interface turned event-related desynchronization in the beta frequency-band (16-22 Hz) during motor imagery into passive hand opening by a robotic orthosis. Thereby, every participant subsequently activated either the dominant (DH) or non-dominant hemisphere (NDH) to control contralateral hand opening. The task-related cortical networks were studied with electroencephalography. The magnitude of the induced oscillatory modulation range in the sensorimotor cortex was independent of both handedness (RH, LH) and hemispheric specialization (DH, NDH). However, the regional beta-band modulation was associated with different alpha-band networks in RH and LH: RH presented a stronger inter-hemispheric connectivity, while LH revealed a stronger intra-hemispheric interaction. Notably, these distinct network entrainments were independent of hemispheric specialization. In healthy subjects, sensorimotor beta-band activity can be robustly modulated by motor imagery and proprioceptive feedback in both hemispheres independent of handedness. However, right and left handers show different oscillatory entrainment of cortical alpha-band networks during neurofeedback. This finding may inform neurofeedback interventions in future to align them more precisely with the underlying physiology.

Combined endogenous and exogenous disinhibition of intracortical circuits augments plasticity induction in the human motor cortex

BACKGROUND

Motor imagery (MI) engages cortical areas in the human brain similar to motor practice. Corticospinal excitability (CSE) is facilitated during but not after MI practice. We hypothesized that lasting CSE changes could be achieved by associatively pairing this endogenous modulation with exogenous stimulation of the same intracortical circuits.

METHODS

We combined MI with a disinhibition protocol (DIS) targeting intracortical circuits by paired-pulse repetitive transcranial magnetic stimulation in one main and three subsequent experiments. The follow-up experiments were applied to increase effects, e.g., by individualizing inter-stimulus intervals, adding neuromuscular stimulation and expanding the intervention period. CSE was captured during (online) and after (offline) the interventions via input-output changes and cortical maps of motor evoked potentials. A total of 35 healthy subjects (mean age 26.1 ± 2.6 years, 20 females) participated in this study.

RESULTS

A short intervention (48 stimuli within ∼90s) increased CSE. This plasticity developed rapidly, was associative (with MI_on, but not MI_off or REST) and persisted beyond the intervention period. Follow-up experiments revealed the relevance of individualizing inter-stimulus intervals and of consistent inter-burst periods for online and offline effects, respectively. Expanding this combined MI/DIS intervention to 480 stimuli amplified the sustainability of CSE changes. When concurrent neuromuscular electrical stimulation was applied, the plasticity induction was cancelled.

CONCLUSIONS

This novel associative stimulation protocol augmented plasticity induction in the human motor cortex within a remarkably short period of time and in the absence of active movements. The combination of endogenous and exogenous disinhibition of intracortical circuits may provide a therapeutic backdoor when active movements are no longer possible, e.g., for hand paralysis after stroke.

A New Frontier: The Convergence of Nanotechnology, Brain Machine Interfaces, and Artificial Intelligence

A confluence of technological capabilities is creating an opportunity for machine learning and artificial intelligence (AI) to enable "smart" nanoengineered brain machine interfaces (BMI). This new generation of technologies will be able to communicate with the brain in ways that support contextual learning and adaptation to changing functional requirements. This applies to both invasive technologies aimed at restoring neurological function, as in the case of neural prosthesis, as well as non-invasive technologies enabled by signals such as electroencephalograph (EEG). Advances in computation, hardware, and algorithms that learn and adapt in a contextually dependent way will be able to leverage the capabilities that nanoengineering offers the design and functionality of BMI. We explore the enabling capabilities that these devices may exhibit, why they matter, and the state of the technologies necessary to build them. We also discuss a number of open technical challenges and problems that will need to be solved in order to achieve this.

Reaction Time Predicts Brain-Computer Interface Aptitude

There is evidence that 15-30% of the general population cannot effectively operate brain-computer interfaces (BCIs). Thus the BCI performance predictors are critically required to pre-screen participants. Current neurophysiological and psychological tests either require complicated equipment or suffer from subjectivity. Thus, a simple and objective BCI performance predictor is desirable. Neurofeedback (NFB) training involves performing a cognitive task (motor imagery) instructed via sensory stimuli and re-adjusted through ongoing real-time feedback. A simple reaction time (SRT) test reflects the time required for a subject to respond to a defined stimulus. Thus, we postulated that individuals with shorter reaction times operate a BCI with rapidly updated feedback better than individuals with longer reaction times. Furthermore, we investigated how changing the feedback update interval (FUI), i.e., modification of the feedback provision frequency, affects the correlation between the SRT and BCI performance. Ten participants attended four NFB sessions with FUIs of 16, 24, 48, and 96 ms in a randomized order. We found that: 1) SRT is correlated with the BCI performance with FUIs of 16 and 96 ms; 2) good and poor performers elicit stronger ERDs and control BCIs more effectively (i.e., produced larger information transfer rates) with 16 and 96 ms FUIs, respectively. Our findings suggest that SRT may be used as a simple and objective surrogate for BCI aptitude with FUIs of 16 and 96 ms. It also implies that the FUI customization according to participants SRT measure may enhance the BCI performance.

Applications of Deep Learning and Reinforcement Learning to Biological Data

Rapid advances in hardware-based technologies during the past decades have opened up new possibilities for life scientists to gather multimodal data in various application domains, such as omics, bioimaging, medical imaging, and (brain/body)-machine interfaces. These have generated novel opportunities for development of dedicated data-intensive machine learning techniques. In particular, recent research in deep learning (DL), reinforcement learning (RL), and their combination (deep RL) promise to revolutionize the future of artificial intelligence. The growth in computational power accompanied by faster and increased data storage, and declining computing costs have already allowed scientists in various fields to apply these techniques on data sets that were previously intractable owing to their size and complexity. This paper provides a comprehensive survey on the application of DL, RL, and deep RL techniques in mining biological data. In addition, we compare the performances of DL techniques when applied to different data sets across various application domains. Finally, we outline open issues in this challenging research area and discuss future development perspectives.

Effect of Different Movement Speed Modes on Human Action Observation: An EEG Study

Action observation (AO) generates event-related desynchronization (ERD) suppressions in the human brain by activating partial regions of the human mirror neuron system (hMNS). The activation of the hMNS response to AO remains controversial for several reasons. Therefore, this study investigated the activation of the hMNS response to a speed factor of AO by controlling the movement speed modes of a humanoid robot's arm movements. Since hMNS activation is reflected by ERD suppressions, electroencephalography (EEG) with BCI analysis methods for ERD suppressions were used as the recording and analysis modalities. Six healthy individuals were asked to participate in experiments comprising five different conditions. Four incremental-speed AO tasks and a motor imagery (MI) task involving imaging of the same movement were presented to the individuals. Occipital and sensorimotor regions were selected for BCI analyses. The experimental results showed that hMNS activation was higher in the occipital region but more robust in the sensorimotor region. Since the attended information impacts the activations of the hMNS during AO, the pattern of hMNS activations first rises and subsequently falls to a stable level during incremental-speed modes of AO. The discipline curves suggested that a moderate speed within a decent inter-stimulus interval (ISI) range produced the highest hMNS activations. Since a brain computer/machine interface (BCI) builds a path-way between human and computer/mahcine, the discipline curves will help to construct BCIs made by patterns of action observation (AO-BCI). Furthermore, a new method for constructing non-invasive brain machine brain interfaces (BMBIs) with moderate AO-BCI and motor imagery BCI (MI-BCI) was inspired by this paper.

Recruitment of Additional Corticospinal Pathways in the Human Brain with State-Dependent Paired Associative Stimulation

Standard brain stimulation protocols modify human motor cortex excitability by modulating the gain of the activated corticospinal pathways. However, the restoration of motor function following lesions of the corticospinal tract requires also the recruitment of additional neurons to increase the net corticospinal output. For this purpose, we investigated a novel protocol based on brain state-dependent paired associative stimulation.Motor imagery (MI)-related electroencephalography was recorded in healthy males and females for brain state-dependent control of both cortical and peripheral stimulation in a brain-machine interface environment. State-dependency was investigated with concurrent, delayed, and independent stimulation relative to the MI task. Specifically, sensorimotor event-related desynchronization (ERD) in the β-band (16-22 Hz) triggered peripheral stimulation through passive hand opening by a robotic orthosis and transcranial magnetic stimulation to the respective cortical motor representation, either synchronously or subsequently. These MI-related paradigms were compared with paired cortical and peripheral input applied independent of the brain state. Cortical stimulation resulted in a significant increase in corticospinal excitability only when applied brain state-dependently and synchronously to peripheral input. These gains were resistant to a depotentiation task, revealed a nonlinear evolution of plasticity, and were mediated via the recruitment of additional corticospinal neurons rather than via synchronization of neuronal firing. Recruitment of additional corticospinal pathways may be achieved when cortical and peripheral inputs are applied concurrently, and during β-ERD. These findings resemble a gating mechanism and are potentially important for developing closed-loop brain stimulation for the treatment of hand paralysis following lesions of the corticospinal tract.SIGNIFICANCE STATEMENT The activity state of the motor system influences the excitability of corticospinal pathways to external input. State-dependent interventions harness this property to increase the connectivity between motor cortex and muscles. These stimulation protocols modulate the gain of the activated pathways, but not the overall corticospinal recruitment. In this study, a brain-machine interface paired peripheral stimulation through passive hand opening with transcranial magnetic stimulation to the respective cortical motor representation during volitional β-band desynchronization. Cortical stimulation resulted in the recruitment of additional corticospinal pathways, but only when applied brain state-dependently and synchronously to peripheral input. These effects resemble a gating mechanism and may be important for the restoration of motor function following lesions of the corticospinal tract.

Therapeutic Applications of BCI Technologies

Brain-computer interface (BCI) technology can restore communication and control to people who are severely paralyzed. There has been speculation that this technology might also be useful for a variety of diverse therapeutic applications. This survey considers possible ways that BCI technology can be applied to motor rehabilitation following stroke, Parkinson's disease, and psychiatric disorders. We consider potential neural signals as well as the design and goals of BCI-based therapeutic applications. These diverse applications all share a reliance on neuroimaging and signal processing technologies. At the same time, each of these potential applications presents a series of unique challenges.

Plasticity of premotor cortico-muscular coherence in severely impaired stroke patients with hand paralysis

Motor recovery in severely impaired stroke patients is often very limited. To refine therapeutic interventions for regaining motor control in this patient group, the functionally relevant mechanisms of neuronal plasticity need to be detected. Cortico-muscular coherence (CMC) may provide physiological and topographic insights to achieve this goal. Synchronizing limb movements to motor-related brain activation is hypothesized to reestablish cortico-motor control indexed by CMC. In the present study, right-handed, chronic stroke patients with right-hemispheric lesions and left hand paralysis participated in a four-week training for their left upper extremity. A brain-robot interface turned event-related beta-band desynchronization of the lesioned sensorimotor cortex during kinesthetic motor-imagery into the opening of the paralyzed hand by a robotic orthosis. Simultaneous MEG/EMG recordings and individual models from MRIs were used for CMC detection and source reconstruction of cortico-muscular connectivity to the affected finger extensors before and after the training program. The upper extremity-FMA of the patients improved significantly from 16.23 ± 6.79 to 19.52 ± 7.91 (p = 0.0015). All patients showed significantly increased CMC in the beta frequency-band, with a distributed, bi-hemispheric pattern and considerable inter-individual variability. The location of CMC changes was not correlated to the severity of the motor impairment, the motor improvement or the lesion volume. Group analysis of the cortical overlap revealed a common feature in all patients following the intervention: a significantly increased level of ipsilesional premotor CMC that extended from the superior to the middle and inferior frontal gyrus, along with a confined area of increased CMC in the contralesional premotor cortex. In conclusion, functionally relevant modulations of CMC can be detected in patients with long-term, severe motor deficits after a brain-robot assisted rehabilitation training. Premotor beta-band CMC may serve as a biomarker and therapeutic target for novel treatment approaches in this patient group.

Constraints and Adaptation of Closed-Loop Neuroprosthetics for Functional Restoration

Closed-loop neuroprosthetics aim to compensate for lost function, e.g., by controlling external devices such as prostheses or wheelchairs. Such assistive approaches seek to maximize speed and classification accuracy for high-dimensional control. More recent approaches use similar technology, but aim to restore lost motor function in the long term. To achieve this goal, restorative neuroprosthetics attempt to facilitate motor re-learning and to strengthen damaged and/or alternative neural connections on the basis of neurofeedback training within rehabilitative environments. Such a restorative approach requires reinforcement learning of self-modulated brain activity which is considered to be beneficial for functional rehabilitation, e.g., improvement of β-power modulation over sensorimotor areas for post-stroke movement restoration. Patients with motor impairments, however, may also have a compromised ability for motor task-related regulation of the targeted brain activity. This would affect the estimation of feature weights and hence the classification accuracy of the feedback device. This, in turn, can frustrate the patients and compromise their motor learning. Furthermore, the feedback training may even become erroneous when unconstrained classifier adaptation-which is often used in assistive approaches-is also applied in this rehabilitation context. In conclusion, the conceptual switch from assistance toward restoration necessitates a methodological paradigm shift from classification accuracy toward instructional efficiency. Furthermore, a constrained feature space, a priori regularized feature weights, and difficulty adaptation present key elements of restorative brain interfaces. These factors need, therefore, to be addressed within a therapeutic framework to facilitate reinforcement learning of brain self-regulation for restorative purposes.

Feedback for reinforcement learning based brain-machine interfaces using confidence metrics

OBJECTIVE

For brain-machine interfaces (BMI) to be used in activities of daily living by paralyzed individuals, the BMI should be as autonomous as possible. One of the challenges is how the feedback is extracted and utilized in the BMI. Our long-term goal is to create autonomous BMIs that can utilize an evaluative feedback from the brain to update the decoding algorithm and use it intelligently in order to adapt the decoder. In this study, we show how to extract the necessary evaluative feedback from a biologically realistic (synthetic) source, use both the quantity and the quality of the feedback, and how that feedback information can be incorporated into a reinforcement learning (RL) controller architecture to maximize its performance.

APPROACH

Motivated by the perception-action-reward cycle (PARC) in the brain which links reward for cognitive decision making and goal-directed behavior, we used a reward-based RL architecture named Actor-Critic RL as the model. Instead of using an error signal towards building an autonomous BMI, we envision to use a reward signal from the nucleus accumbens (NAcc) which plays a key role in the linking of reward to motor behaviors. To deal with the complexity and non-stationarity of biological reward signals, we used a confidence metric which was used to indicate the degree of feedback accuracy. This confidence was added to the Actor's weight update equation in the RL controller architecture. If the confidence was high (>0.2), the BMI decoder used this feedback to update its parameters. However, when the confidence was low, the BMI decoder ignored the feedback and did not update its parameters. The range between high confidence and low confidence was termed as the 'ambiguous' region. When the feedback was within this region, the BMI decoder updated its weight at a lower rate than when fully confident, which was decided by the confidence. We used two biologically realistic models to generate synthetic data for MI (Izhikevich model) and NAcc (Humphries model) to validate proposed controller architecture.

MAIN RESULTS

In this work, we show how the overall performance of the BMI was improved by using a threshold close to the decision boundary to reject erroneous feedback. Additionally, we show the stability of the system improved when the feedback was used with a threshold.

SIGNIFICANCE

The result of this study is a step towards making BMIs autonomous. While our method is not fully autonomous, the results demonstrate that extensive training times necessary at the beginning of each BMI session can be significantly decreased. In our approach, decoder training time was only limited to 10 trials in the first BMI session. Subsequent sessions used previous session weights to initialize the decoder. We also present a method where the use of a threshold can be applied to any decoder with a feedback signal that is less than perfect so that erroneous feedback can be avoided and the stability of the system can be increased.

Proprioceptive Feedback Facilitates Motor Imagery-Related Operant Learning of Sensorimotor β-Band Modulation

Motor imagery (MI) activates the sensorimotor system independent of actual movements and might be facilitated by neurofeedback. Knowledge on the interaction between feedback modality and the involved frequency bands during MI-related brain self-regulation is still scarce. Previous studies compared the cortical activity during the MI task with concurrent feedback (MI with feedback condition) to cortical activity during the relaxation task where no feedback was provided (relaxation without feedback condition). The observed differences might, therefore, be related to either the task or the feedback. A proper comparison would necessitate studying a relaxation condition with feedback and a MI task condition without feedback as well. Right-handed healthy subjects performed two tasks, i.e., MI and relaxation, in alternating order. Each of the tasks (MI vs. relaxation) was studied with and without feedback. The respective event-driven oscillatory activity, i.e., sensorimotor desynchronization (during MI) or synchronization (during relaxation), was rewarded with contingent feedback. Importantly, feedback onset was delayed to study the task-related cortical activity in the absence of feedback provision during the delay period. The reward modality was alternated every 15 trials between proprioceptive and visual feedback. Proprioceptive input was superior to visual input to increase the range of task-related spectral perturbations in the α- and β-band, and was necessary to consistently achieve MI-related sensorimotor desynchronization (ERD) significantly below baseline. These effects occurred in task periods without feedback as well. The increased accuracy and duration of learned brain self-regulation achieved in the proprioceptive condition was specific to the β-band. MI-related operant learning of brain self-regulation is facilitated by proprioceptive feedback and mediated in the sensorimotor β-band.

Closed-Loop Task Difficulty Adaptation during Virtual Reality Reach-to-Grasp Training Assisted with an Exoskeleton for Stroke Rehabilitation

Stroke patients with severe motor deficits of the upper extremity may practice rehabilitation exercises with the assistance of a multi-joint exoskeleton. Although this technology enables intensive task-oriented training, it may also lead to slacking when the assistance is too supportive. Preserving the engagement of the patients while providing “assistance-as-needed” during the exercises, therefore remains an ongoing challenge. We applied a commercially available seven degree-of-freedom arm exoskeleton to provide passive gravity compensation during task-oriented training in a virtual environment. During this 4-week pilot study, five severely affected chronic stroke patients performed reach-to-grasp exercises resembling activities of daily living. The subjects received virtual reality feedback from their three-dimensional movements. The level of difficulty for the exercise was adjusted by a performance-dependent real-time adaptation algorithm. The goal of this algorithm was the automated improvement of the range of motion. In the course of 20 training and feedback sessions, this unsupervised adaptive training concept led to a progressive increase of the virtual training space (p < 0.001) in accordance with the subjects' abilities. This learning curve was paralleled by a concurrent improvement of real world kinematic parameters, i.e., range of motion (p = 0.008), accuracy of movement (p = 0.01), and movement velocity (p < 0.001). Notably, these kinematic gains were paralleled by motor improvements such as increased elbow movement (p = 0.001), grip force (p < 0.001), and upper extremity Fugl-Meyer-Assessment score from 14.3 ± 5 to 16.9 ± 6.1 (p = 0.026). Combining gravity-compensating assistance with adaptive closed-loop feedback in virtual reality provides customized rehabilitation environments for severely affected stroke patients. This approach may facilitate motor learning by progressively challenging the subject in accordance with the individual capacity for functional restoration. It might be necessary to apply concurrent restorative interventions to translate these improvements into relevant functional gains of severely motor impaired patients in activities of daily living.

What Turns Assistive into Restorative Brain-Machine Interfaces?

Brain-machine interfaces (BMI) may support motor impaired patients during activities of daily living by controlling external devices such as prostheses (assistive BMI). Moreover, BMIs are applied in conjunction with robotic orthoses for rehabilitation of lost motor function via neurofeedback training (restorative BMI). Using assistive BMI in a rehabilitation context does not automatically turn them into restorative devices. This perspective article suggests key features of restorative BMI and provides the supporting evidence: In summary, BMI may be referred to as restorative tools when demonstrating subsequently (i) operant learning and progressive evolution of specific brain states/dynamics, (ii) correlated modulations of functional networks related to the therapeutic goal, (iii) subsequent improvement in a specific task, and (iv) an explicit correlation between the modulated brain dynamics and the achieved behavioral gains. Such findings would provide the rationale for translating BMI-based interventions into clinical settings for reinforcement learning and motor rehabilitation following stroke.

Hybrid Neuroprosthesis for the Upper Limb: Combining Brain-Controlled Neuromuscular Stimulation with a Multi-Joint Arm Exoskeleton

Brain-machine interface-controlled (BMI) neurofeedback training aims to modulate cortical physiology and is applied during neurorehabilitation to increase the responsiveness of the brain to subsequent physiotherapy. In a parallel line of research, robotic exoskeletons are used in goal-oriented rehabilitation exercises for patients with severe motor impairment to extend their range of motion (ROM) and the intensity of training. Furthermore, neuromuscular electrical stimulation (NMES) is applied in neurologically impaired patients to restore muscle strength by closing the sensorimotor loop. In this proof-of-principle study, we explored an integrated approach for providing assistance as needed to amplify the task-related ROM and the movement-related brain modulation during rehabilitation exercises of severely impaired patients. For this purpose, we combined these three approaches (BMI, NMES, and exoskeleton) in an integrated neuroprosthesis and studied the feasibility of this device in seven severely affected chronic stroke patients who performed wrist flexion and extension exercises while receiving feedback via a virtual environment. They were assisted by a gravity-compensating, seven degree-of-freedom exoskeleton which was attached to the paretic arm. NMES was applied to the wrist extensor and flexor muscles during the exercises and was controlled by a hybrid BMI based on both sensorimotor cortical desynchronization (ERD) and electromyography (EMG) activity. The stimulation intensity was individualized for each targeted muscle and remained subthreshold, i.e., induced no overt support. The hybrid BMI controlled the stimulation significantly better than the offline analyzed ERD (p = 0.028) or EMG (p = 0.021) modality alone. Neuromuscular stimulation could be well integrated into the exoskeleton-based training and amplified both the task-related ROM (p = 0.009) and the movement-related brain modulation (p = 0.019). Combining a hybrid BMI with neuromuscular stimulation and antigravity assistance augments upper limb function and brain activity during rehabilitation exercises and may thus provide a novel restorative framework for severely affected stroke patients.

Closed-Loop Neuroprosthesis for Reach-to-Grasp Assistance: Combining Adaptive Multi-channel Neuromuscular Stimulation with a Multi-joint Arm Exoskeleton

Stroke patients with severe motor deficits cannot execute task-oriented rehabilitation exercises with their affected upper extremity. Advanced rehabilitation technology may support them in performing such reach-to-grasp movements. The challenge is, however, to provide assistance as needed, while maintaining the participants' commitment during the exercises. In this feasibility study, we introduced a closed-loop neuroprosthesis for reach-to-grasp assistance which combines adaptive multi-channel neuromuscular stimulation with a multi-joint arm exoskeleton. Eighteen severely affected chronic stroke patients were assisted by a gravity-compensating, seven-degree-of-freedom exoskeleton which was attached to the paretic arm for performing reach-to-grasp exercises resembling activities of daily living in a virtual environment. During the exercises, adaptive electrical stimulation was applied to seven different muscles of the upper extremity in a performance-dependent way to enhance the task-oriented movement trajectory. The stimulation intensity was individualized for each targeted muscle and remained subthreshold, i.e., induced no overt support. Closed-loop neuromuscular stimulation could be well integrated into the exoskeleton-based training, and increased the task-related range of motion (p = 0.0004) and movement velocity (p = 0.015), while preserving accuracy. The highest relative stimulation intensity was required to facilitate the grasping function. The facilitated range of motion correlated with the upper extremity Fugl-Meyer Assessment score of the patients (p = 0.028). Combining adaptive multi-channel neuromuscular stimulation with antigravity assistance amplifies the residual motor capabilities of severely affected stroke patients during rehabilitation exercises and may thus provide a customized training environment for patient-tailored support while preserving the participants' engagement.

Compensation or Restoration: Closed-Loop Feedback of Movement Quality for Assisted Reach-to-Grasp Exercises with a Multi-Joint Arm Exoskeleton

Assistive technology allows for intensive practice and kinematic measurements during rehabilitation exercises. More recent approaches attach a gravity-compensating multi-joint exoskeleton to the upper extremity to facilitate task-oriented training in three-dimensional space with virtual reality feedback. The movement quality, however, is mostly captured through end-point measures that lack information on proximal inter-joint coordination. This limits the differentiation between compensation strategies and genuine restoration both during the exercise and in the course of rehabilitation. We extended in this proof-of-concept study a commercially available seven degree-of-freedom arm exoskeleton by using the real-time sensor data to display a three-dimensional multi-joint visualization of the user's arm. Ten healthy subjects and three severely affected chronic stroke patients performed reach-to-grasp exercises resembling activities of daily living assisted by the attached exoskeleton and received closed-loop online feedback of the three-dimensional movement in virtual reality. Patients in this pilot study differed significantly with regard to motor performance (accuracy, temporal efficiency, range of motion) and movement quality (proximal inter-joint coordination) from the healthy control group. In the course of 20 training and feedback sessions over 4 weeks, these pathological measures improved significantly toward the reference parameters of healthy participants. It was moreover feasible to capture the evolution of movement pattern kinematics of the shoulder and elbow and to quantify the individual degree of natural movement restoration for each patient. The virtual reality visualization and closed-loop feedback of joint-specific movement kinematics makes it possible to detect compensation strategies and may provide a tool to achieve the rehabilitation goals in accordance with the individual capacity for genuine functional restoration; a proposal that warrants further investigation in controlled studies with a larger cohort of stroke patients.

Brain State-Dependent Closed-Loop Modulation of Paired Associative Stimulation Controlled by Sensorimotor Desynchronization

BACKGROUND

Pairing peripheral electrical stimulation (ES) and transcranial magnetic stimulation (TMS) increases corticospinal excitability when applied with a specific temporal pattern. When the two stimulation techniques are applied separately, motor imagery (MI)-related oscillatory modulation amplifies both ES-related cortical effects-sensorimotor event-related desynchronization (ERD), and TMS-induced peripheral responses-motor-evoked potentials (MEP). However, the influence of brain self-regulation on the associative pairing of these stimulation techniques is still unclear.

OBJECTIVE

The aim of this pilot study was to investigate the effects of MI-related ERD during associative ES and TMS on subsequent corticospinal excitability.

METHOD

The paired application of functional electrical stimulation (FES) of the extensor digitorum communis (EDC) muscle and subsequent single-pulse TMS (110% resting motor threshold (RMT)) of the contralateral primary motor cortex (M1) was controlled by beta-band (16-22 Hz) ERD during MI of finger extension and applied within a brain-machine interface environment in six healthy subjects. Neural correlates were probed by acquiring the stimulus-response curve (SRC) of both MEP peak-to-peak amplitude and area under the curve (AUC) before and after the intervention.

RESULT

The application of approximately 150 pairs of associative FES and TMS resulted in a significant increase of MEP amplitudes and AUC, indicating that the induced increase of corticospinal excitability was mediated by the recruitment of additional neuronal pools. MEP increases were brain state-dependent and correlated with beta-band ERD, but not with the background EDC muscle activity; this finding was independent of the FES intensity applied.

CONCLUSION

These results could be relevant for developing closed-loop therapeutic approaches such as the application of brain state-dependent, paired associative stimulation (PAS) in the context of neurorehabilitation.

Reinforcement learning of self-regulated sensorimotor β-oscillations improves motor performance

Self-regulation of sensorimotor oscillations is currently researched in neurorehabilitation, e.g. for priming subsequent physiotherapy in stroke patients, and may be modulated by neurofeedback or transcranial brain stimulation. It has still to be demonstrated, however, whether and under which training conditions such brain self-regulation could also result in motor gains. Thirty-two right-handed, healthy subjects participated in a three-day intervention during which they performed 462 trials of kinesthetic motor-imagery while a brain-robot interface (BRI) turned event-related β-band desynchronization of the left sensorimotor cortex into the opening of the right hand by a robotic orthosis. Different training conditions were compared in a parallel-group design: (i) adaptive classifier thresholding and contingent feedback, (ii) adaptive classifier thresholding and non-contingent feedback, (iii) non-adaptive classifier thresholding and contingent feedback, and (iv) non-adaptive classifier thresholding and non-contingent feedback. We studied the task-related cortical physiology with electroencephalography and the behavioral performance in a subsequent isometric motor task. Contingent neurofeedback and adaptive classifier thresholding were critical for learning brain self-regulation which, in turn, led to behavioral gains after the intervention. The acquired skill for sustained sensorimotor β-desynchronization correlated significantly with subsequent motor improvement. Operant learning of brain self-regulation with a BRI may offer a therapeutic perspective for severely affected stroke patients lacking residual hand function.

Brain state-dependent robotic reaching movement with a multi-joint arm exoskeleton: combining brain-machine interfacing and robotic rehabilitation

While robot-assisted arm and hand training after stroke allows for intensive task-oriented practice, it has provided only limited additional benefit over dose-matched physiotherapy up to now. These rehabilitation devices are possibly too supportive during the exercises. Neurophysiological signals might be one way of avoiding slacking and providing robotic support only when the brain is particularly responsive to peripheral input. We tested the feasibility of three-dimensional robotic assistance for reaching movements with a multi-joint exoskeleton during motor imagery (MI)-related desynchronization of sensorimotor oscillations in the β-band. We also registered task-related network changes of cortical functional connectivity by electroencephalography via the imaginary part of the coherence function. Healthy subjects and stroke survivors showed similar patterns—but different aptitudes—of controlling the robotic movement. All participants in this pilot study with nine healthy subjects and two stroke patients achieved their maximum performance during the early stages of the task. Robotic control was significantly higher and less variable when proprioceptive feedback was provided in addition to visual feedback, i.e., when the orthosis was actually attached to the subject’s arm during the task. A distributed cortical network of task-related coherent activity in the θ-band showed significant differences between healthy subjects and stroke patients as well as between early and late periods of the task. Brain-robot interfaces (BRIs) may successfully link three-dimensional robotic training to the participants’ efforts and allow for task-oriented practice of activities of daily living with a physiologically controlled multi-joint exoskeleton. Changes of cortical physiology during the task might also help to make subject-specific adjustments of task difficulty and guide adjunct interventions to facilitate motor learning for functional restoration, a proposal that warrants further investigation in a larger cohort of stroke patients.

Self-regulation of circumscribed brain activity modulates spatially selective and frequency specific connectivity of distributed resting state networks

The mechanisms of learning involved in brain self-regulation have still to be unveiled to exploit the full potential of this methodology for therapeutic interventions. This skill of volitionally changing brain activity presumably resembles motor skill learning which in turn is accompanied by plastic changes modulating resting state networks. Along these lines, we hypothesized that brain regulation and neurofeedback would similarly modify intrinsic networks at rest while presenting a distinct spatio-temporal pattern. High-resolution electroencephalography preceded and followed a single neurofeedback training intervention of modulating circumscribed sensorimotor low β-activity by kinesthetic motor imagery in eleven healthy participants. The participants were kept in the deliberative phase of skill acquisition with high demands for learning self-regulation through stepwise increases of task difficulty. By applying the corrected imaginary part of the coherency function, we observed increased functional connectivity of both the primary motor and the primary somatosensory cortex with their respective contralateral homologous cortices in the low β-frequency band which was self-regulated during feedback. At the same time, the primary motor cortex—but none of the surrounding cortical areas—showed connectivity to contralateral supplementary motor and dorsal premotor areas in the high β-band. Simultaneously, the neurofeedback target displayed a specific increase of functional connectivity with an ipsilateral fronto-parietal network in the α-band while presenting a de-coupling with contralateral primary and secondary sensorimotor areas in the very same frequency band. Brain self-regulation modifies resting state connections spatially selective to the neurofeedback target of the dominant hemisphere. These are anatomically distinct with regard to the cortico-cortical connectivity pattern and are functionally specific with regard to the time domain of coherent activity consistent with a Hebbian-like sharpening concept.

Reinforcement learning of self-regulated β-oscillations for motor restoration in chronic stroke

Neurofeedback training of Motor imagery (MI)-related brain-states with brain-computer/brain-machine interfaces (BCI/BMI) is currently being explored as an experimental intervention prior to standard physiotherapy to improve the motor outcome of stroke rehabilitation. The use of BCI/BMI technology increases the adherence to MI training more efficiently than interventions with sham or no feedback. Moreover, pilot studies suggest that such a priming intervention before physiotherapy might—like some brain stimulation techniques—increase the responsiveness of the brain to the subsequent physiotherapy, thereby improving the general clinical outcome. However, there is little evidence up to now that these BCI/BMI-based interventions have achieved operate conditioning of specific brain states that facilitate task-specific functional gains beyond the practice of primed physiotherapy. In this context, we argue that BCI/BMI technology provides a valuable neurofeedback tool for rehabilitation but needs to aim at physiological features relevant for the targeted behavioral gain. Moreover, this therapeutic intervention has to be informed by concepts of reinforcement learning to develop its full potential. Such a refined neurofeedback approach would need to address the following issues: (1) Defining a physiological feedback target specific to the intended behavioral gain, e.g., β-band oscillations for cortico-muscular communication. This targeted brain state could well be different from the brain state optimal for the neurofeedback task, e.g., α-band oscillations for differentiating MI from rest; (2) Selecting a BCI/BMI classification and thresholding approach on the basis of learning principles, i.e., balancing challenge and reward of the neurofeedback task instead of maximizing the classification accuracy of the difficulty level device; and (3) Adjusting the difficulty level in the course of the training period to account for the cognitive load and the learning experience of the participant. Here, we propose a comprehensive neurofeedback strategy for motor restoration after stroke that addresses these aspects, and provide evidence for the feasibility of the suggested approach by demonstrating that dynamic threshold adaptation based on reinforcement learning may lead to frequency-specific operant conditioning of β-band oscillations paralleled by task-specific motor improvement; a proposal that requires investigation in a larger cohort of stroke patients.

Oscillatory entrainment of the motor cortical network during motor imagery is modulated by the feedback modality

Neurofeedback of self-regulated brain activity in circumscribed cortical regions is used as a novel strategy to facilitate functional restoration following stroke. Basic knowledge about its impact on motor system oscillations and functional connectivity is however scarce. Specifically, a direct comparison between different feedback modalities and their neural signatures is missing. We assessed a neurofeedback training intervention of modulating β-activity in circumscribed sensorimotor regions by kinesthetic motor imagery (MI). Right-handed healthy participants received two different feedback modalities contingent to their MI-associated brain activity in a cross-over design: (I) visual feedback with a brain-computer interface (BCI) and (II) proprioceptive feedback with a brain-robot interface (BRI) orthosis attached to the right hand. High-density electroencephalography was used to examine the reactivity of the cortical motor system during the training session of each task by studying both local oscillatory power entrainment and distributed functional connectivity. Both feedback modalities activated a distributed functional connectivity network of coherent oscillations. A significantly higher skill and lower variability of self-controlled sensorimotor β-band modulation could, however, be achieved in the BRI condition. This gain in controlling regional motor oscillations was accompanied by functional coupling of remote β-band and θ-band activity in bilateral fronto-central regions and left parieto-occipital regions, respectively. The functional coupling of coherent θ-band oscillations correlated moreover with the skill of regional β-modulation thus revealing a motor learning related network. Our findings indicate that proprioceptive feedback is more suitable than visual feedback to entrain the motor network architecture during the interplay between motor imagery and feedback processing thus resulting in better volitional control of regional brain activity.