Selection and parameterization of cortical neurons for neuroprosthetic control

Exploring Neurofeedback Training for BMI Power Augmentation of Upper Limbs: A Pilot Study

Electroencephalography neurofeedback (EEG-NFB) training can induce changes in the power of targeted EEG bands. The objective of this study is to enhance and evaluate the specific changes of EEG power spectral density that the brain-machine interface (BMI) users can reliably generate for power augmentation through EEG-NFB training. First, we constructed an EEG-NFB training system for power augmentation. Then, three subjects were assigned to three NFB training stages, based on a 6-day consecutive training session as one stage. The subjects received real-time feedback from their EEG signals by a robotic arm while conducting flexion and extension movement with their elbow and shoulder joints, respectively. EEG signals were compared with each NFB training stage. The training results showed that EEG beta (12-40 Hz) power increased after the NFB training for both the elbow and the shoulder joints' movements. EEG beta power showed sustained improvements during the 3-stage training, which revealed that even the short-term training could improve EEG signals significantly. Moreover, the training effect of the shoulder joints was more obvious than that of the elbow joints. These results suggest that NFB training can improve EEG signals and clarify the specific EEG changes during the movement. Our results may even provide insights into how the neural effects of NFB can be better applied to the BMI power augmentation system and improve the performance of healthy individuals.

Feature-Selection-Based Transfer Learning for Intracortical Brain-Machine Interface Decoding

The time spent in collecting current samples for decoder calibration and the computational burden brought by high-dimensional neural recordings remain two challenging problems in intracortical brain-machine interfaces (iBMIs). Decoder calibration optimization approaches have been proposed, and neuron selection methods have been used to reduce computational burden. However, few methods can solve both problems simultaneously. In this article, we present a symmetrical-uncertainty-based transfer learning (SUTL) method that combines transfer learning with feature selection. The proposed method uses symmetrical uncertainty to quantitatively measure three indices for feature selection: stationarity, importance and redundancy of the feature. By selecting the stationary features, the disparities between the historical data and current data can be diminished, and the historical data can be effectively used for decoder calibration, thereby reducing the demand for current data. After selecting the important and non-redundant features, only the channels corresponding to them need to work; thus, the computational burden is reduced. The proposed method was tested on neural data recorded from two rhesus macaques to decode the reaching position or grasping gesture. The results showed that the SUTL method diminished the disparities between the historical data and current data, while achieving superior decoding performance with the needs of only ten current samples each category, less than 10% the number of features and 30% the number of neural recording channels. Additionally, unlike most studies on iBMIs, feature selection was implemented instead of neuron selection, and the average decoding accuracy achieved by the former was 6.6% higher.

A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces

The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.

Coding of Self and Other's Future Choices in Dorsal Premotor Cortex during Social Interaction

Representing others' intentions is central to primate social life. We explored the role of dorsal premotor cortex (PMd) in discriminating between self and others' behavior while two male rhesus monkeys performed a non-match-to-goal task in a monkey-human paradigm. During each trial, two of four potential targets were randomly presented on the right and left parts of a screen, and the monkey or the human was required to choose the one that did not match the previously chosen target. Each agent had to monitor the other's action in order to select the correct target in that agent's own turn. We report neurons that selectively encoded the future choice of the monkey, the human agent, or both. Our findings suggest that PMd activity shows a high degree of self-other differentiation during face-to-face interactions, leading to an independent representation of what others will do instead of entailing self-centered mental rehearsal or mirror-like activities.

Design and testing of a 96-channel neural interface module for the Networked Neuroprosthesis system

Background

The loss of motor functions resulting from spinal cord injury can have devastating implications on the quality of one’s life. Functional electrical stimulation has been used to help restore mobility, however, current functional electrical stimulation (FES) systems require residual movements to control stimulation patterns, which may be unintuitive and not useful for individuals with higher level cervical injuries. Brain machine interfaces (BMI) offer a promising approach for controlling such systems; however, they currently still require transcutaneous leads connecting indwelling electrodes to external recording devices. While several wireless BMI systems have been designed, high signal bandwidth requirements limit clinical translation. Case Western Reserve University has developed an implantable, modular FES system, the Networked Neuroprosthesis (NNP), to perform combinations of myoelectric recording and neural stimulation for controlling motor functions. However, currently the existing module capabilities are not sufficient for intracortical recordings.

Methods

Here we designed and tested a 1 × 4 cm, 96-channel neural recording module prototype to fit within the specifications to mate with the NNP. The neural recording module extracts power between 0.3–1 kHz, instead of transmitting the raw, high bandwidth neural data to decrease power requirements.

Results

The module consumed 33.6 mW while sampling 96 channels at approximately 2 kSps. We also investigated the relationship between average spiking band power and neural spike rate, which produced a maximum correlation of R = 0.8656 (Monkey N) and R = 0.8023 (Monkey W).

Conclusion

Our experimental results show that we can record and transmit 96 channels at 2ksps within the power restrictions of the NNP system and successfully communicate over the NNP network. We believe this device can be used as an extension to the NNP to produce a clinically viable, fully implantable, intracortically-controlled FES system and advance the field of bioelectronic medicine.

Invasive Brain Machine Interface System

Because of high spatial-temporal resolution of neural signals obtained by invasive recording, the invasive brain-machine interfaces (BMI) have achieved great progress in the past two decades. With success in animal research, BMI technology is transferring to clinical trials for helping paralyzed people to restore their lost motor functions. This chapter gives a brief review of BMI development from animal experiments to human clinical studies in the following aspects: (1) BMIs based on rodent animals; (2) BMI based on non-human primates; and (3) pilot BMIs studies in clinical trials. In the end, the chapter concludes with a summary of potential opportunities and future challenges in BMI technology.

Watch, Imagine, Attempt: Motor Cortex Single-Unit Activity Reveals Context-Dependent Movement Encoding in Humans With Tetraplegia

Planning and performing volitional movement engages widespread networks in the human brain, with motor cortex considered critical to the performance of skilled limb actions. Motor cortex is also engaged when actions are observed or imagined, but the manner in which ensembles of neurons represent these volitional states (VoSs) is unknown. Here we provide direct demonstration that observing, imagining or attempting action activates shared neural ensembles in human motor cortex. Two individuals with tetraplegia (due to brainstem stroke or amyotrophic lateral sclerosis, ALS) were verbally instructed to watch, imagine, or attempt reaching actions displayed on a computer screen. Neural activity in the precentral gyrus incorporated information about both cognitive state and movement kinematics; the three conditions presented overlapping but unique, statistically distinct activity patterns. These findings demonstrate that individual neurons in human motor cortex reflect information related to sensory inputs and VoS in addition to movement features, and are a key part of a broader network linking perception and cognition to action.

Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter

Objective: To date, many brain-machine interface (BMI) studies have developed decoding algorithms for neuroprostheses that provide users with precise control of upper arm reaches with some limited grasping capabilities. However, comparatively few have focused on quantifying the performance of precise finger control. Here we expand upon this work by investigating online control of individual finger groups.

Approach: We have developed a novel training manipulandum for non-human primate (NHP) studies to isolate the movements of two specific finger groups: index and middle-ring-pinkie (MRP) fingers. We use this device in combination with the ReFIT (Recalibrated Feedback Intention-Trained) Kalman filter to decode the position of each finger group during a single degree of freedom task in two rhesus macaques with Utah arrays in motor cortex. The ReFIT Kalman filter uses a two-stage training approach that improves online control of upper arm tasks with substantial reductions in orbiting time, thus making it a logical first choice for precise finger control.

Results: Both animals were able to reliably acquire fingertip targets with both index and MRP fingers, which they did in blocks of finger group specific trials. Decoding from motor signals online, the ReFIT Kalman filter reliably outperformed the standard Kalman filter, measured by bit rate, across all tested finger groups and movements by 31.0 and 35.2%. These decoders were robust when the manipulandum was removed during online control. While index finger movements and middle-ring-pinkie finger movements could be differentiated from each other with 81.7% accuracy across both subjects, the linear Kalman filter was not sufficient for decoding both finger groups together due to significant unwanted movement in the stationary finger, potentially due to co-contraction.

Significance: To our knowledge, this is the first systematic and biomimetic separation of digits for continuous online decoding in a NHP as well as the first demonstration of the ReFIT Kalman filter improving the performance of precise finger decoding. These results suggest that novel nonlinear approaches, apparently not necessary for center out reaches or gross hand motions, may be necessary to achieve independent and precise control of individual fingers.

Anodal tDCS over Primary Motor Cortex Provides No Advantage to Learning Motor Sequences via Observation

When learning a new motor skill, we benefit from watching others. It has been suggested that observation of others' actions can build a motor representation in the observer, and as such, physical and observational learning might share a similar neural basis. If physical and observational learning share a similar neural basis, then motor cortex stimulation during observational practice should similarly enhance learning by observation as it does through physical practice. Here, we used transcranial direct-current stimulation (tDCS) to address whether anodal stimulation to M1 during observational training facilitates skill acquisition. Participants learned keypress sequences across four consecutive days of observational practice while receiving active or sham stimulation over M1. The results demonstrated that active stimulation provided no advantage to skill learning over sham stimulation. Further, Bayesian analyses revealed evidence in favour of the null hypothesis across our dependent measures. Our findings therefore provide no support for the hypothesis that excitatory M1 stimulation can enhance observational learning in a similar manner to physical learning. More generally, the results add to a growing literature that suggests that the effects of tDCS tend to be small, inconsistent, and hard to replicate. Future tDCS research should consider these factors when designing experimental procedures.

Leveraging historical knowledge of neural dynamics to rescue decoder performance as neural channels are lost: "Decoder hysteresis"

An intracortical brain-machine interface (BMI) decodes spiking activity recorded from motor cortical neurons to drive a prosthetic device (e.g., a computer cursor or robotic arm). As the number of recorded neurons decreases over time due to decay in recording quality, the performance of a BMI decreases. We asked: can degrading BMI performance be rescued by using prior information from when more neurons were observed? This would entail augmenting a decoder by using previously learned knowledge about motor cortex (at an earlier point in the array lifetime). We implemented this idea by modeling low-dimensional dynamics of the neural population, which describe how the population evolves through time. We posit that if the neural dynamics accurately reflect properties of motor cortex, then having a better estimate of these dynamics should result in a better decoder. Using previously collected (offline) experimental data, we found that a decoder using dynamics inferred in the past (when more neural channels were available) outperformed the same decoder using dynamics inferred from the (fewer) remaining neural channels. These results suggest that neural dynamics capture important features of the neural population responses in motor cortex, and that knowledge of these dynamics may rescue BMI performance even as array signal quality degrades.

Neuron-Type-Specific Utility in a Brain-Machine Interface: a Pilot Study

CONTEXT

Firing rates of single cortical neurons can be volitionally modulated through biofeedback (i.e. operant conditioning), and this information can be transformed to control external devices (i.e. brain-machine interfaces; BMIs). However, not all neurons respond to operant conditioning in BMI implementation. Establishing criteria that predict neuron utility will assist translation of BMI research to clinical applications.

FINDINGS

Single cortical neurons (n=7) were recorded extracellularly from primary motor cortex of a Long-Evans rat. Recordings were incorporated into a BMI involving up-regulation of firing rate to control the brightness of a light-emitting-diode and subsequent reward. Neurons were classified as 'fast-spiking', 'bursting' or 'regular-spiking' according to waveform-width and intrinsic firing patterns. Fast-spiking and bursting neurons were found to up-regulate firing rate by a factor of 2.43±1.16, demonstrating high utility, while regular-spiking neurons decreased firing rates on average by a factor of 0.73±0.23, demonstrating low utility.

CONCLUSION/CLINICAL RELEVANCE

The ability to select neurons with high utility will be important to minimize training times and maximize information yield in future clinical BMI applications. The highly contrasting utility observed between fast-spiking and bursting neurons versus regular-spiking neurons allows for the hypothesis to be advanced that intrinsic electrophysiological properties may be useful criteria that predict neuron utility in BMI implementation.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Decoding Lower Limb Muscle Activity and Kinematics from Cortical Neural Spike Trains during Monkey Performing Stand and Squat Movements

Extensive literatures have shown approaches for decoding upper limb kinematics or muscle activity using multichannel cortical spike recordings toward brain machine interface (BMI) applications. However, similar topics regarding lower limb remain relatively scarce. We previously reported a system for training monkeys to perform visually guided stand and squat tasks. The current study, as a follow-up extension, investigates whether lower limb kinematics and muscle activity characterized by electromyography (EMG) signals during monkey performing stand/squat movements can be accurately decoded from neural spike trains in primary motor cortex (M1). Two monkeys were used in this study. Subdermal intramuscular EMG electrodes were implanted to 8 right leg/thigh muscles. With ample data collected from neurons from a large brain area, we performed a spike triggered average (SpTA) analysis and got a series of density contours which revealed the spatial distributions of different muscle-innervating neurons corresponding to each given muscle. Based on the guidance of these results, we identified the locations optimal for chronic electrode implantation and subsequently carried on chronic neural data recordings. A recursive Bayesian estimation framework was proposed for decoding EMG signals together with kinematics from M1 spike trains. Two specific algorithms were implemented: a standard Kalman filter and an unscented Kalman filter. For the latter one, an artificial neural network was incorporated to deal with the nonlinearity in neural tuning. High correlation coefficient and signal to noise ratio between the predicted and the actual data were achieved for both EMG signals and kinematics on both monkeys. Higher decoding accuracy and faster convergence rate could be achieved with the unscented Kalman filter. These results demonstrate that lower limb EMG signals and kinematics during monkey stand/squat can be accurately decoded from a group of M1 neurons with the proposed algorithms. Our findings provide new insights for extending current BMI design concepts and techniques on upper limbs to lower limb circumstances. Brain controlled exoskeleton, prostheses or neuromuscular electrical stimulators for lower limbs are expected to be developed, which enables the subject to manipulate complex biomechatronic devices with mind in more harmonized manner.

A Sliced Inverse Regression (SIR) Decoding the Forelimb Movement from Neuronal Spikes in the Rat Motor Cortex

Several neural decoding algorithms have successfully converted brain signals into commands to control a computer cursor and prosthetic devices. A majority of decoding methods, such as population vector algorithms (PVA), optimal linear estimators (OLE), and neural networks (NN), are effective in predicting movement kinematics, including movement direction, speed and trajectory but usually require a large number of neurons to achieve desirable performance. This study proposed a novel decoding algorithm even with signals obtained from a smaller numbers of neurons. We adopted sliced inverse regression (SIR) to predict forelimb movement from single-unit activities recorded in the rat primary motor (M1) cortex in a water-reward lever-pressing task. SIR performed weighted principal component analysis (PCA) to achieve effective dimension reduction for nonlinear regression. To demonstrate the decoding performance, SIR was compared to PVA, OLE, and NN. Furthermore, PCA and sequential feature selection (SFS) which are popular feature selection techniques were implemented for comparison of feature selection effectiveness. Among SIR, PVA, OLE, PCA, SFS, and NN decoding methods, the trajectories predicted by SIR (with a root mean square error, RMSE, of 8.47 ± 1.32 mm) was closer to the actual trajectories compared with those predicted by PVA (30.41 ± 11.73 mm), OLE (20.17 ± 6.43 mm), PCA (19.13 ± 0.75 mm), SFS (22.75 ± 2.01 mm), and NN (16.75 ± 2.02 mm). The superiority of SIR was most obvious when the sample size of neurons was small. We concluded that SIR sorted the input data to obtain the effective transform matrices for movement prediction, making it a robust decoding method for conditions with sparse neuronal information.

Uniform and Non-uniform Perturbations in Brain-Machine Interface Task Elicit Similar Neural Strategies

The neural mechanisms that take place during learning and adaptation can be directly probed with brain-machine interfaces (BMIs). We developed a BMI controlled paradigm that enabled us to enforce learning by introducing perturbations which changed the relationship between neural activity and the BMI's output. We introduced a uniform perturbation to the system, through a visuomotor rotation (VMR), and a non-uniform perturbation, through a decorrelation task. The controller in the VMR was essentially unchanged, but produced an output rotated at 30° from the neurally specified output. The controller in the decorrelation trials decoupled the activity of neurons that were highly correlated in the BMI task by selectively forcing the preferred directions of these cell pairs to be orthogonal. We report that movement errors were larger in the decorrelation task, and subjects needed more trials to restore performance back to baseline. During learning, we measured decreasing trends in preferred direction changes and cross-correlation coefficients regardless of task type. Conversely, final adaptations in neural tunings were dependent on the type controller used (VMR or decorrelation). These results hint to the similar process the neural population might engage while adapting to new tasks, and how, through a global process, the neural system can arrive to individual solutions.

Adaptive neuron-to-EMG decoder training for FES neuroprostheses

OBJECTIVE

We have previously demonstrated a brain-machine interface neuroprosthetic system that provided continuous control of functional electrical stimulation (FES) and restoration of grasp in a primate model of spinal cord injury (SCI). Predicting intended EMG directly from cortical recordings provides a flexible high-dimensional control signal for FES. However, no peripheral signal such as force or EMG is available for training EMG decoders in paralyzed individuals.

APPROACH

Here we present a method for training an EMG decoder in the absence of muscle activity recordings; the decoder relies on mapping behaviorally relevant cortical activity to the inferred EMG activity underlying an intended action. Monkeys were trained at a 2D isometric wrist force task to control a computer cursor by applying force in the flexion, extension, ulnar, and radial directions and execute a center-out task. We used a generic muscle force-to-endpoint force model based on muscle pulling directions to relate each target force to an optimal EMG pattern that attained the target force while minimizing overall muscle activity. We trained EMG decoders during the target hold periods using a gradient descent algorithm that compared EMG predictions to optimal EMG patterns.

MAIN RESULTS

We tested this method both offline and online. We quantified both the accuracy of offline force predictions and the ability of a monkey to use these real-time force predictions for closed-loop cursor control. We compared both offline and online results to those obtained with several other direct force decoders, including an optimal decoder computed from concurrently measured neural and force signals.

SIGNIFICANCE

This novel approach to training an adaptive EMG decoder could make a brain-control FES neuroprosthesis an effective tool to restore the hand function of paralyzed individuals. Clinical implementation would make use of individualized EMG-to-force models. Broad generalization could be achieved by including data from multiple grasping tasks in the training of the neuron-to-EMG decoder. Our approach would make it possible for persons with SCI to grasp objects with their own hands, using near-normal motor intent.

Observation-based training for neuroprosthetic control of grasping by amputees

Current brain-machine interfaces (BMIs) allow upper limb amputees to position robotic arms with a high degree of accuracy, but lack the ability to control hand pre-shaping for grasping different objects. We have previously shown that low frequency (0.1-1 Hz) time domain cortical activity recorded at the scalp via electroencephalography (EEG) encodes information about grasp pre-shaping. To transfer this technology to clinical populations such as amputees, the challenge lies in constructing BMI models in the absence of overt training hand movements. Here we show that it is possible to train BMI models using observed grasping movements performed by a robotic hand attached to amputees' residual limb. Three transradial amputees controlled the grasping motion of an attached robotic hand via their EEG, following the action-observation training phase. Over multiple sessions, subjects successfully grasped the presented object (a bottle or a credit card) in 53±16 % of trials, demonstrating the validity of the BMI models. Importantly, the validation of the BMI model was through closed-loop performance, which demonstrates generalization of the model to unseen data. These results suggest `mirror neuron system' properties captured by delta band EEG that allows neural representation for action observation to be used for action control in an EEG-based BMI system.

Enabling Low-Power, Multi-Modal Neural Interfaces Through a Common, Low-Bandwidth Feature Space

Brain-Machine Interfaces (BMIs) have shown great potential for generating prosthetic control signals. Translating BMIs into the clinic requires fully implantable, wireless systems; however, current solutions have high power requirements which limit their usability. Lowering this power consumption typically limits the system to a single neural modality, or signal type, and thus to a relatively small clinical market. Here, we address both of these issues by investigating the use of signal power in a single narrow frequency band as a decoding feature for extracting information from electrocorticographic (ECoG), electromyographic (EMG), and intracortical neural data. We have designed and tested the Multi-modal Implantable Neural Interface (MINI), a wireless recording system which extracts and transmits signal power in a single, configurable frequency band. In prerecorded datasets, we used the MINI to explore low frequency signal features and any resulting tradeoff between power savings and decoding performance losses. When processing intracortical data, the MINI achieved a power consumption 89.7% less than a more typical system designed to extract action potential waveforms. When processing ECoG and EMG data, the MINI achieved similar power reductions of 62.7% and 78.8%. At the same time, using the single signal feature extracted by the MINI, we were able to decode all three modalities with less than a 9% drop in accuracy relative to using high-bandwidth, modality-specific signal features. We believe this system architecture can be used to produce a viable, cost-effective, clinical BMI.

Brain-controlled muscle stimulation for the restoration of motor function

Loss of the ability to move, as a consequence of spinal cord injury or neuromuscular disorder, has devastating consequences for the paralyzed individual, and great economic consequences for society. Functional electrical stimulation (FES) offers one means to restore some mobility to these individuals, improving not only their autonomy, but potentially their general health and well-being as well. FES uses electrical stimulation to cause the paralyzed muscles to contract. Existing clinical systems require the stimulation to be preprogrammed, with the patient typically using residual voluntary movement of another body part to trigger and control the patterned stimulation. The rapid development of neural interfacing in the past decade offers the promise of dramatically improved control for these patients, potentially allowing continuous control of FES through signals recorded from motor cortex, as the patient attempts to control the paralyzed body part. While application of these 'brain-machine interfaces' (BMIs) has undergone dramatic development for control of computer cursors and even robotic limbs, their use as an interface for FES has been much more limited. In this review, we consider both FES and BMI technologies and discuss the prospect for combining the two to provide important new options for paralyzed individuals.

Modulation of the Intracortical LFP during Action Execution and Observation

The activity of mirror neurons in macaque ventral premotor cortex (PMv) and primary motor cortex (M1) is modulated by the observation of another's movements. This modulation could underpin well documented changes in EEG/MEG activity indicating the existence of a mirror neuron system in humans. Because the local field potential (LFP) represents an important link between macaque single neuron and human noninvasive studies, we focused on mirror properties of intracortical LFPs recorded in the PMv and M1 hand regions in two macaques while they reached, grasped and held different objects, or observed the same actions performed by an experimenter. Upper limb EMGs were recorded to control for covert muscle activity during observation.The movement-related potential (MRP), investigated as intracortical low-frequency LFP activity (<9 Hz), was modulated in both M1 and PMv, not only during action execution but also during action observation. Moreover, the temporal LFP modulations during execution and observation were highly correlated in both cortical areas. Beta power in both PMv and M1 was clearly modulated in both conditions. Although the MRP was detected only during dynamic periods of the task (reach/grasp/release), beta decreased during dynamic and increased during static periods (hold).Comparison of LFPs for different grasps provided evidence for partially nonoverlapping networks being active during execution and observation, which might be related to different inputs to motor areas during these conditions. We found substantial information about grasp in the MRP corroborating its suitability for brain-machine interfaces, although information about grasp was generally low during action observation.

Toward Building Hybrid Biological/in silico Neural Networks for Motor Neuroprosthetic Control

In this article, we introduce the Bioinspired Neuroprosthetic Design Environment (BNDE) as a practical platform for the development of novel brain–machine interface (BMI) controllers, which are based on spiking model neurons. We built the BNDE around a hard real-time system so that it is capable of creating simulated synapses from extracellularly recorded neurons to model neurons. In order to evaluate the practicality of the BNDE for neuroprosthetic control experiments, a novel, adaptive BMI controller was developed and tested using real-time closed-loop simulations. The present controller consists of two in silico medium spiny neurons, which receive simulated synaptic inputs from recorded motor cortical neurons. In the closed-loop simulations, the recordings from the cortical neurons were imitated using an external, hardware-based neural signal synthesizer. By implementing a reward-modulated spike timing-dependent plasticity rule, the controller achieved perfect target reach accuracy for a two-target reaching task in one-dimensional space. The BNDE combines the flexibility of software-based spiking neural network (SNN) simulations with powerful online data visualization tools and is a low-cost, PC-based, and all-in-one solution for developing neurally inspired BMI controllers. We believe that the BNDE is the first implementation, which is capable of creating hybrid biological/in silico neural networks for motor neuroprosthetic control and utilizes multiple CPU cores for computationally intensive real-time SNN simulations.

A systematic approach to selecting task relevant neurons

BACKGROUND

Since task related neurons cannot be specifically targeted during surgery, a critical decision to make is to select which neurons are task-related when performing data analysis. Including neurons unrelated to the task degrade decoding accuracy and confound neurophysiological results. Traditionally, task-related neurons are selected as those with significant changes in firing rate when a stimulus is applied. However, this assumes that neurons' encoding of stimuli are dominated by their firing rate with little regard to temporal dynamics.

NEW METHOD

This paper proposes a systematic approach for neuron selection, which uses a likelihood ratio test to capture the contribution of stimulus to spiking activity while taking into account task-irrelevant intrinsic dynamics that affect firing rates. This approach is denoted as the model deterioration excluding stimulus (MDES) test.

RESULTS

MDES is compared to firing rate selection in four case studies: a simulation, a decoding example, and two neurophysiology examples.

COMPARISON WITH EXISTING METHODS

The MDES rankings in the simulation match closely with ideal rankings, while firing rate rankings are skewed by task-irrelevant parameters. For decoding, 95% accuracy is achieved using the top 8 MDES-ranked neurons, while the top 12 firing-rate ranked neurons are needed. In the neurophysiological examples, MDES matches published results when firing rates do encode salient stimulus information, and uncovers oscillatory modulations in task-related neurons that are not captured when neurons are selected using firing rates.

CONCLUSIONS

These case studies illustrate the importance of accounting for intrinsic dynamics when selecting task-related neurons and following the MDES approach accomplishes that. MDES selects neurons that encode task-related information irrespective of these intrinsic dynamics which can bias firing rate based selection.

Combining decoder design and neural adaptation in brain-machine interfaces

Brain-machine interfaces (BMIs) aim to help people with paralysis by decoding movement-related neural signals into control signals for guiding computer cursors, prosthetic arms, and other assistive devices. Despite compelling laboratory experiments and ongoing FDA pilot clinical trials, system performance, robustness, and generalization remain challenges. We provide a perspective on how two complementary lines of investigation, that have focused on decoder design and neural adaptation largely separately, could be brought together to advance BMIs. This BMI paradigm should also yield new scientific insights into the function and dysfunction of the nervous system.

Neuron selection based on deflection coefficient maximization for the neural decoding of dexterous finger movements

Future generations of brain-machine interface (BMI) will require more dexterous motion control such as hand and finger movements. Since a population of neurons in the primary motor cortex (M1) area is correlated with finger movements, neural activities recorded in M1 area are used to reconstruct an intended finger movement. In a BMI system, decoding discrete finger movements from a large number of input neurons does not guarantee a higher decoding accuracy in spite of the increase in computational burden. Hence, we hypothesize that selecting neurons important for coding dexterous flexion/extension of finger movements would improve the BMI performance. In this paper, two metrics are presented to quantitatively measure the importance of each neuron based on Bayes risk minimization and deflection coefficient maximization in a statistical decision problem. Since motor cortical neurons are active with movements of several different fingers, the proposed method is more suitable for a discrete decoding of flexion-extension finger movements than the previous methods for decoding reaching movements. In particular, the proposed metrics yielded high decoding accuracies across all subjects and also in the case of including six combined two-finger movements. While our data acquisition and analysis was done off-line and post processing, our results point to the significance of highly coding neurons in improving BMI performance.

Toward more versatile and intuitive cortical brain-machine interfaces

Brain-machine interfaces have great potential for the development of neuroprosthetic applications to assist patients suffering from brain injury or neurodegenerative disease. One type of brain-machine interface is a cortical motor prosthetic, which is used to assist paralyzed subjects. Motor prosthetics to date have typically used the motor cortex as a source of neural signals for controlling external devices. The review will focus on several new topics in the arena of cortical prosthetics. These include using: recordings from cortical areas outside motor cortex; local field potentials as a source of recorded signals; somatosensory feedback for more dexterous control of robotics; and new decoding methods that work in concert to form an ecology of decode algorithms. These new advances promise to greatly accelerate the applicability and ease of operation of motor prosthetics.

A brain-machine-muscle interface for restoring hindlimb locomotion after complete spinal transection in rats

A brain-machine interface (BMI) is a neuroprosthetic device that can restore motor function of individuals with paralysis. Although the feasibility of BMI control of upper-limb neuroprostheses has been demonstrated, a BMI for the restoration of lower-limb motor functions has not yet been developed. The objective of this study was to determine if gait-related information can be captured from neural activity recorded from the primary motor cortex of rats, and if this neural information can be used to stimulate paralysed hindlimb muscles after complete spinal cord transection. Neural activity was recorded from the hindlimb area of the primary motor cortex of six female Sprague Dawley rats during treadmill locomotion before and after mid-thoracic transection. Before spinal transection there was a strong association between neural activity and the step cycle. This association decreased after spinal transection. However, the locomotive state (standing vs. walking) could still be successfully decoded from neural recordings made after spinal transection. A novel BMI device was developed that processed this neural information in real-time and used it to control electrical stimulation of paralysed hindlimb muscles. This system was able to elicit hindlimb muscle contractions that mimicked forelimb stepping. We propose this lower-limb BMI as a future neuroprosthesis for human paraplegics.

A brain-machine interface enables bimanual arm movements in monkeys

Brain-machine interfaces (BMIs) are artificial systems that aim to restore sensation and movement to paralyzed patients. So far, BMIs have enabled only one arm to be moved at a time. Control of bimanual arm movements remains a major challenge. We have developed and tested a bimanual BMI that enables rhesus monkeys to control two avatar arms simultaneously. The bimanual BMI was based on the extracellular activity of 374 to 497 neurons recorded from several frontal and parietal cortical areas of both cerebral hemispheres. Cortical activity was transformed into movements of the two arms with a decoding algorithm called a fifth-order unscented Kalman filter (UKF). The UKF was trained either during a manual task performed with two joysticks or by having the monkeys passively observe the movements of avatar arms. Most cortical neurons changed their modulation patterns when both arms were engaged simultaneously. Representing the two arms jointly in a single UKF decoder resulted in improved decoding performance compared with using separate decoders for each arm. As the animals' performance in bimanual BMI control improved over time, we observed widespread plasticity in frontal and parietal cortical areas. Neuronal representation of the avatar and reach targets was enhanced with learning, whereas pairwise correlations between neurons initially increased and then decreased. These results suggest that cortical networks may assimilate the two avatar arms through BMI control. These findings should help in the design of more sophisticated BMIs capable of enabling bimanual motor control in human patients.

Continuous closed-loop decoder adaptation with a recursive maximum likelihood algorithm allows for rapid performance acquisition in brain-machine interfaces

Closed-loop decoder adaptation (CLDA) is an emerging paradigm for both improving and maintaining online performance in brain-machine interfaces (BMIs). The time required for initial decoder training and any subsequent decoder recalibrations could be potentially reduced by performing continuous adaptation, in which decoder parameters are updated at every time step during these procedures, rather than waiting to update the decoder at periodic intervals in a more batch-based process. Here, we present recursive maximum likelihood (RML), a CLDA algorithm that performs continuous adaptation of a Kalman filter decoder's parameters. We demonstrate that RML possesses a variety of useful properties and practical algorithmic advantages. First, we show how RML leverages the accuracy of updates based on a batch of data while still adapting parameters on every time step. Second, we illustrate how the RML algorithm is parameterized by a single, intuitive half-life parameter that can be used to adjust the rate of adaptation in real time. Third, we show how even when the number of neural features is very large, RML's memory-efficient recursive update rules can be reformulated to also be computationally fast so that continuous adaptation is still feasible. To test the algorithm in closed-loop experiments, we trained three macaque monkeys to perform a center-out reaching task by using either spiking activity or local field potentials to control a 2D computer cursor. RML achieved higher levels of performance more rapidly in comparison to a previous CLDA algorithm that adapts parameters on a more intermediate timescale. Overall, our results indicate that RML is an effective CLDA algorithm for achieving rapid performance acquisition using continuous adaptation.

Restoring sensorimotor function through intracortical interfaces: progress and looming challenges

The loss of a limb or paralysis resulting from spinal cord injury has devastating consequences on quality of life. One approach to restoring lost sensory and motor abilities in amputees and patients with tetraplegia is to supply them with implants that provide a direct interface with the CNS. Such brain-machine interfaces might enable a patient to exert voluntary control over a prosthetic or robotic limb or over the electrically induced contractions of paralysed muscles. A parallel interface could convey sensory information about the consequences of these movements back to the patient. Recent developments in the algorithms that decode motor intention from neuronal activity and in approaches to convey sensory feedback by electrically stimulating neurons, using biomimetic and adaptation-based approaches, have shown the promise of invasive interfaces with sensorimotor cortices, although substantial challenges remain.

Selection of cortical neurons for identifying movement transitions in stand and squat

Neural signals collected from motor cortex were quantified for identification of subject's specific movement intentions in a Brain Machine Interface (BMI). Neuron selection serves as an important procedure in this decoding process. In this study, we proposed a neuron selection method for identifying movement transitions in standing and squatting tasks by analyzing cortical neuron spike train patterns. A nonparametric analysis of variation, Kruskal-Wallis test, was introduced to evaluate whether the average discharging rate of each neuron changed significantly among different motion stages, and thereby categorize the neurons according to their active periods. Selection was performed based on neuron categorizing information. Finally, the average firing rates of selected neurons were assembled as feature vectors and a classifier based on support vector machines (SVM) was employed to discriminate different movement stages and identify transitions. The results indicate that our neuron selection method is accurate and efficient for finding neurons correlated with movement transitions in standing and squatting tasks.

Transfer of information by BMI

Brain machine interfaces (BMI) have become important in systems neuroscience with the goal to restore motor function in paralyzed patients. We assess the current ability of BMI devices to move objects. The topics discussed include: (1) the bits of information generated by a BMI signal, (2) the limitations of including more neurons for generating a BMI signal, (3) the superiority of a BMI signal using single cells versus electroencephalography, (4) plasticity and BMI, (5) the selection of a neural code for generating BMI, (6) the suppression of body movements during BMI, and (7) the role of vision in BMI. We conclude that further research on understanding how the brain generates movement is necessary before BMI can become a reasonable option for paralyzed patients.

Advantages of closed-loop calibration in intracortical brain-computer interfaces for people with tetraplegia

OBJECTIVE

Brain-computer interfaces (BCIs) aim to provide a means for people with severe motor disabilities to control their environment directly with neural activity. In intracortical BCIs for people with tetraplegia, the decoder that maps neural activity to desired movements has typically been calibrated using 'open-loop' (OL) imagination of control while a cursor automatically moves to targets on a computer screen. However, because neural activity can vary across contexts, a decoder calibrated using OL data may not be optimal for 'closed-loop' (CL) neural control. Here, we tested whether CL calibration creates a better decoder than OL calibration even when all other factors that might influence performance are held constant, including the amount of data used for calibration and the amount of elapsed time between calibration and testing.

APPROACH

Two people with tetraplegia enrolled in the BrainGate2 pilot clinical trial performed a center-out-back task using an intracortical BCI, switching between decoders that had been calibrated on OL versus CL data.

MAIN RESULTS

Even when all other variables were held constant, CL calibration improved neural control as well as the accuracy and strength of the tuning model. Updating the CL decoder using additional and more recent data resulted in further improvements.

SIGNIFICANCE

Differences in neural activity between OL and CL contexts contribute to the superiority of CL decoders, even prior to their additional 'adaptive' advantage. In the near future, CL decoder calibration may enable robust neural control without needing to pause ongoing, practical use of BCIs, an important step toward clinical utility.

Design and Analysis of Closed-Loop Decoder Adaptation Algorithms for Brain-Machine Interfaces

Closed-loop decoder adaptation (CLDA) is an emerging paradigm for achieving rapid performance improvements in online brain-machine interface (BMI) operation. Designing an effective CLDA algorithm requires making multiple important decisions, including choosing the timescale of adaptation, selecting which decoder parameters to adapt, crafting the corresponding update rules, and designing CLDA parameters. These design choices, combined with the specific settings of CLDA parameters, will directly affect the algorithm's ability to make decoder parameters converge to values that optimize performance. In this article, we present a general framework for the design and analysis of CLDA algorithms and support our results with experimental data of two monkeys performing a BMI task. First, we analyze and compare existing CLDA algorithms to highlight the importance of four critical design elements: the adaptation timescale, selective parameter adaptation, smooth decoder updates, and intuitive CLDA parameters. Second, we introduce mathematical convergence analysis using measures such as mean-squared error and KL divergence as a useful paradigm for evaluating the convergence properties of a prototype CLDA algorithm before experimental testing. By applying these measures to an existing CLDA algorithm, we demonstrate that our convergence analysis is an effective analytical tool that can ultimately inform and improve the design of CLDA algorithms.

Differentiating closed-loop cortical intention from rest: building an asynchronous electrocorticographic BCI

OBJECTIVE

Recent experiments have shown that electrocorticography (ECoG) can provide robust control signals for a brain-computer interface (BCI). Strategies that attempt to adapt a BCI control algorithm by learning from past trials often assume that the subject is attending to each training trial. Likewise, automatic disabling of movement control would be desirable during resting periods when random brain fluctuations might cause unintended movements of a device. To this end, our goal was to identify ECoG differences that arise between periods of active BCI use and rest.

APPROACH

We examined spectral differences in multi-channel, epidural micro-ECoG signals recorded from non-human primates when rest periods were interleaved between blocks of an active BCI control task.

MAIN RESULTS

Post-hoc analyses demonstrated that these states can be decoded accurately on both a trial-by-trial and real-time basis, and this discriminability remains robust over a period of weeks. In addition, high gamma frequencies showed greater modulation with desired movement direction, while lower frequency components demonstrated greater amplitude differences between task and rest periods, suggesting possible specialized BCI roles for these frequencies.

SIGNIFICANCE

The results presented here provide valuable insight into the neurophysiology of BCI control as well as important considerations toward the design of an asynchronous BCI system.

Mutual information analysis on non-stationary neuron importance for brain machine interfaces

Decoding with the important neuron subset has been widely used in brain machine interfaces (BMIs), as an effective strategy to reduce computational complexity. Previous works usually assume stationary of neuron importance, which may not be true according to recent research. We propose to conduct a mutual information evaluation to track the time-varying neuron importance over time. We found worth noting changes both in information amount and space distribution in our experiment. When the method is applied with a Kalman filter, the decoding performance achieve is better (with higher correlation coefficient) than when a fixed subset, which shows that time-varying neuron importance should be considered in adaptive algorithms.

Stochastic optimal control as a theory of brain-machine interface operation

The closed-loop operation of brain-machine interfaces (BMI) provides a framework to study the mechanisms behind neural control through a restricted output channel, with emerging clinical applications to stroke, degenerative disease, and trauma. Despite significant empirically driven improvements in closed-loop BMI systems, a fundamental, experimentally validated theory of closed-loop BMI operation is lacking. Here we propose a compact model based on stochastic optimal control to describe the brain in skillfully operating canonical decoding algorithms. The model produces goal-directed BMI movements with sensory feedback and intrinsically noisy neural output signals. Various experimentally validated phenomena emerge naturally from this model, including performance deterioration with bin width, compensation of biased decoders, and shifts in tuning curves between arm control and BMI control. Analysis of the model provides insight into possible mechanisms underlying these behaviors, with testable predictions. Spike binning may erode performance in part from intrinsic control-dependent constraints, regardless of decoding accuracy. In compensating decoder bias, the brain may incur an energetic cost associated with action potential production. Tuning curve shifts, seen after the mastery of a BMI-based skill, may reflect the brain's implementation of a new closed-loop control policy. The direction and magnitude of tuning curve shifts may be altered by decoder structure, ensemble size, and the costs of closed-loop control. Looking forward, the model provides a framework for the design and simulated testing of an emerging class of BMI algorithms that seek to directly exploit the presence of a human in the loop.

Neuron Selection by Relative Importance for Neural Decoding of Dexterous Finger Prosthesis Control Application

Future generations of upper limb prosthesis will have dexterous hand with individual fingers and will be controlled directly by neural signals. Neurons from the primary motor (M1) cortex code for finger movements and provide the source for neural control of dexterous prosthesis. Each neuron's activation can be quantified by the change in firing rate before and after finger movement, and the quantified value is then represented by the neural activity over each trial for the intended movement. Since this neural activity varies with the intended movement, we define the relative importance of each neuron independent of specific intended movements. The relative importance of each neuron is determined by the inter-movement variance of the neural activities for respective intended movements. Neurons are ranked by the relative importance and then a subpopulation of rank-ordered neurons is selected for the neural decoding. The use of the proposed neuron selection method in individual finger movements improved decoding accuracy by 21.5% in the case of decoding with only 5 neurons and by 9.2% in the case of decoding with only 10 neurons. With only 15 highly-ranked neurons, a decoding accuracy of 99.5% was achieved. The performance improvement is still maintained when combined movements of two fingers were included though the decoding accuracy fell to 95.7%. Since the proposed neuron selection method can achieve the targeting accuracy of decoding algorithms with less number of input neurons, it can be significant for developing brain-machine interfaces for direct neural control of hand prostheses.

Neuron selection for decoding dexterous finger movements

Many brain machine interfaces (BMI) seek to use the activity from hundreds of simultaneously recorded neurons to reconstruct an individual's kinematics. However, many of these neurons are not task related since there is no way to surgically target those neurons. This causes model based decoding to suffer easily from over-fitting on noisy unrelated neurons. Previous methods, such as correlation analysis and sensitivity analysis, seek to select neurons based on which reduced order model best matches the ensemble model and thus does not worry about over fitting. To address this issue, this paper presents a new method, cross model validation, that ranks neuron importance on the neuron model's ability to generalize well to data from correct movements and poorly to data from incorrect movements. This method attempts to highlight the neurons that are able to distinguish between movements the best and decode accurately. Selecting neurons using cross model validation scores as opposed to randomly selecting them can increase decoding accuracy up to 2.5 times or by 44%. These results showcase the importance of neuron selection in decoding and the ability of cross model validation in discerning each neuron's utility in decoding.

EMG prediction from motor cortical recordings via a nonnegative point-process filter

A constrained point-process filtering mechanism for prediction of electromyogram (EMG) signals from multichannel neural spike recordings is proposed here. Filters from the Kalman family are inherently suboptimal in dealing with non-Gaussian observations, or a state evolution that deviates from the Gaussianity assumption. To address these limitations, we modeled the non-Gaussian neural spike train observations by using a generalized linear model that encapsulates covariates of neural activity, including the neurons' own spiking history, concurrent ensemble activity, and extrinsic covariates (EMG signals). In order to predict the envelopes of EMGs, we reformulated the Kalman filter in an optimization framework and utilized a nonnegativity constraint. This structure characterizes the nonlinear correspondence between neural activity and EMG signals reasonably. The EMGs were recorded from 12 forearm and hand muscles of a behaving monkey during a grip-force task. In the case of limited training data, the constrained point-process filter improved the prediction accuracy when compared to a conventional Wiener cascade filter (a linear causal filter followed by a static nonlinearity) for different bin sizes and delays between input spikes and EMG output. For longer training datasets, results of the proposed filter and that of the Wiener cascade filter were comparable.

Comparisons between linear and nonlinear methods for decoding motor cortical activities of monkey

Brain Machine Interfaces (BMI) aim at building a direct communication link between the neural system and external devices. The decoding of neuronal signals is one of the important steps in BMI systems. Existing decoding methods commonly fall into two categories, i.e., linear methods and nonlinear methods. This paper compares the performance between the two kinds of methods in the decoding of motor cortical activities of a monkey. Kalman filter (KF) is chosen as an example of linear methods, and General Regression Neural Network (GRNN) and Support Vector Regression (SVR) are two nonlinear approaches evaluated in our work. The experiments are conducted to reconstruct 2D trajectories in a center-out task. The correlation coefficient (CC) and the root mean square error (RMSE) are used to assess the performance. The experimental results show that GRNN and SVR achieve better performance than Kalman filter with average improvements of about 30% in CC and 40% in RMSE. This demonstrates that nonlinear models can better encode the relationship between the neuronal signals and response. In addition, GRNN and SVR are more effective than Kalman filter on noisy data.

Sensing with the motor cortex

The primary motor cortex is a critical node in the network of brain regions responsible for voluntary motor behavior. It has been less appreciated, however, that the motor cortex exhibits sensory responses in a variety of modalities including vision and somatosensation. We review current work that emphasizes the heterogeneity in sensorimotor responses in the motor cortex and focus on its implications for cortical control of movement as well as for brain-machine interface development.

Adaptive decoding for brain-machine interfaces through Bayesian parameter updates

Brain-machine interfaces (BMIs) transform the activity of neurons recorded in motor areas of the brain into movements of external actuators. Representation of movements by neuronal populations varies over time, during both voluntary limb movements and movements controlled through BMIs, due to motor learning, neuronal plasticity, and instability in recordings. To ensure accurate BMI performance over long time spans, BMI decoders must adapt to these changes. We propose the Bayesian regression self-training method for updating the parameters of an unscented Kalman filter decoder. This novel paradigm uses the decoder's output to periodically update its neuronal tuning model in a Bayesian linear regression. We use two previously known statistical formulations of Bayesian linear regression: a joint formulation, which allows fast and exact inference, and a factorized formulation, which allows the addition and temporary omission of neurons from updates but requires approximate variational inference. To evaluate these methods, we performed offline reconstructions and closed-loop experiments with rhesus monkeys implanted cortically with microwire electrodes. Offline reconstructions used data recorded in areas M1, S1, PMd, SMA, and PP of three monkeys while they controlled a cursor using a handheld joystick. The Bayesian regression self-training updates significantly improved the accuracy of offline reconstructions compared to the same decoder without updates. We performed 11 sessions of real-time, closed-loop experiments with a monkey implanted in areas M1 and S1. These sessions spanned 29 days. The monkey controlled the cursor using the decoder with and without updates. The updates maintained control accuracy and did not require information about monkey hand movements, assumptions about desired movements, or knowledge of the intended movement goals as training signals. These results indicate that Bayesian regression self-training can maintain BMI control accuracy over long periods, making clinical neuroprosthetics more viable.

Use of a Bayesian maximum-likelihood classifier to generate training data for brain-machine interfaces

Brain-machine interface decoding algorithms need to be predicated on assumptions that are easily met outside of an experimental setting to enable a practical clinical device. Given present technological limitations, there is a need for decoding algorithms which (a) are not dependent upon a large number of neurons for control, (b) are adaptable to alternative sources of neuronal input such as local field potentials (LFPs), and (c) require only marginal training data for daily calibrations. Moreover, practical algorithms must recognize when the user is not intending to generate a control output and eliminate poor training data. In this paper, we introduce and evaluate a Bayesian maximum-likelihood estimation strategy to address the issues of isolating quality training data and self-paced control. Six animal subjects demonstrate that a multiple state classification task, loosely based on the standard center-out task, can be accomplished with fewer than five engaged neurons while requiring less than ten trials for algorithm training. In addition, untrained animals quickly obtained accurate device control, utilizing LFPs as well as neurons in cingulate cortex, two non-traditional neural inputs.

Fast attainment of computer cursor control with noninvasively acquired brain signals

Brain-computer interface (BCI) systems are allowing humans and non-human primates to drive prosthetic devices such as computer cursors and artificial arms with just their thoughts. Invasive BCI systems acquire neural signals with intracranial or subdural electrodes, while noninvasive BCI systems typically acquire neural signals with scalp electroencephalography (EEG). Some drawbacks of invasive BCI systems are the inherent risks of surgery and gradual degradation of signal integrity. A limitation of noninvasive BCI systems for two-dimensional control of a cursor, in particular those based on sensorimotor rhythms, is the lengthy training time required by users to achieve satisfactory performance. Here we describe a novel approach to continuously decoding imagined movements from EEG signals in a BCI experiment with reduced training time. We demonstrate that, using our noninvasive BCI system and observational learning, subjects were able to accomplish two-dimensional control of a cursor with performance levels comparable to those of invasive BCI systems. Compared to other studies of noninvasive BCI systems, training time was substantially reduced, requiring only a single session of decoder calibration (∼ 20 min) and subject practice (∼ 20 min). In addition, we used standardized low-resolution brain electromagnetic tomography to reveal that the neural sources that encoded observed cursor movement may implicate a human mirror neuron system. These findings offer the potential to continuously control complex devices such as robotic arms with one's mind without lengthy training or surgery.

Inference from populations: going beyond models

How are abstract signals, like intent, represented in neural populations? By creating a direct link between neural activity and behavior, brain-computer interfaces (BCIs) can help answer this question. Early instantiations of these devices sought mainly to mimic arm movements: by building models of arm tuning for the neurons, desired arm movements could be read out and used to control various prosthetic devices. However, as the functionality of these devices increases, a more general approach that relies less on endogenous control signals may be required. Here we review some of the current, model-based approaches for finding volitional control signals for spiking-based BCIs, and present some new approaches for finding control signals without resorting to parametric models of neural activity.

Breaking the fixed-arrival-time restriction in reaching movements of neural prosthetic devices

We routinely generate reaching arm movements to function independently. For paralyzed users of upper extremity neural prosthetic devices, flexible, high-performance reaching algorithms will be critical to restoring quality-of-life. Previously, algorithms called real-time reach state equations (RSE) were developed to integrate the user's plan and execution-related neural activity to drive reaching movements to arbitrary targets. Preliminary validation under restricted conditions suggested that RSE might yield dramatic performance improvements. Unfortunately, real-world applications of RSE have been impeded because the RSE assumes a fixed, known arrival time. Recent animal-based prototypes attempted to break the fixed-arrival-time assumption by proposing a standard model (SM) that instead restricted the user's movements to a fixed, known set of targets. Here, we leverage general purpose filter design (GPFD) to break both of these critical restrictions, freeing the paralyzed user to make reaching movements to arbitrary target sets with various arrival times and definitive stopping. In silico validation predicts that the new approach, GPFD-RSE, outperforms the SM while offering greater flexibility. We demonstrate the GPFD-RSE against SM in the simulated control of an overactuated 3-D virtual robotic arm with a real-time inverse kinematics engine.

A method for investigating cortical control of stand and squat in conscious behavioral monkeys

Technical challenges to record cortical unit spikes in conscious behavior monkeys have been presenting constraints in the investigation of cortical control of lower limb. Here we report a novel experimental system for investigation of correlation between cortical neuronal signals and standing and squatting movement. The system consists of a specially designed chair, a standing and squatting task training paradigm and a neuron recording setup with microdrivable electrodes. With this system, a monkey can perform standing up and squatting down tasks while the head is relatively stationary to allow accurate single cortical unit recordings. The spike activities of single motor cortical units, the kinematics and muscle activities from monkeys functioning lower limb motion tasks were successfully recorded simultaneously for analyses of cortical control of lower limb functions and correlations between activity patterns of cortical neurons and lower limb muscles.

Spiking neural networks for cortical neuronal spike train decoding

Recent investigation of cortical coding and computation indicates that temporal coding is probably a more biologically plausible scheme used by neurons than the rate coding used commonly in most published work. We propose and demonstrate in this letter that spiking neural networks (SNN), consisting of spiking neurons that propagate information by the timing of spikes, are a better alternative to the coding scheme based on spike frequency (histogram) alone. The SNN model analyzes cortical neural spike trains directly without losing temporal information for generating more reliable motor command for cortically controlled prosthetics. In this letter, we compared the temporal pattern classification result from the SNN approach with results generated from firing-rate-based approaches: conventional artificial neural networks, support vector machines, and linear regression. The results show that the SNN algorithm can achieve higher classification accuracy and identify the spiking activity related to movement control earlier than the other methods. Both are desirable characteristics for fast neural information processing and reliable control command pattern recognition for neuroprosthetic applications.