Selection and parameterization of cortical neurons for neuroprosthetic control

Neurons in primary motor cortex engaged during action observation

Neurons in higher cortical areas appear to become active during action observation, either by mirroring observed actions (termed mirror neurons) or by eliciting mental rehearsal of observed motor acts. We report the existence of neurons in the primary motor cortex (M1), an area that is generally considered to initiate and guide movement performance, responding to viewed actions. Multielectrode recordings in monkeys performing or observing a well-learned step-tracking task showed that approximately half of the M1 neurons that were active when monkeys performed the task were also active when they observed the action being performed by a human. These 'view' neurons were spatially intermingled with 'do' neurons, which are active only during movement performance. Simultaneously recorded 'view' neurons comprised two groups: approximately 38% retained the same preferred direction (PD) and timing during performance and viewing, and the remainder (62%) changed their PDs and time lag during viewing as compared with performance. Nevertheless, population activity during viewing was sufficient to predict the direction and trajectory of viewed movements as action unfolded, although less accurately than during performance. 'View' neurons became less active and contained poorer representations of action when only subcomponents of the task were being viewed. M1 'view' neurons thus appear to reflect aspects of a learned movement when observed in others, and form part of a broadly engaged set of cortical areas routinely responding to learned behaviors. These findings suggest that viewing a learned action elicits replay of aspects of M1 activity needed to perform the observed action, and could additionally reflect processing related to understanding, learning or mentally rehearsing action.

Ensemble fractional sensitivity: a quantitative approach to neuron selection for decoding motor tasks

A robust method to help identify the population of neurons used for decoding motor tasks is developed. We use sensitivity analysis to develop a new metric for quantifying the relative contribution of a neuron towards the decoded output, called "fractional sensitivity." Previous model-based approaches for neuron ranking have been shown to largely depend on the collection of training data. We suggest the use of an ensemble of models that are trained on random subsets of trials to rank neurons. For this work, we tested a decoding algorithm on neuronal data recorded from two male rhesus monkeys while they performed a reach to grasp a bar at three orientations (45 degrees, 90 degrees, or 135 degrees). An ensemble approach led to a statistically significant increase of 5% in decoding accuracy and 25% increase in identification accuracy of simulated noisy neurons, when compared to a single model. Furthermore, ranking neurons based on the ensemble fractional sensitivities resulted in decoding accuracies 10%-20% greater than when randomly selecting neurons or ranking based on firing rates alone. By systematically reducing the size of the input space, we determine the optimal number of neurons needed for decoding the motor output. This selection approach has practical benefits for other BMI applications where limited number of electrodes and training datasets are available, but high decoding accuracies are desirable.

Unscented Kalman filter for brain-machine interfaces

Brain machine interfaces (BMIs) are devices that convert neural signals into commands to directly control artificial actuators, such as limb prostheses. Previous real-time methods applied to decoding behavioral commands from the activity of populations of neurons have generally relied upon linear models of neural tuning and were limited in the way they used the abundant statistical information contained in the movement profiles of motor tasks. Here, we propose an n-th order unscented Kalman filter which implements two key features: (1) use of a non-linear (quadratic) model of neural tuning which describes neural activity significantly better than commonly-used linear tuning models, and (2) augmentation of the movement state variables with a history of n-1 recent states, which improves prediction of the desired command even before incorporating neural activity information and allows the tuning model to capture relationships between neural activity and movement at multiple time offsets simultaneously. This new filter was tested in BMI experiments in which rhesus monkeys used their cortical activity, recorded through chronically implanted multielectrode arrays, to directly control computer cursors. The 10th order unscented Kalman filter outperformed the standard Kalman filter and the Wiener filter in both off-line reconstruction of movement trajectories and real-time, closed-loop BMI operation.

Exploiting multiple sensory modalities in brain-machine interfaces

Recent improvements in cortically-controlled brain-machine interfaces (BMIs) have raised hopes that such technologies may improve the quality of life of severely motor-disabled patients. However, current generation BMIs do not perform up to their potential due to the neglect of the full range of sensory feedback in their strategies for training and control. Here we confirm that neurons in the primary motor cortex (MI) encode sensory information and demonstrate a significant heterogeneity in their responses with respect to the type of sensory modality available to the subject about a reaching task. We further show using mutual information and directional tuning analyses that the presence of multi-sensory feedback (i.e. vision and proprioception) during replay of movements evokes neural responses in MI that are almost indistinguishable from those responses measured during overt movement. Finally, we suggest how these playback-evoked responses may be used to improve BMI performance.

Optimal input selection for neural machine interfaces predicting multiple non-explicit outputs

This study implemented a novel algorithm that optimally selects inputs for neural machine interface (NMI) devices intended to control multiple outputs and evaluated its performance on systems lacking explicit output. NMIs often incorporate signals from multiple physiological sources and provide predictions for multidimensional control, leading to multiple-input multiple-output systems. Further, NMIs often are used with subjects who have motor disabilities and thus lack explicit motor outputs. Our algorithm was tested on simulated multiple-input multiple-output systems and on electromyogram and kinematic data collected from healthy subjects performing arm reaches. Effects of output noise in simulated systems indicated that the algorithm could be useful for systems with poor estimates of the output states, as is true for systems lacking explicit motor output. To test efficacy on physiological data, selection was performed using inputs from one subject and outputs from a different subject. Selection was effective for these cases, again indicating that this algorithm will be useful for predictions where there is no motor output, as often is the case for disabled subjects. Further, prediction results generalized for different movement types not used for estimation. These results demonstrate the efficacy of this algorithm for the development of neural machine interfaces.

A cluster analysis of neuronal activity in the dorsal premotor cortical area for neuroprosthetic control

With the use of the neuronal data acquisition technology, millisecond-level multi-electrode data from several regions of the premotor area were obtained from two rhesus monkeys trained to perform arm-reach tasks with visual cues in virtual reality. In each trial, animals were required to select and perform one of the four possible arm reaching movements to the target on the top-left or top-right of the virtual reality space. They were also required to decide whether they would move their arms straight to the target or curve them in order to avoid the obstacle that was presented. After the acquired neuronal signals were processed, unsupervised Hierarchical clustering and K-means clustering were performed to uncover the similarity and difference in the average firing rate of spike train data between neurons and phases for each experiment condition. The clustering results indicate the similarity of neuronal data in the movement planning and actual movement phases, and the difference of such data from the data in information processing phases. Furthermore, the clustering results show that when the target location is on the right, the move planning started earlier. The analysis of variance (ANOVA) on the neuronal data confirms the results from the hierarchical clustering.

Observation-based learning for brain-machine interfaces

Canonically, 'mirror neurons' are cells in area F5 of the ventral premotor cortex that are active during both observation and execution of goal-directed movements. Recently, cells with similar properties have been observed in a number of other areas in the motor system, including the primary motor cortex. Mirror neurons are a part of a system whose function is thought to involve the prediction and interpretation of the sensory consequences of our own actions as well as the actions of others. Mirror-like responses are relevant to the development of brain-machine interfaces (BMIs) because they provide a robust way to map neural activity to behavior, and because they represent high-level information about goals and intentions that may have utility in future BMI applications.

Toward optimal target placement for neural prosthetic devices

Neural prosthetic systems have been designed to estimate continuous reach trajectories (motor prostheses) and to predict discrete reach targets (communication prostheses). In the latter case, reach targets are typically decoded from neural spiking activity during an instructed delay period before the reach begins. Such systems use targets placed in radially symmetric geometries independent of the tuning properties of the neurons available. Here we seek to automate the target placement process and increase decode accuracy in communication prostheses by selecting target locations based on the neural population at hand. Motor prostheses that incorporate intended target information could also benefit from this consideration. We present an optimal target placement algorithm that approximately maximizes decode accuracy with respect to target locations. In simulated neural spiking data fit from two monkeys, the optimal target placement algorithm yielded statistically significant improvements up to 8 and 9% for two and sixteen targets, respectively. For four and eight targets, gains were more modest, as the target layouts found by the algorithm closely resembled the canonical layouts. We trained a monkey in this paradigm and tested the algorithm with experimental neural data to confirm some of the results found in simulation. In all, the algorithm can serve not only to create new target layouts that outperform canonical layouts, but it can also confirm or help select among multiple canonical layouts. The optimal target placement algorithm developed here is the first algorithm of its kind, and it should both improve decode accuracy and help automate target placement for neural prostheses.

Cortical control of a prosthetic arm for self-feeding

Arm movement is well represented in populations of neurons recorded from the motor cortex. Cortical activity patterns have been used in the new field of brain-machine interfaces to show how cursors on computer displays can be moved in two- and three-dimensional space. Although the ability to move a cursor can be useful in its own right, this technology could be applied to restore arm and hand function for amputees and paralysed persons. However, the use of cortical signals to control a multi-jointed prosthetic device for direct real-time interaction with the physical environment ('embodiment') has not been demonstrated. Here we describe a system that permits embodied prosthetic control; we show how monkeys (Macaca mulatta) use their motor cortical activity to control a mechanized arm replica in a self-feeding task. In addition to the three dimensions of movement, the subjects' cortical signals also proportionally controlled a gripper on the end of the arm. Owing to the physical interaction between the monkey, the robotic arm and objects in the workspace, this new task presented a higher level of difficulty than previous virtual (cursor-control) experiments. Apart from an example of simple one-dimensional control, previous experiments have lacked physical interaction even in cases where a robotic arm or hand was included in the control loop, because the subjects did not use it to interact with physical objects-an interaction that cannot be fully simulated. This demonstration of multi-degree-of-freedom embodied prosthetic control paves the way towards the development of dexterous prosthetic devices that could ultimately achieve arm and hand function at a near-natural level.

Congruent activity during action and action observation in motor cortex

A variety of studies have shown that motor cortical areas can be activated by observation of familiar actions. Here, we describe single-neuron responses in monkey primary motor (MI) and dorsal premotor (PMd) cortices during passive observation and execution of a familiar task. We show that the spiking modulation, preferred directions, and encoded information of cells in MI and PMd remain consistent during both observation and movement. Furthermore, we find that the presence of a visual target is necessary to elicit this congruent neural activity during observation. These findings along with results from our analysis of the oscillatory power in the beta frequency of the local field potential are consistent with previous imaging and EEG studies that have suggested that congruence between observation and action is a general feature of the motor system, even outside of canonical "mirror" areas. Such congruent activity has proposed relevance to motor learning, mimicry, and communication and has practical applications for the development of motor-cortical neuroprostheses in paralyzed patients.

Biomimetic brain machine interfaces for the control of movement

Quite recently, it has become possible to use signals recorded simultaneously from large numbers of cortical neurons for real-time control. Such brain machine interfaces (BMIs) have allowed animal subjects and human patients to control the position of a computer cursor or robotic limb under the guidance of visual feedback. Although impressive, such approaches essentially ignore the dynamics of the musculoskeletal system, and they lack potentially critical somatosensory feedback. In this mini-symposium, we will initiate a discussion of systems that more nearly mimic the control of natural limb movement. The work that we will describe is based on fundamental observations of sensorimotor physiology that have inspired novel BMI approaches. We will focus on what we consider to be three of the most important new directions for BMI development related to the control of movement. (1) We will present alternative methods for building decoders, including structured, nonlinear models, the explicit incorporation of limb state information, and novel approaches to the development of decoders for paralyzed subjects unable to generate an output signal. (2) We will describe the real-time prediction of dynamical signals, including joint torque, force, and EMG, and the real-time control of physical plants with dynamics like that of the real limb. (3) We will discuss critical factors that must be considered to incorporate somatosensory feedback to the BMI user, including its potential benefits, the differing representations of sensation and perception across cortical areas, and the changes in the cortical representation of tactile events after spinal injury.

General-Purpose Filter Design for Neural Prosthetic Devices

Brain-driven interfaces depend on estimation procedures to convert neural signals to inputs for prosthetic devices that can assist individuals with severe motor deficits. Previous estimation procedures were developed on an application-specific basis. Here we report a coherent estimation framework that unifies these procedures and motivates new applications of prosthetic devices driven by action potentials, local field potentials (LFPs), electrocorticography (ECoG), electroencephalography (EEG), electromyography (EMG), or optical methods. The brain-driven interface is described as a probabilistic relationship between neural activity and components of a prosthetic device that may take on discrete or continuous values. A new estimation procedure is developed for action potentials, and a corresponding procedure is described for field potentials and optical measurements. We test our framework against dominant approaches in an arm reaching task using simulated traces of ensemble spiking activity from primary motor cortex (MI) and a wheelchair navigation task using simulated traces of EEG-band power. Adaptive filtering is incorporated to demonstrate performance under neuron death and discovery. Finally, we characterize performance under model misspecification using physiologically realistic history dependence in MI spiking. These simulated results predict that the unified framework outperforms previous approaches under various conditions, in the control of position and velocity, based on trajectory and endpoint mean squared errors.

Brain-Controlled Interfaces: Movement Restoration with Neural Prosthetics

Brain-controlled interfaces are devices that capture brain transmissions involved in a subject's intention to act, with the potential to restore communication and movement to those who are immobilized. Current devices record electrical activity from the scalp, on the surface of the brain, and within the cerebral cortex. These signals are being translated to command signals driving prosthetic limbs and computer displays. Somatosensory feedback is being added to this control as generated behaviors become more complex. New technology to engineer the tissue-electrode interface, electrode design, and extraction algorithms to transform the recorded signal to movement will help translate exciting laboratory demonstrations to patient practice in the near future.