Grasp movement decoding from premotor and parietal cortex

Accurate neural control of a hand prosthesis by posture-related activity in the primate grasping circuit

Brain-computer interfaces (BCIs) have the potential to restore hand movement for people with paralysis, but current devices still lack the fine control required to interact with objects of daily living. Following our understanding of cortical activity during arm reaches, hand BCI studies have focused primarily on velocity control. However, mounting evidence suggests that posture, and not velocity, dominates in hand-related areas. To explore whether this signal can causally control a prosthesis, we developed a BCI training paradigm centered on the reproduction of posture transitions. Monkeys trained with this protocol were able to control a multidimensional hand prosthesis with high accuracy, including execution of the very intricate precision grip. Analysis revealed that the posture signal in the target grasping areas was the main contributor to control. We present, for the first time, neural posture control of a multidimensional hand prosthesis, opening the door for future interfaces to leverage this additional information channel.

Application of deep reinforcement learning for spike sorting under multi-class imbalance

Spike sorting is the basis for analyzing spike firing patterns encoded in high-dimensional information spaces. With the fact that high-density microelectrode arrays record multiple neurons simultaneously, the data collected often suffers from two problems: a few overlapping spikes and different neuronal firing rates, which both belong to the multi-class imbalance problem. Since deep reinforcement learning (DRL) assign targeted attention to categories through reward functions, we propose ImbSorter to implement spike sorting under multi-class imbalance. We describe spike sorting as a Markov sequence decision and construct a dynamic reward function (DRF) to improve the sensitivity of the agent to minor classes based on the inter-class imbalance ratios. The agent is eventually guided by the optimal strategy to classify spikes. We consider the Wave_Clus dataset, which contains overlapping spikes and diverse noise levels, and the macaque dataset, which has a multi-scale imbalance. ImbSorter is compared with classical DRL architectures, traditional machine learning algorithms, and advanced overlapping spike sorting techniques on these two above datasets. ImbSorter obtained improved results on the Macro_F1. The results show ImbSorter has a promising ability to resist overlapping and noise interference. It has high stability and promising performance in processing spikes with different degrees of skewed distribution.

Thin flexible arrays for long-term multi-electrode recordings in macaque primary visual cortex

Objective.Basic, translational and clinical neuroscience are increasingly focusing on large-scale invasive recordings of neuronal activity. However, in large animals such as nonhuman primates and humans-in which the larger brain size with sulci and gyri imposes additional challenges compared to rodents, there is a huge unmet need to record from hundreds of neurons simultaneously anywhere in the brain for long periods of time. Here, we tested the electrical and mechanical properties of thin, flexible multi-electrode arrays (MEAs) inserted into the primary visual cortex of two macaque monkeys, and assessed their magnetic resonance imaging (MRI) compatibility and their capacity to record extracellular activity over a period of 1 year.Approach.To allow insertion of the floating arrays into the visual cortex, the 20 by 100µm²shafts were temporarily strengthened by means of a resorbable poly(lactic-co-glycolic acid) coating.Main results. After manual insertion of the arrays, theex vivoandin vivoMRI compatibility of the arrays proved to be excellent. We recorded clear single-unit activity from up to 50% of the electrodes, and multi-unit activity (MUA) on 60%-100% of the electrodes, which allowed detailed measurements of the receptive fields and the orientation selectivity of the neurons. Even 1 year after insertion, we obtained significant MUA responses on 70%-100% of the electrodes, while the receptive fields remained remarkably stable over the entire recording period.Significance.Thus, the thin and flexible MEAs we tested offer several crucial advantages compared to existing arrays, most notably in terms of brain tissue compliance, scalability, and brain coverage. Future brain-machine interface applications in humans may strongly benefit from this new generation of chronically implanted MEAs.

Reconstructing Synergy-Based Hand Grasp Kinematics from Electroencephalographic Signals

Brain-machine interfaces (BMIs) have become increasingly popular in restoring the lost motor function in individuals with disabilities. Several research studies suggest that the CNS may employ synergies or movement primitives to reduce the complexity of control rather than controlling each DoF independently, and the synergies can be used as an optimal control mechanism by the CNS in simplifying and achieving complex movements. Our group has previously demonstrated neural decoding of synergy-based hand movements and used synergies effectively in driving hand exoskeletons. In this study, ten healthy right-handed participants were asked to perform six types of hand grasps representative of the activities of daily living while their neural activities were recorded using electroencephalography (EEG). From half of the participants, hand kinematic synergies were derived, and a neural decoder was developed, based on the correlation between hand synergies and corresponding cortical activity, using multivariate linear regression. Using the synergies and the neural decoder derived from the first half of the participants and only cortical activities from the remaining half of the participants, their hand kinematics were reconstructed with an average accuracy above 70%. Potential applications of synergy-based BMIs for controlling assistive devices in individuals with upper limb motor deficits, implications of the results in individuals with stroke and the limitations of the study were discussed.

Navigating the Statistical Minefield of Model Selection and Clustering in Neuroscience

Model selection is often implicit: when performing an ANOVA, one assumes that the normal distribution is a good model of the data; fitting a tuning curve implies that an additive and a multiplicative scaler describes the behavior of the neuron; even calculating an average implicitly assumes that the data were sampled from a distribution that has a finite first statistical moment: the mean. Model selection may be explicit, when the aim is to test whether one model provides a better description of the data than a competing one. As a special case, clustering algorithms identify groups with similar properties within the data. They are widely used from spike sorting to cell type identification to gene expression analysis. We discuss model selection and clustering techniques from a statistician's point of view, revealing the assumptions behind, and the logic that governs the various approaches. We also showcase important neuroscience applications and provide suggestions how neuroscientists could put model selection algorithms to best use as well as what mistakes should be avoided.

Theta low-gamma phase amplitude coupling in the human orbitofrontal cortex increases during a conflict-processing task

Objective. The human orbitofrontal cortex (OFC) is involved in automatic response inhibition and conflict processing, but the mechanism of frequency-specific power changes that control these functions is unknown. Theta and gamma activity have been independently observed in the OFC during conflict processing, while theta-gamma interactions in other brain areas have been noted primarily in studies of memory. Within the OFC, it is possible that theta-gamma phase amplitude coupling (PAC) drives conflict processing. This study aims to characterize the coupled relationship between theta and gamma frequency bands in the OFC during conflict processing using a modified Stroop task.Approach. Eight epilepsy patients implanted with OFC stereotactic electroencephalography electrodes participated in a color-word modified Stroop task. PAC between theta phase and gamma amplitude was assessed to determine the timing and magnitude of neural oscillatory changes. Group analysis was conducted using a non-parametric cluster-permutationt-test on coherence values.Main results.Theta-low gamma (LG) PAC significantly increased in five out of eight patients during successful trials of the incongruent condition compared with the congruent condition. Significant increases in theta-LG PAC were most prominent during cue processing 200-800 ms after cue presentation. On group analysis, trial-averaged mean theta-LG PAC was statistically significantly greater in the incongruent condition compared to the congruent condition (p< 0.001, Cohen'sd= 0.51).Significance.For the first time, we report that OFC theta phase and LG amplitude coupling increases during conflict resolution. Given the delayed onset after cue presentation, OFC theta-LG PAC may contribute to conflict processing after conflict detection and before motor response. This explanation follows the hypothesis that global theta waves modulate local gamma signals. Understanding this relationship within the OFC will help further elucidate the neural mechanisms of human conflict resolution.

Hippocampal and Orbitofrontal Theta Band Coherence Diminishes During Conflict Resolution

OBJECTIVE

Coherence between the hippocampus and other brain structures has been shown with the theta frequency (3-8 Hz). Cortical decreases in theta coherence are believed to reflect response accuracy efficiency. However, the role of theta coherence during conflict resolution is poorly understood in noncortical areas. In this study, coherence between the hippocampus and orbitofrontal cortex (OFC) was measured during a conflict resolution task. Although both brain areas have been previously implicated in the Stroop task, their interactions are not well understood.

METHODS

Nine patients were implanted with stereotactic electroencephalography contacts in the hippocampus and OFC. Local field potential data were sampled throughout discrete phases of a Stroop task. Coherence was calculated for hippocampal and OFC contact pairs, and coherence spectrograms were constructed for congruent and incongruent conditions. Coherence changes during cue processing were identified using a nonparametric cluster-permutation t test. Group analysis was conducted to compare overall theta coherence changes among conditions.

RESULTS

In 6 of 9 patients, decreased theta coherence was observed only during the incongruent condition (P < 0.05). Congruent theta coherence did not change from baseline. Group analysis showed lower theta coherence for the incongruent condition compared with the congruent condition (P < 0.05).

CONCLUSIONS

Theta coherence between the hippocampus and OFC decreased during conflict. This finding supports existing theories that theta coherence desynchronization contributes to improved response accuracy and processing efficiency during conflict resolution. The underlying theta coherence observed between the hippocampus and OFC during conflict may be distinct from its previously observed role in memory.

The Neural Representation of Force across Grasp Types in Motor Cortex of Humans with Tetraplegia

Intracortical brain-computer interfaces (iBCIs) have the potential to restore hand grasping and object interaction to individuals with tetraplegia. Optimal grasping and object interaction require simultaneous production of both force and grasp outputs. However, since overlapping neural populations are modulated by both parameters, grasp type could affect how well forces are decoded from motor cortex in a closed-loop force iBCI. Therefore, this work quantified the neural representation and offline decoding performance of discrete hand grasps and force levels in two human participants with tetraplegia. Participants attempted to produce three discrete forces (light, medium, hard) using up to five hand grasp configurations. A two-way Welch ANOVA was implemented on multiunit neural features to assess their modulation to force and grasp Demixed principal component analysis (dPCA) was used to assess for population-level tuning to force and grasp and to predict these parameters from neural activity. Three major findings emerged from this work: (1) force information was neurally represented and could be decoded across multiple hand grasps (and, in one participant, across attempted elbow extension as well); (2) grasp type affected force representation within multiunit neural features and offline force classification accuracy; and (3) grasp was classified more accurately and had greater population-level representation than force. These findings suggest that force and grasp have both independent and interacting representations within cortex, and that incorporating force control into real-time iBCI systems is feasible across multiple hand grasps if the decoder also accounts for grasp type.

Feature-Selection-Based Transfer Learning for Intracortical Brain-Machine Interface Decoding

The time spent in collecting current samples for decoder calibration and the computational burden brought by high-dimensional neural recordings remain two challenging problems in intracortical brain-machine interfaces (iBMIs). Decoder calibration optimization approaches have been proposed, and neuron selection methods have been used to reduce computational burden. However, few methods can solve both problems simultaneously. In this article, we present a symmetrical-uncertainty-based transfer learning (SUTL) method that combines transfer learning with feature selection. The proposed method uses symmetrical uncertainty to quantitatively measure three indices for feature selection: stationarity, importance and redundancy of the feature. By selecting the stationary features, the disparities between the historical data and current data can be diminished, and the historical data can be effectively used for decoder calibration, thereby reducing the demand for current data. After selecting the important and non-redundant features, only the channels corresponding to them need to work; thus, the computational burden is reduced. The proposed method was tested on neural data recorded from two rhesus macaques to decode the reaching position or grasping gesture. The results showed that the SUTL method diminished the disparities between the historical data and current data, while achieving superior decoding performance with the needs of only ten current samples each category, less than 10% the number of features and 30% the number of neural recording channels. Additionally, unlike most studies on iBMIs, feature selection was implemented instead of neuron selection, and the average decoding accuracy achieved by the former was 6.6% higher.

Multi-source domain adaptation for decoder calibration of intracortical brain-machine interface

OBJECTIVE

For nonstationarity of neural recordings, daily retraining is required in the decoder calibration of intracortical brain-machine interfaces (iBMIs). Domain adaptation has started to be applied in iBMIs to solve the problem of daily retraining by taking advantage of historical data. However, previous domain adaptation studies used only a single source domain, which might lead to performance instability. In this study, we proposed a multi-source domain adaptation algorithm, by fully utilizing the historical data, to achieve a better and more robust decoding performance while reducing the decoder calibration time.

APPROACH

The neural signals were recorded from two rhesus macaques using intracortical electrodes to decode the reaching and grasping movements. A principal component analysis-based multi-source domain adaptation algorithm was proposed to apply the feature transfer to diminish the disparities between the target domain and each source domain. Moreover, the multiple weighted sub-classifiers based on multi-source domain data and small current sample set were constructed to accomplish the decoding.

MAIN RESULTS

Our algorithm was able to make use of the multi-source domain data and achieve better and more robust decoding performance compared with other methods. Only a small current sample set was needed by our algorithm in order for the decoder calibration time to be effectively reduced.

SIGNIFICANCE

(1) The idea of the multi-source domain adaptation was introduced into the iBMIs to solve the problem of time consumption in the daily decoder retraining. (2) Instead of using only single-source domain data in the previous study, our algorithm made use of multi-day historical data, resulting in better and more robust decoding performance. (3) Our algorithm could be accomplished with only a small current sample set, and it can effectively reduce the decoder calibration time, which is important for further clinical applications.

Beta-band modulation in the human hippocampus during a conflict response task

Objective Identify the role of beta-band (13-30 Hz) power modulation in the human hippocampus during conflict processing. Approach We investigated changes in the spectral power of the beta band (13-30 Hz) as measured by depth electrode leads in the hippocampus during a modified Stroop task in six patients with medically-refractory epilepsy. Previous work done with direct electrophysiological recordings in humans has shown hippocampal theta-band (3-8 Hz) modulation during conflict processing. Local field potentials (LFP) sampled at 2k Hz were used for analysis and a non-parametric cluster-permutation t-test was used to identify the time period and frequency ranges of significant power change during cue processing (i.e. post-stimulus, pre-response). Main Results In five of the six patients, we observe a statistically significant increase in hippocampal beta-band power during successful conflict processing in the incongruent trial condition (cluster-based correction for multiple comparisons, p < 0.05). There was no significant beta-band power change observed during the cue processing period of the congruent condition in the hippocampus of these patients. Significance The beta-power changes during conflict processing represented here are consistent with previous studies suggesting that the hippocampus plays a role in conflict processing, but it is the first time that the beta band has been shown to be involved in humans with direct electrophysiological evidence. We propose that beta-band modulation plays a role in successful conflict detection and automatic response inhibition in the human hippocampus as studied during a conflict response task.

Decoding Grasping Movements from the Parieto-Frontal Reaching Circuit in the Nonhuman Primate

Prehension movements typically include a reaching phase, guiding the hand toward the object, and a grip phase, shaping the hand around it. The dominant view posits that these components rely upon largely independent parieto-frontal circuits: a dorso-medial circuit involved in reaching and a dorso-lateral circuit involved in grasping. However, mounting evidence suggests a more complex arrangement, with dorso-medial areas contributing to both reaching and grasping. To investigate the role of the dorso-medial reaching circuit in grasping, we trained monkeys to reach-and-grasp different objects in the dark and determined if hand configurations could be decoded from functional magnetic resonance imaging (MRI) responses obtained from the reaching and grasping circuits. Indicative of their established role in grasping, object-specific grasp decoding was found in anterior intraparietal (AIP) area, inferior parietal lobule area PFG and ventral premotor region F5 of the lateral grasping circuit, and primary motor cortex. Importantly, the medial reaching circuit also conveyed robust grasp-specific information, as evidenced by significant decoding in parietal reach regions (particular V6A) and dorsal premotor region F2. These data support the proposed role of dorso-medial "reach" regions in controlling aspects of grasping and demonstrate the value of complementing univariate with more sensitive multivariate analyses of functional MRI (fMRI) data in uncovering information coding in the brain.

Exploring representations of human grasping in neural, muscle and kinematic signals

Movement covariates, such as electromyographic or kinematic activity, have been proposed as candidates for the neural representation of hand control. However, it remains unclear how these movement covariates are reflected in electroencephalographic (EEG) activity during different stages of grasping movements. In this exploratory study, we simultaneously acquired EEG, kinematic and electromyographic recordings of human subjects performing 33 types of grasps, yielding the largest such dataset to date. We observed that EEG activity reflected different movement covariates in different stages of grasping. During the pre-shaping stage, centro-parietal EEG in the lower beta frequency band reflected the object's shape and size, whereas during the finalization and holding stages, contralateral parietal EEG in the mu frequency band reflected muscle activity. These findings contribute to the understanding of the temporal organization of neural grasping patterns, and could inform the design of noninvasive neuroprosthetics and brain-computer interfaces with more natural control.

Neural Dynamics of Variable Grasp-Movement Preparation in the Macaque Frontoparietal Network

Our voluntary grasping actions lie on a continuum between immediate action and waiting for the right moment, depending on the context. Therefore, studying grasping requires an investigation into how preparation time affects this process. Two macaque monkeys (Macaca mulatta; one male, one female) performed a grasping task with a short instruction followed by an immediate or delayed go cue (0-1300 ms) while we recorded in parallel from neurons in the grasp preparation relevant area F5 that is part of the ventral premotor cortex, and the anterior intraparietal area (AIP). Initial population dynamics followed a fixed trajectory in the neural state space unique to each grip type, reflecting unavoidable movement selection, then diverged depending on the delay, reaching unique states not achieved for immediately cued movements. Population activity in the AIP was less dynamic, whereas F5 activity continued to evolve throughout the delay. Interestingly, neuronal populations from both areas allowed for a readout tracking subjective anticipation of the go cue that predicted single-trial reaction time. However, the prediction of reaction time was better from F5 activity. Intriguingly, activity during movement initiation clustered into two trajectory groups, corresponding to movements that were either "as fast as possible" or withheld movements, demonstrating a widespread state shift in the frontoparietal grasping network when movements must be withheld. Our results reveal how dissociation between immediate and delay-specific preparatory activity, as well as differentiation between cortical areas, is possible through population-level analysis.SIGNIFICANCE STATEMENT Sometimes when we move, we consciously plan our movements. At other times, we move instantly, seemingly with no planning at all. Yet, it's unclear how preparation for movements along this spectrum of planned and seemingly unplanned movement differs in the brain. Two macaque monkeys made reach-to-grasp movements after varying amounts of preparation time while we recorded from the premotor and parietal cortex. We found that the initial response to a grasp instruction was specific to the required movement, but not to the preparation time, reflecting required movement selection. However, when more preparation time was given, neural activity achieved unique states that likely related to withholding movements and anticipation of movement, shedding light on the roles of the premotor and parietal cortex in grasp planning.

Hierarchical Organization Within the Ventral Premotor Cortex of the Macaque Monkey

Recent studies have revealed that the ventral premotor cortex (PMv) of nonhuman primates plays a pivotal role in various behaviors that require the transformation of sensory cues to appropriate actions. Examples include decision-making based on various sensory cues, preparation for upcoming motor behavior, adaptive sensorimotor transformation, and the generation of motor commands using rapid sensory feedback. Although the PMv has frequently been regarded as a single entity, it can be divided into at least five functionally distinct regions: F4, a dorsal convexity region immediately rostral to the primary motor cortex (M1); F5p, a cortical region immediately rostral to F4, lying within the arcuate sulcus; F5c, a ventral convexity region rostral to F4; and F5a, located in the caudal bank of the arcuate sulcus inferior limb lateral to F5p. Among these, F4 can be further divided into dorsal and ventral subregions (F4d and F4v), which are involved in forelimb and orofacial movements, respectively. F5p contains "mirror neurons" to understand others' actions based on visual and other types of information, and F4d and F5p work together as a functional complex involved in controlling forelimb and eye movements, most efficiently in the execution and completion of coordinated eye-hand movements for reaching and grasping under visual guidance. In contrast, F5c and F5a are hierarchically higher than the F4d, F5p, and F5v complexes, and play a role in decision-making based on various sensory discriminations. Hence, the PMv subregions form a hierarchically organized integral system from decision-making to eye-hand coordination under various behavioral circumstances.

Modeling task-specific neuronal ensembles improves decoding of grasp

OBJECTIVE

Dexterous movement involves the activation and coordination of networks of neuronal populations across multiple cortical regions. Attempts to model firing of individual neurons commonly treat the firing rate as directly modulating with motor behavior. However, motor behavior may additionally be associated with modulations in the activity and functional connectivity of neurons in a broader ensemble. Accounting for variations in neural ensemble connectivity may provide additional information about the behavior being performed.

APPROACH

In this study, we examined neural ensemble activity in primary motor cortex (M1) and premotor cortex (PM) of two male rhesus monkeys during performance of a center-out reach, grasp and manipulate task. We constructed point process encoding models of neuronal firing that incorporated task-specific variations in the baseline firing rate as well as variations in functional connectivity with the neural ensemble. Models were evaluated both in terms of their encoding capabilities and their ability to properly classify the grasp being performed.

MAIN RESULTS

Task-specific ensemble models correctly predicted the performed grasp with over 95% accuracy and were shown to outperform models of neuronal activity that assume only a variable baseline firing rate. Task-specific ensemble models exhibited superior decoding performance in 82% of units in both monkeys (p  <  0.01). Inclusion of ensemble activity also broadly improved the ability of models to describe observed spiking. Encoding performance of task-specific ensemble models, measured by spike timing predictability, improved upon baseline models in 62% of units.

SIGNIFICANCE

These results suggest that additional discriminative information about motor behavior found in the variations in functional connectivity of neuronal ensembles located in motor-related cortical regions is relevant to decode complex tasks such as grasping objects, and may serve the basis for more reliable and accurate neural prosthesis.

Population coding of grasp and laterality-related information in the macaque fronto-parietal network

Preparing and executing grasping movements demands the coordination of sensory information across multiple scales. The position of an object, required hand shape, and which of our hands to extend must all be coordinated in parallel. The network formed by the macaque anterior intraparietal area (AIP) and hand area (F5) of the ventral premotor cortex is essential in the generation of grasping movements. Yet, the role of this circuit in hand selection is unclear. We recorded from 1342 single- and multi-units in AIP and F5 of two macaque monkeys (Macaca mulatta) during a delayed grasping task in which monkeys were instructed by a visual cue to perform power or precision grips on a handle presented in five different orientations with either the left or right hand, as instructed by an auditory tone. In AIP, intended hand use (left vs. right) was only weakly represented during preparation, while hand use was robustly present in F5 during preparation. Interestingly, visual-centric handle orientation information dominated AIP, while F5 contained an additional body-centric frame during preparation and movement. Together, our results implicate F5 as a site of visuo-motor transformation and advocate a strong transition between hand-independent and hand-dependent representations in this parieto-frontal circuit.

Time-resolved decoding of planned delayed and immediate prehension movements

Different contexts require us either to react immediately, or to delay (or suppress) a planned movement. Previous studies that aimed at decoding movement plans typically dissociated movement preparation and execution by means of delayed-movement paradigms. Here we asked whether these results can be generalized to the planning and execution of immediate movements. To directly compare delayed, non-delayed, and suppressed reaching and grasping movements, we used a slow event-related functional magnetic resonance imaging (fMRI) design. To examine how neural representations evolved throughout movement planning, execution, and suppression, we performed time-resolved multivariate pattern analysis (MVPA). During the planning phase, we were able to decode upcoming reaching and grasping movements in contralateral parietal and premotor areas. During the execution phase, we were able to decode movements in a widespread bilateral network of motor, premotor, and somatosensory areas. Moreover, we obtained significant decoding across delayed and non-delayed movement plans in contralateral primary motor cortex. Our results demonstrate the feasibility of time-resolved MVPA and provide new insights into the dynamics of the prehension network, suggesting early neural representations of movement plans in the primary motor cortex that are shared between delayed and non-delayed contexts.

Motor cortical activity changes during neuroprosthetic-controlled object interaction

Brain-computer interface (BCI) controlled prosthetic arms are being developed to restore function to people with upper-limb paralysis. This work provides an opportunity to analyze human cortical activity during complex tasks. Previously we observed that BCI control became more difficult during interactions with objects, although we did not quantify the neural origins of this phenomena. Here, we investigated how motor cortical activity changed in the presence of an object independently of the kinematics that were being generated using intracortical recordings from two people with tetraplegia. After identifying a population-wide increase in neural firing rates that corresponded with the hand being near an object, we developed an online scaling feature in the BCI system that operated without knowledge of the task. Online scaling increased the ability of two subjects to control the robotic arm when reaching to grasp and transport objects. This work suggests that neural representations of the environment, in this case the presence of an object, are strongly and consistently represented in motor cortex but can be accounted for to improve BCI performance.

Disentangling Representations of Object and Grasp Properties in the Human Brain

The properties of objects, such as shape, influence the way we grasp them. To quantify the role of different brain regions during grasping, it is necessary to disentangle the processing of visual dimensions related to object properties from the motor aspects related to the specific hand configuration. We orthogonally varied object properties (shape, size, and elongation) and task (passive viewing, precision grip with two or five digits, or coarse grip with five digits) and used representational similarity analysis of functional magnetic resonance imaging data to infer the representation of object properties and hand configuration in the human brain. We found that object elongation is the most strongly represented object feature during grasping and is coded preferentially in the primary visual cortex as well as the anterior and posterior superior-parieto-occipital cortex. By contrast, primary somatosensory, motor, and ventral premotor cortices coded preferentially the number of digits while ventral-stream and dorsal-stream regions coded a mix of visual and motor dimensions. The representation of object features varied with task modality, as object elongation was less relevant during passive viewing than grasping. To summarize, this study shows that elongation is a particularly relevant property of the object to grasp, which along with the number of digits used, is represented within both ventral-stream and parietal regions, suggesting that communication between the two streams about these specific visual and motor dimensions might be relevant to the execution of efficient grasping actions.

SIGNIFICANCE STATEMENT

To grasp something, the visual properties of an object guide preshaping of the hand into the appropriate configuration. Different grips can be used, and different objects require different hand configurations. However, in natural actions, grip and object type are often confounded, and the few experiments that have attempted to separate them have produced conflicting results. As such, it is unclear how visual and motor properties are represented across brain regions during grasping. Here we orthogonally manipulated object properties and grip, and revealed the visual dimension (object elongation) and the motor dimension (number of digits) that are more strongly coded in ventral and dorsal streams. These results suggest that both streams play a role in the visuomotor coding essential for grasping.

Decoding Information for Grasping from the Macaque Dorsomedial Visual Stream

Neurodecoders have been developed by researchers mostly to control neuroprosthetic devices, but also to shed new light on neural functions. In this study, we show that signals representing grip configurations can be reliably decoded from neural data acquired from area V6A of the monkey medial posterior parietal cortex. Two Macaca fascicularis monkeys were trained to perform an instructed-delay reach-to-grasp task in the dark and in the light toward objects of different shapes. Population neural activity was extracted at various time intervals on vision of the objects, the delay before movement, and grasp execution. This activity was used to train and validate a Bayes classifier used for decoding objects and grip types. Recognition rates were well over chance level for all the epochs analyzed in this study. Furthermore, we detected slightly different decoding accuracies, depending on the task's visual condition. Generalization analysis was performed by training and testing the system during different time intervals. This analysis demonstrated that a change of code occurred during the course of the task. Our classifier was able to discriminate grasp types fairly well in advance with respect to grasping onset. This feature might be important when the timing is critical to send signals to external devices before the movement start. Our results suggest that the neural signals from the dorsomedial visual pathway can be a good substrate to feed neural prostheses for prehensile actions.SIGNIFICANCE STATEMENT Recordings of neural activity from nonhuman primate frontal and parietal cortex have led to the development of methods of decoding movement information to restore coordinated arm actions in paralyzed human beings. Our results show that the signals measured from the monkey medial posterior parietal cortex are valid for correctly decoding information relevant for grasping. Together with previous studies on decoding reach trajectories from the medial posterior parietal cortex, this highlights the medial parietal cortex as a target site for transforming neural activity into control signals to command prostheses to allow human patients to dexterously perform grasping actions.

Neural Data-Driven Musculoskeletal Modeling for Personalized Neurorehabilitation Technologies

OBJECTIVES

The development of neurorehabilitation technologies requires the profound understanding of the mechanisms underlying an individual's motor ability and impairment. A major factor limiting this understanding is the difficulty of bridging between events taking place at the neurophysiologic level (i.e., motor neuron firings) with those emerging at the musculoskeletal level (i.e. joint actuation), in vivo in the intact moving human. This review presents emerging model-based methodologies for filling this gap that are promising for developing clinically viable technologies.

METHODS

We provide a design overview of musculoskeletal modeling formulations driven by recordings of neuromuscular activity with a critical comparison to alternative model-free approaches in the context of neurorehabilitation technologies. We present advanced electromyography-based techniques for interfacing with the human nervous system and model-based techniques for translating the extracted neural information into estimates of motor function.

RESULTS

We introduce representative application areas where modeling is relevant for accessing neuromuscular variables that could not be measured experimentally. We then show how these variables are used for designing personalized rehabilitation interventions, biologically inspired limbs, and human-machine interfaces.

CONCLUSION

The ability of using electrophysiological recordings to inform biomechanical models enables accessing a broader range of neuromechanical variables than analyzing electrophysiological data or movement data individually. This enables understanding the neuromechanical interplay underlying in vivo movement function, pathology, and robot-assisted motor recovery.

SIGNIFICANCE

Filling the gap between our understandings of movement neural and mechanical functions is central for addressing one of the major challenges in neurorehabilitation: personalizing current technologies and interventions to an individual's anatomy and impairment.

Hand Shape Representations in the Human Posterior Parietal Cortex

Humans shape their hands to grasp, manipulate objects, and to communicate. From nonhuman primate studies, we know that visual and motor properties for grasps can be derived from cells in the posterior parietal cortex (PPC). Are non-grasp-related hand shapes in humans represented similarly? Here we show for the first time how single neurons in the PPC of humans are selective for particular imagined hand shapes independent of graspable objects. We find that motor imagery to shape the hand can be successfully decoded from the PPC by implementing a version of the popular Rock-Paper-Scissors game and its extension Rock-Paper-Scissors-Lizard-Spock. By simultaneous presentation of visual and auditory cues, we can discriminate motor imagery from visual information and show differences in auditory and visual information processing in the PPC. These results also demonstrate that neural signals from human PPC can be used to drive a dexterous cortical neuroprosthesis.

SIGNIFICANCE STATEMENT

This study shows for the first time hand-shape decoding from human PPC. Unlike nonhuman primate studies in which the visual stimuli are the objects to be grasped, the visually cued hand shapes that we use are independent of the stimuli. Furthermore, we can show that distinct neuronal populations are activated for the visual cue and the imagined hand shape. Additionally we found that auditory and visual stimuli that cue the same hand shape are processed differently in PPC. Early on in a trial, only the visual stimuli and not the auditory stimuli can be decoded. During the later stages of a trial, the motor imagery for a particular hand shape can be decoded for both modalities.

Decoding Internally and Externally Driven Movement Plans

During movement planning, brain activity within parietofrontal networks encodes information about upcoming actions that can be driven either externally (e.g., by a sensory cue) or internally (i.e., by a choice/decision). Here we used multivariate pattern analysis (MVPA) of fMRI data to distinguish between areas that represent (1) abstract movement plans that generalize across the way in which these were driven, (2) internally driven movement plans, or (3) externally driven movement plans. In a delayed-movement paradigm, human volunteers were asked to plan and execute three types of nonvisually guided right-handed reaching movements toward a central target object: using a precision grip, a power grip, or touching the object without hand preshaping. On separate blocks of trials, movements were either instructed via color cues (Instructed condition), or chosen by the participant (Free-Choice condition). Using ROI-based and whole-brain searchlight-based MVPA, we found abstract representations of planned movements that generalize across the way these movements are selected (internally vs externally driven) in parietal cortex, dorsal premotor cortex, and primary motor cortex contralateral to the acting hand. In addition, we revealed representations specific for internally driven movement plans in contralateral ventral premotor cortex, dorsolateral prefrontal cortex, supramarginal gyrus, and in ipsilateral posterior parietotemporal regions, suggesting that these regions are recruited during movement selection. Finally, we observed representations of externally driven movement plans in bilateral supplementary motor cortex and a similar trend in presupplementary motor cortex, suggesting a role in stimulus-response mapping.

SIGNIFICANCE STATEMENT

The way the human brain prepares the body for action constitutes an essential part of our ability to interact with our environment. Previous studies demonstrated that patterns of neuronal activity encode upcoming movements. Here we used multivariate pattern analysis of human fMRI data to distinguish between brain regions containing movement plans for instructed (externally driven) movements, areas involved in movement selection (internally driven), and areas containing abstract movement plans that are invariant to the way these were generated (i.e., that generalize across externally and internally driven movement plans). Our findings extend our understanding of the neural basis of movement planning and have the potential to contribute to the development of brain-controlled neural prosthetic devices.

Spatial Representations in Local Field Potential Activity of Primate Anterior Intraparietal Cortex (AIP)

The execution of reach-to-grasp movements in order to interact with our environment is an important subset of the human movement repertoire. To coordinate such goal-directed movements, information about the relative spatial position of target and effector (in this case the hand) has to be continuously integrated and processed. Recently, we reported the existence of spatial representations in spiking-activity of the cortical fronto-parietal grasp network (Lehmann & Scherberger 2013), and in particular in the anterior intraparietal cortex (AIP). To further investigate the nature of these spatial representations, we explored in two rhesus monkeys (Macaca mulatta) how different frequency bands of the local field potential (LFP) in AIP are modulated by grip type, target position, and gaze position, during the planning and execution of reach-to-grasp movements. We systematically varied grasp type, spatial target, and gaze position and found that both spatial and grasp information were encoded in a variety of frequency bands (1–13Hz, 13–30Hz, 30–60Hz, and 60–100Hz, respectively). Whereas the representation of grasp type strongly increased towards and during movement execution, spatial information was represented throughout the task. Both spatial and grasp type representations could be readily decoded from all frequency bands. The fact that grasp type and spatial (reach) information was found not only in spiking activity, but also in various LFP frequency bands of AIP, might significantly contribute to the development of LFP-based neural interfaces for the control of upper limb prostheses.

Predicting Reaction Time from the Neural State Space of the Premotor and Parietal Grasping Network

Neural networks of the brain involved in the planning and execution of grasping movements are not fully understood. The network formed by macaque anterior intraparietal area (AIP) and hand area (F5) of the ventral premotor cortex is implicated strongly in the generation of grasping movements. However, the differential role of each area in this frontoparietal network is unclear. We recorded spiking activity from many electrodes in parallel in AIP and F5 while three macaque monkeys (Macaca mulatta) performed a delayed grasping task. By analyzing neural population activity during action preparation, we found that state space analysis of simultaneously recorded units is significantly more predictive of subsequent reaction times (RTs) than traditional methods. Furthermore, because we observed a wide variety of individual unit characteristics, we developed the sign-corrected average rate (SCAR) method of neural population averaging. The SCAR method was able to explain at least as much variance in RT overall as state space methods. Overall, F5 activity predicted RT (18% variance explained) significantly better than AIP (6%). The SCAR methods provides a straightforward interpretation of population activity, although other state space methods could provide richer descriptions of population dynamics. Together, these results lend support to the differential role of the parietal and frontal cortices in preparation for grasping, suggesting that variability in preparatory activity in F5 has a more potent effect on trial-to-trial RT variability than AIP.

SIGNIFICANCE STATEMENT

Grasping movements are planned before they are executed, but how is the preparatory activity in a population of neurons related to the subsequent reaction time (RT)? A population analysis of the activity of many neurons recorded in parallel in macaque premotor (F5) and parietal (AIP) cortices during a delayed grasping task revealed that preparatory activity in F5 could explain a threefold larger fraction of variability in trial-to-trial RT than AIP. These striking differences lend additional support to a differential role of the parietal and premotor cortices in grasp movement preparation, suggesting that F5 has a more direct influence on trial-to-trial variability and movement timing, whereas AIP might be more closely linked to overall movement intentions.

Linking Objects to Actions: Encoding of Target Object and Grasping Strategy in Primate Ventral Premotor Cortex

Neural activity in ventral premotor cortex (PMv) has been associated with the process of matching perceived objects with the motor commands needed to grasp them. It remains unclear how PMv networks can flexibly link percepts of objects affording multiple grasp options into a final desired hand action. Here, we use a relational encoding approach to track the functional state of PMv neuronal ensembles in macaque monkeys through the process of passive viewing, grip planning, and grasping movement execution. We used objects affording multiple possible grip strategies. The task included separate instructed delay periods for object presentation and grip instruction. This approach allowed us to distinguish responses elicited by the visual presentation of the objects from those associated with selecting a given motor plan for grasping. We show that PMv continuously incorporates information related to object shape and grip strategy as it becomes available, revealing a transition from a set of ensemble states initially most closely related to objects, to a new set of ensemble patterns reflecting unique object-grip combinations. These results suggest that PMv dynamically combines percepts, gradually navigating toward activity patterns associated with specific volitional actions, rather than directly mapping perceptual object properties onto categorical grip representations. Our results support the idea that PMv is part of a network that dynamically computes motor plans from perceptual information.

SIGNIFICANCE STATEMENT

The present work demonstrates that the activity of groups of neurons in primate ventral premotor cortex reflects information related to visually presented objects, as well as the motor strategy used to grasp them, linking individual objects to multiple possible grips. PMv could provide useful control signals for neuroprosthetic assistive devices designed to interact with objects in a flexible way.

Representation of continuous hand and arm movements in macaque areas M1, F5, and AIP: a comparative decoding study

OBJECTIVE

In the last decade, multiple brain areas have been investigated with respect to their decoding capability of continuous arm or hand movements. So far, these studies have mainly focused on motor or premotor areas like M1 and F5. However, there is accumulating evidence that anterior intraparietal area (AIP) in the parietal cortex also contains information about continuous movement.

APPROACH

In this study, we decoded 27 degrees of freedom representing complete hand and arm kinematics during a delayed grasping task from simultaneously recorded activity in areas M1, F5, and AIP of two macaque monkeys (Macaca mulatta).

MAIN RESULTS

We found that all three areas provided decoding performances that lay significantly above chance. In particular, M1 yielded highest decoding accuracy followed by F5 and AIP. Furthermore, we provide support for the notion that AIP does not only code categorical visual features of objects to be grasped, but also contains a substantial amount of temporal kinematic information.

SIGNIFICANCE

This fact could be utilized in future developments of neural interfaces restoring hand and arm movements.

Decoding bipedal locomotion from the rat sensorimotor cortex

OBJECTIVE

Decoding forelimb movements from the firing activity of cortical neurons has been interfaced with robotic and prosthetic systems to replace lost upper limb functions in humans. Despite the potential of this approach to improve locomotion and facilitate gait rehabilitation, decoding lower limb movement from the motor cortex has received comparatively little attention. Here, we performed experiments to identify the type and amount of information that can be decoded from neuronal ensemble activity in the hindlimb area of the rat motor cortex during bipedal locomotor tasks.

APPROACH

Rats were trained to stand, step on a treadmill, walk overground and climb staircases in a bipedal posture. To impose this gait, the rats were secured in a robotic interface that provided support against the direction of gravity and in the mediolateral direction, but behaved transparently in the forward direction. After completion of training, rats were chronically implanted with a micro-wire array spanning the left hindlimb motor cortex to record single and multi-unit activity, and bipolar electrodes into 10 muscles of the right hindlimb to monitor electromyographic signals. Whole-body kinematics, muscle activity, and neural signals were simultaneously recorded during execution of the trained tasks over multiple days of testing. Hindlimb kinematics, muscle activity, gait phases, and locomotor tasks were decoded using offline classification algorithms.

MAIN RESULTS

We found that the stance and swing phases of gait and the locomotor tasks were detected with accuracies as robust as 90% in all rats. Decoded hindlimb kinematics and muscle activity exhibited a larger variability across rats and tasks.

SIGNIFICANCE

Our study shows that the rodent motor cortex contains useful information for lower limb neuroprosthetic development. However, brain-machine interfaces estimating gait phases or locomotor behaviors, instead of continuous variables such as limb joint positions or speeds, are likely to provide more robust control strategies for the design of such neuroprostheses.

Local field potentials in primate motor cortex encode grasp kinetic parameters

Reach and grasp kinematics are known to be encoded in the spiking activity of neuronal ensembles and in local field potentials (LFPs) recorded from primate motor cortex during movement planning and execution. However, little is known, especially in LFPs, about the encoding of kinetic parameters, such as forces exerted on the object during the same actions. We implanted two monkeys with microelectrode arrays in the motor cortical areas MI and PMd to investigate encoding of grasp-related parameters in motor cortical LFPs during planning and execution of reach-and-grasp movements. We identified three components of the LFP that modulated during grasps corresponding to low (0.3-7Hz), intermediate (~10-~40Hz) and high (~80-250Hz) frequency bands. We show that all three components can be used to classify not only grip types but also object loads during planning and execution of a grasping movement. In addition, we demonstrate that all three components recorded during planning or execution can be used to continuously decode finger pressure forces and hand position related to the grasping movement. Low and high frequency components provide similar classification and decoding accuracies, which were substantially higher than those obtained from the intermediate frequency component. Our results demonstrate that intended reach and grasp kinetic parameters are encoded in multiple LFP bands during both movement planning and execution. These findings also suggest that the LFP is a reliable signal for the control of parameters related to object load and applied pressure forces in brain-machine interfaces.

Global cortical activity predicts shape of hand during grasping

Recent studies show that the amplitude of cortical field potentials is modulated in the time domain by grasping kinematics. However, it is unknown if these low frequency modulations persist and contain enough information to decode grasp kinematics in macro-scale activity measured at the scalp via electroencephalography (EEG). Further, it is unclear as to whether joint angle velocities or movement synergies are the optimal kinematics spaces to decode. In this offline decoding study, we infer from human EEG, hand joint angular velocities as well as synergistic trajectories as subjects perform natural reach-to-grasp movements. Decoding accuracy, measured as the correlation coefficient (r) between the predicted and actual movement kinematics, was r = 0.49 ± 0.02 across 15 hand joints. Across the first three kinematic synergies, decoding accuracies were r = 0.59 ± 0.04, 0.47 ± 0.06, and 0.32 ± 0.05. The spatial-temporal pattern of EEG channel recruitment showed early involvement of contralateral frontal-central scalp areas followed by later activation of central electrodes over primary sensorimotor cortical areas. Information content in EEG about the grasp type peaked at 250 ms after movement onset. The high decoding accuracies in this study are significant not only as evidence for time-domain modulation in macro-scale brain activity, but for the field of brain-machine interfaces as well. Our decoding strategy, which harnesses the neural “symphony” as opposed to local members of the neural ensemble (as in intracranial approaches), may provide a means of extracting information about motor intent for grasping without the need for penetrating electrodes and suggests that it may be soon possible to develop non-invasive neural interfaces for the control of prosthetic limbs.

Decoding a wide range of hand configurations from macaque motor, premotor, and parietal cortices

Despite recent advances in decoding cortical activity for motor control, the development of hand prosthetics remains a major challenge. To reduce the complexity of such applications, higher cortical areas that also represent motor plans rather than just the individual movements might be advantageous. We investigated the decoding of many grip types using spiking activity from the anterior intraparietal (AIP), ventral premotor (F5), and primary motor (M1) cortices. Two rhesus monkeys were trained to grasp 50 objects in a delayed task while hand kinematics and spiking activity from six implanted electrode arrays (total of 192 electrodes) were recorded. Offline, we determined 20 grip types from the kinematic data and decoded these hand configurations and the grasped objects with a simple Bayesian classifier. When decoding from AIP, F5, and M1 combined, the mean accuracy was 50% (using planning activity) and 62% (during motor execution) for predicting the 50 objects (chance level, 2%) and substantially larger when predicting the 20 grip types (planning, 74%; execution, 86%; chance level, 5%). When decoding from individual arrays, objects and grip types could be predicted well during movement planning from AIP (medial array) and F5 (lateral array), whereas M1 predictions were poor. In contrast, predictions during movement execution were best from M1, whereas F5 performed only slightly worse. These results demonstrate for the first time that a large number of grip types can be decoded from higher cortical areas during movement preparation and execution, which could be relevant for future neuroprosthetic devices that decode motor plans.

Functional Properties of Parietal Hand Manipulation–related Neurons and Mirror Neurons Responding to Vision of Own Hand Action

Parietofrontal pathways play an important role in visually guided motor control. In this pathway, hand manipulation-related neurons in the inferior parietal lobule represent 3-D properties of an object and motor patterns to grasp it. Furthermore, mirror neurons show visual responses that are concerned with the actions of others and motor-related activity during execution of the same grasping action. Because both of these categories of neurons integrate visual and motor signals, these neurons may play a role in motor control based on visual feedback signals. The aim of this study was to investigate whether these neurons in inferior parietal lobule including the anterior intraparietal area and PFG of macaques represent visual images of the monkey's own hand during a self-generated grasping action. We recorded 235 neurons related to hand manipulation tasks. Of these, 54 responded to video clips of the monkey's own hand action, the same as visual feedback during that action or clips of the experimenter's hand action in a lateral view. Of these 54 neurons, 25 responded to video clips of the monkey's own hand, even without an image of the target object. We designated these 25 neurons as “hand-type.” Thirty-three of 54 neurons that were defined as mirror neurons showed visual responses to the experimenter's action and motor responses. Thirteen of these mirror neurons were classified as hand-type. These results suggest that activity of hand manipulation-related and mirror neurons in anterior intraparietal/PFG plays a fundamental role in monitoring one's own body state based on visual feedback.

Distinct neural patterns enable grasp types decoding in monkey dorsal premotor cortex

OBJECTIVE

Recent studies have shown that dorsal premotor cortex (PMd), a cortical area in the dorsomedial grasp pathway, is involved in grasp movements. However, the neural ensemble firing property of PMd during grasp movements and the extent to which it can be used for grasp decoding are still unclear.

APPROACH

To address these issues, we used multielectrode arrays to record both spike and local field potential (LFP) signals in PMd in macaque monkeys performing reaching and grasping of one of four differently shaped objects.

MAIN RESULTS

Single and population neuronal activity showed distinct patterns during execution of different grip types. Cluster analysis of neural ensemble signals indicated that the grasp related patterns emerged soon (200-300 ms) after the go cue signal, and faded away during the hold period. The timing and duration of the patterns varied depending on the behaviors of individual monkey. Application of support vector machine model to stable activity patterns revealed classification accuracies of 94% and 89% for each of the two monkeys, indicating a robust, decodable grasp pattern encoded in the PMd. Grasp decoding using LFPs, especially the high-frequency bands, also produced high decoding accuracies.

SIGNIFICANCE

This study is the first to specify the neuronal population encoding of grasp during the time course of grasp. We demonstrate high grasp decoding performance in PMd. These findings, combined with previous evidence for reach related modulation studies, suggest that PMd may play an important role in generation and maintenance of grasp action and may be a suitable locus for brain-machine interface applications.

Neural correlates of grasping

Prehension, the capacity to reach and grasp objects, comprises two main components: reaching, i.e., moving the hand towards an object, and grasping, i.e., shaping the hand with respect to its properties. Knowledge of this topic has gained a huge advance in recent years, dramatically changing our view on how prehension is represented within the dorsal stream. While our understanding of the various nodes coding the grasp component is rapidly progressing, little is known of the integration between grasping and reaching. With this Mini Review we aim to provide an up-to-date overview of the recent developments on the coding of prehension. We will start with a description of the regions coding various aspects of grasping in humans and monkeys, delineating where it might be integrated with reaching. To gain insights into the causal role of these nodes in the coding of prehension, we will link this functional description to lesion studies. Finally, we will discuss future directions that might be promising to unveil new insights on the coding of prehension movements.

Extracting kinetic information from human motor cortical signals

Brain machine interfaces (BMIs) have the potential to provide intuitive control of neuroprostheses to restore grasp to patients with paralyzed or amputated upper limbs. For these neuroprostheses to function, the ability to accurately control grasp force is critical. Grasp force can be decoded from neuronal spikes in monkeys, and hand kinematics can be decoded using electrocorticogram (ECoG) signals recorded from the surface of the human motor cortex. We hypothesized that kinetic information about grasping could also be extracted from ECoG, and sought to decode continuously-graded grasp force. In this study, we decoded isometric pinch force with high accuracy from ECoG in 10 human subjects. The predicted signals explained from 22% to 88% (60 ± 6%, mean ± SE) of the variance in the actual force generated. We also decoded muscle activity in the finger flexors, with similar accuracy to force decoding. We found that high gamma band and time domain features of the ECoG signal were most informative about kinetics, similar to our previous findings with intracortical LFPs. In addition, we found that peak cortical representations of force applied by the index and little fingers were separated by only about 4mm. Thus, ECoG can be used to decode not only kinematics, but also kinetics of movement. This is an important step toward restoring intuitively-controlled grasp to impaired patients.

Simultaneous real-time monitoring of multiple cortical systems

OBJECTIVE

Real-time monitoring of the brain is potentially valuable for performance monitoring, communication, training or rehabilitation. In natural situations, the brain performs a complex mix of various sensory, motor or cognitive functions. Thus, real-time brain monitoring would be most valuable if (a) it could decode information from multiple brain systems simultaneously, and (b) this decoding of each brain system were robust to variations in the activity of other (unrelated) brain systems. Previous studies showed that it is possible to decode some information from different brain systems in retrospect and/or in isolation. In our study, we set out to determine whether it is possible to simultaneously decode important information about a user from different brain systems in real time, and to evaluate the impact of concurrent activity in different brain systems on decoding performance.

APPROACH

We study these questions using electrocorticographic signals recorded in humans. We first document procedures for generating stable decoding models given little training data, and then report their use for offline and for real-time decoding from 12 subjects (six for offline parameter optimization, six for online experimentation). The subjects engage in tasks that involve movement intention, movement execution and auditory functions, separately, and then simultaneously. Main Results: Our real-time results demonstrate that our system can identify intention and movement periods in single trials with an accuracy of 80.4% and 86.8%, respectively (where 50% would be expected by chance). Simultaneously, the decoding of the power envelope of an auditory stimulus resulted in an average correlation coefficient of 0.37 between the actual and decoded power envelopes. These decoders were trained separately and executed simultaneously in real time.

SIGNIFICANCE

This study yielded the first demonstration that it is possible to decode simultaneously the functional activity of multiple independent brain systems. Our comparison of univariate and multivariate decoding strategies, and our analysis of the influence of their decoding parameters, provides benchmarks and guidelines for future research on this topic.

Motor cortical correlates of arm resting in the context of a reaching task and implications for prosthetic control

Prosthetic devices are being developed to restore movement for motor-impaired individuals. A robotic arm can be controlled based on models that relate motor-cortical ensemble activity to kinematic parameters. The models are typically built and validated on data from structured trial periods during which a subject actively performs specific movements, but real-world prosthetic devices will need to operate correctly during rest periods as well. To develop a model of motor cortical modulation during rest, we trained monkeys (Macaca mulatta) to perform a reaching task with their own arm while recording motor-cortical single-unit activity. When a monkey spontaneously put its arm down to rest between trials, our traditional movement decoder produced a nonzero velocity prediction, which would cause undesired motion when applied to a prosthetic arm. During these rest periods, a marked shift was found in individual units' tuning functions. The activity pattern of the whole population during rest (Idle state) was highly distinct from that during reaching movements (Active state), allowing us to predict arm resting from instantaneous firing rates with 98% accuracy using a simple classifier. By cascading this state classifier and the movement decoder, we were able to predict zero velocity correctly, which would avoid undesired motion in a prosthetic application. Interestingly, firing rates during hold periods followed the Active pattern even though hold kinematics were similar to those during rest with near-zero velocity. These findings expand our concept of motor-cortical function by showing that population activity reflects behavioral context in addition to the direct parameters of the movement itself.

Restoring sensorimotor function through intracortical interfaces: progress and looming challenges

The loss of a limb or paralysis resulting from spinal cord injury has devastating consequences on quality of life. One approach to restoring lost sensory and motor abilities in amputees and patients with tetraplegia is to supply them with implants that provide a direct interface with the CNS. Such brain-machine interfaces might enable a patient to exert voluntary control over a prosthetic or robotic limb or over the electrically induced contractions of paralysed muscles. A parallel interface could convey sensory information about the consequences of these movements back to the patient. Recent developments in the algorithms that decode motor intention from neuronal activity and in approaches to convey sensory feedback by electrically stimulating neurons, using biomimetic and adaptation-based approaches, have shown the promise of invasive interfaces with sensorimotor cortices, although substantial challenges remain.

Decoding repetitive finger movements with brain activity acquired via non-invasive electroencephalography

We investigated how well repetitive finger tapping movements can be decoded from scalp electroencephalography (EEG) signals. A linear decoder with memory was used to infer continuous index finger angular velocities from the low-pass filtered fluctuations of the amplitude of a plurality of EEG signals distributed across the scalp. To evaluate the accuracy of the decoder, the Pearson's correlation coefficient (r) between the observed and predicted trajectories was calculated in a 10-fold cross-validation scheme. We also assessed attempts to decode finger kinematics from EEG data that was cleaned with independent component analysis (ICA), EEG data from peripheral sensors, and EEG data from rest periods. A genetic algorithm (GA) was used to select combinations of EEG channels that maximized decoding accuracies. Our results (lower quartile r = 0.18, median r = 0.36, upper quartile r = 0.50) show that delta-band EEG signals contain useful information that can be used to infer finger kinematics. Further, the highest decoding accuracies were characterized by highly correlated delta band EEG activity mostly localized to the contralateral central areas of the scalp. Spectral analysis of EEG also showed bilateral alpha band (8–13 Hz) event related desynchronizations (ERDs) and contralateral beta band (20–30 Hz) event related synchronizations (ERSs) localized over central scalp areas. Overall, this study demonstrates the feasibility of decoding finger kinematics from scalp EEG signals.

Decoding grasp types with high frequency of local field potentials from primate primary dorsal premotor cortex

Recently, local field potentials (LFPs) have been successfully used to extract information of arm and hand movement in some brain-machine interfaces (BMIs) studies, which suggested that LFPs would improve the performance of BMI applications because of its long-term stability. However, the performance of LFPs in different frequency bands has not been investigated in decoding hand grasp types. Here, the LFPs from the monkey's dorsal premotor cortices were collected by microelectrode array when monkey was performing grip-specific grasp task. A K-nearest neighbor classifier performed on the power spectrum of LFPs was used to decode grasping movements. The decoding powers of LFPs in different frequency bands, channels and trials used for training were also studied. The results show that the broad high frequency band (200-400Hz) LFPs achieved the best performance with classification accuracy reaching over 0.9. It infers that high frequency band LFPs in PMd cortex could be a promising source of control signals in developing functional BMIs for hand grasping.

Optic ataxia as a model to investigate the role of the posterior parietal cortex in visually guided action: evidence from studies of patient M.H

Optic ataxia is a neuropsychological disorder that affects the ability to interact with objects presented in the visual modality following either unilateral or bilateral lesions of the posterior parietal cortex (PPC). Patients with optic ataxia fail to reach accurately for objects, particularly when they are presented in peripheral vision. The present review will focus on a series of experiments performed on patient M.H. Following a lesion restricted largely to the left PPC, he developed mis-reaching behavior when using his contralesional right arm for movements directed toward the contralesional (right) visual half-field. Given the clear-cut specificity of this patient's deficit, whereby reaching actions are essentially spared when executed toward his ipsilateral space or when using his left arm, M.H. provides a valuable "experiment of nature" for investigating the role of the PPC in performing different visually guided actions. In order to address this, we used kinematic measurement techniques to investigate M.H.'s reaching and grasping behavior in various tasks. Our experiments support the idea that optic ataxia is highly function-specific: it affects a specific sub-category of visually guided actions (reaching but not grasping), regardless of their specific end goal (both reaching toward an object and reaching to avoid an obstacle); and finally, is independent of the limb used to perform the action (whether the arm or the leg). Critically, these results are congruent with recent functional MRI experiments in neurologically intact subjects which suggest that the PPC is organized in a function-specific, rather than effector-specific, manner with different sub-portions of its mantle devoted to guiding actions according to their specific end-goal (reaching, grasping, or looking), rather than according to the effector used to perform them (leg, arm, hand, or eyes).