76
|
Kaufman MT, Churchland MM, Ryu SI, Shenoy KV. Vacillation, indecision and hesitation in moment-by-moment decoding of monkey motor cortex. eLife 2015; 4:e04677. [PMID: 25942352 PMCID: PMC4415122 DOI: 10.7554/elife.04677] [Citation(s) in RCA: 65] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2014] [Accepted: 04/03/2015] [Indexed: 11/13/2022] Open
Abstract
When choosing actions, we can act decisively, vacillate, or suffer momentary indecision. Studying how individual decisions unfold requires moment-by-moment readouts of brain state. Here we provide such a view from dorsal premotor and primary motor cortex. Two monkeys performed a novel decision task while we recorded from many neurons simultaneously. We found that a decoder trained using 'forced choices' (one target viable) was highly reliable when applied to 'free choices'. However, during free choices internal events formed three categories. Typically, neural activity was consistent with rapid, unwavering choices. Sometimes, though, we observed presumed 'changes of mind': the neural state initially reflected one choice before changing to reflect the final choice. Finally, we observed momentary 'indecision': delay forming any clear motor plan. Further, moments of neural indecision accompanied moments of behavioral indecision. Together, these results reveal the rich and diverse set of internal events long suspected to occur during free choice.
Collapse
|
77
|
Christie BP, Tat DM, Irwin ZT, Gilja V, Nuyujukian P, Foster JD, Ryu SI, Shenoy KV, Thompson DE, Chestek CA. Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. J Neural Eng 2014; 12:016009. [PMID: 25504690 DOI: 10.1088/1741-2560/12/1/016009] [Citation(s) in RCA: 41] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
OBJECTIVE For intracortical brain-machine interfaces (BMIs), action potential voltage waveforms are often sorted to separate out individual neurons. If these neurons contain independent tuning information, this process could increase BMI performance. However, the sorting of action potentials ('spikes') requires high sampling rates and is computationally expensive. To explicitly define the difference between spike sorting and alternative methods, we quantified BMI decoder performance when using threshold-crossing events versus sorted action potentials. APPROACH We used data sets from 58 experimental sessions from two rhesus macaques implanted with Utah arrays. Data were recorded while the animals performed a center-out reaching task with seven different angles. For spike sorting, neural signals were sorted into individual units by using a mixture of Gaussians to cluster the first four principal components of the waveforms. For thresholding events, spikes that simply crossed a set threshold were retained. We decoded the data offline using both a Naïve Bayes classifier for reaching direction and a linear regression to evaluate hand position. MAIN RESULTS We found the highest performance for thresholding when placing a threshold between -3 and -4.5 × Vrms. Spike sorted data outperformed thresholded data for one animal but not the other. The mean Naïve Bayes classification accuracy for sorted data was 88.5% and changed by 5% on average when data were thresholded. The mean correlation coefficient for sorted data was 0.92, and changed by 0.015 on average when thresholded. SIGNIFICANCE For prosthetics applications, these results imply that when thresholding is used instead of spike sorting, only a small amount of performance may be lost. The utilization of threshold-crossing events may significantly extend the lifetime of a device because these events are often still detectable once single neurons are no longer isolated.
Collapse
|
78
|
Abstract
Brain-machine interfaces (BMIs) aim to help people with paralysis by decoding movement-related neural signals into control signals for guiding computer cursors, prosthetic arms, and other assistive devices. Despite compelling laboratory experiments and ongoing FDA pilot clinical trials, system performance, robustness, and generalization remain challenges. We provide a perspective on how two complementary lines of investigation, that have focused on decoder design and neural adaptation largely separately, could be brought together to advance BMIs. This BMI paradigm should also yield new scientific insights into the function and dysfunction of the nervous system.
Collapse
|
79
|
Nuyujukian P, Kao JC, Fan JM, Stavisky SD, Ryu SI, Shenoy KV. Performance sustaining intracortical neural prostheses. J Neural Eng 2014; 11:066003. [PMID: 25307561 DOI: 10.1088/1741-2560/11/6/066003] [Citation(s) in RCA: 50] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Neural prostheses, or brain-machine interfaces, aim to restore efficient communication and movement ability to those suffering from paralysis. A major challenge these systems face is robust performance, particularly with aging signal sources. The aim in this study was to develop a neural prosthesis that could sustain high performance in spite of signal instability while still minimizing retraining time. APPROACH We trained two rhesus macaques implanted with intracortical microelectrode arrays 1-4 years prior to this study to acquire targets with a neurally-controlled cursor. We measured their performance via achieved bitrate (bits per second, bps). This task was repeated over contiguous days to evaluate the sustained performance across time. MAIN RESULTS We found that in the monkey with a younger (i.e., two year old) implant and better signal quality, a fixed decoder could sustain performance for a month at a rate of 4 bps, the highest achieved communication rate reported to date. This fixed decoder was evaluated across 22 months and experienced a performance decline at a rate of 0.24 bps yr(-1). In the monkey with the older (i.e., 3.5 year old) implant and poorer signal quality, a fixed decoder could not sustain performance for more than a few days. Nevertheless, performance in this monkey was maintained for two weeks without requiring additional online retraining time by utilizing prior days' experimental data. Upon analysis of the changes in channel tuning, we found that this stability appeared partially attributable to the cancelling-out of neural tuning fluctuations when projected to two-dimensional cursor movements. SIGNIFICANCE The findings in this study (1) document the highest-performing communication neural prosthesis in monkeys, (2) confirm and extend prior reports of the stability of fixed decoders, and (3) demonstrate a protocol for system stability under conditions where fixed decoders would otherwise fail. These improvements to decoder stability are important for minimizing training time and should make neural prostheses more practical to use.
Collapse
|
80
|
Fan JM, Nuyujukian P, Kao JC, Chestek CA, Ryu SI, Shenoy KV. Intention estimation in brain-machine interfaces. J Neural Eng 2014; 11:016004. [PMID: 24654266 DOI: 10.1088/1741-2560/11/1/016004] [Citation(s) in RCA: 52] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
OBJECTIVE The objective of this work was to quantitatively investigate the mechanisms underlying the performance gains of the recently reported 'recalibrated feedback intention-trained Kalman Filter' (ReFIT-KF). APPROACH This was accomplished by designing variants of the ReFIT-KF algorithm and evaluating training and online data to understand the neural basis of this improvement. We focused on assessing the contribution of two training set innovations of the ReFIT-KF algorithm: intention estimation and the two-stage training paradigm. MAIN RESULTS Within the two-stage training paradigm, we found that intention estimation independently increased target acquisition rates by 37% and 59%, respectively, across two monkeys implanted with multiunit intracortical arrays. Intention estimation improved performance by enhancing the tuning properties and the mutual information between the kinematic and neural training data. Furthermore, intention estimation led to fewer shifts in channel tuning between the training set and online control, suggesting that less adaptation was required during online control. Retraining the decoder with online BMI training data also reduced shifts in tuning, suggesting a benefit of training a decoder in the same behavioral context; however, retraining also led to slower online decode velocities. Finally, we demonstrated that one- and two-stage training paradigms performed comparably when intention estimation is applied. SIGNIFICANCE These findings highlight the utility of intention estimation in reducing the need for adaptive strategies and improving the online performance of BMIs, helping to guide future BMI design decisions.
Collapse
|
81
|
Nuyujukian P, Fan JM, Kao JC, Ryu SI, Shenoy KV. A high-performance keyboard neural prosthesis enabled by task optimization. IEEE Trans Biomed Eng 2014; 62:21-29. [PMID: 25203982 DOI: 10.1109/tbme.2014.2354697] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Communication neural prostheses are an emerging class of medical devices that aim to restore efficient communication to people suffering from paralysis. These systems rely on an interface with the user, either via the use of a continuously moving cursor (e.g., mouse) or the discrete selection of symbols (e.g., keyboard). In developing these interfaces, many design choices have a significant impact on the performance of the system. The objective of this study was to explore the design choices of a continuously moving cursor neural prosthesis and optimize the interface to maximize information theoretic performance. We swept interface parameters of two keyboard-like tasks to find task and subject-specific optimal parameters as measured by achieved bitrate using two rhesus macaques implanted with multielectrode arrays. In this paper, we present the highest performing free-paced neural prosthesis under any recording modality with sustainable communication rates of up to 3.5 bits/s. These findings demonstrate that meaningful high performance can be achieved using an intracortical neural prosthesis, and that, when optimized, these systems may be appropriate for use as communication devices for those with physical disabilities.
Collapse
|
82
|
Foster JD, Nuyujukian P, Freifeld O, Gao H, Walker R, I Ryu S, H Meng T, Murmann B, J Black M, Shenoy KV. A freely-moving monkey treadmill model. J Neural Eng 2014; 11:046020. [PMID: 24995476 DOI: 10.1088/1741-2560/11/4/046020] [Citation(s) in RCA: 47] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Motor neuroscience and brain-machine interface (BMI) design is based on examining how the brain controls voluntary movement, typically by recording neural activity and behavior from animal models. Recording technologies used with these animal models have traditionally limited the range of behaviors that can be studied, and thus the generality of science and engineering research. We aim to design a freely-moving animal model using neural and behavioral recording technologies that do not constrain movement. APPROACH We have established a freely-moving rhesus monkey model employing technology that transmits neural activity from an intracortical array using a head-mounted device and records behavior through computer vision using markerless motion capture. We demonstrate the flexibility and utility of this new monkey model, including the first recordings from motor cortex while rhesus monkeys walk quadrupedally on a treadmill. MAIN RESULTS Using this monkey model, we show that multi-unit threshold-crossing neural activity encodes the phase of walking and that the average firing rate of the threshold crossings covaries with the speed of individual steps. On a population level, we find that neural state-space trajectories of walking at different speeds have similar rotational dynamics in some dimensions that evolve at the step rate of walking, yet robustly separate by speed in other state-space dimensions. SIGNIFICANCE Freely-moving animal models may allow neuroscientists to examine a wider range of behaviors and can provide a flexible experimental paradigm for examining the neural mechanisms that underlie movement generation across behaviors and environments. For BMIs, freely-moving animal models have the potential to aid prosthetic design by examining how neural encoding changes with posture, environment and other real-world context changes. Understanding this new realm of behavior in more naturalistic settings is essential for overall progress of basic motor neuroscience and for the successful translation of BMIs to people with paralysis.
Collapse
|
83
|
Kerr CC, O'Shea DJ, Goo W, Dura-Bernal S, Francis JT, Diester I, Kalanithi P, Deisseroth K, Shenoy KV, Lytton WW. Network-level effects of optogenetic stimulation in a computer model of macaque primary motor cortex. BMC Neurosci 2014. [PMCID: PMC4124983 DOI: 10.1186/1471-2202-15-s1-p107] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
|
84
|
Ames KC, Ryu SI, Shenoy KV. Neural dynamics of reaching following incorrect or absent motor preparation. Neuron 2014; 81:438-51. [PMID: 24462104 DOI: 10.1016/j.neuron.2013.11.003] [Citation(s) in RCA: 98] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/29/2013] [Indexed: 12/13/2022]
Abstract
Moving is thought to take separate preparation and execution steps. During preparation, neural activity in primary motor and dorsal premotor cortices achieves a state specific to an upcoming action but movements are not performed until the execution phase. We investigated whether this preparatory state (more precisely, prepare-and-hold state) is required for movement execution using two complementary experiments. We compared monkeys' neural activity during delayed and nondelayed reaches and in a delayed reaching task in which the target switched locations on a small percentage of trials. Neural population activity bypassed the prepare-and-hold state both in the absence of a delay and if the wrong reach was prepared. However, the initial neural response to the target was similar across behavioral conditions. This suggests that the prepare-and-hold state can be bypassed if needed, but there is a short-latency preparatory step that is performed prior to movement even without a delay.
Collapse
|
85
|
Bishop W, Chestek CC, Gilja V, Nuyujukian P, Foster JD, Ryu SI, Shenoy KV, Yu BM. Self-recalibrating classifiers for intracortical brain-computer interfaces. J Neural Eng 2014; 11:026001. [PMID: 24503597 DOI: 10.1088/1741-2560/11/2/026001] [Citation(s) in RCA: 37] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Intracortical brain-computer interface (BCI) decoders are typically retrained daily to maintain stable performance. Self-recalibrating decoders aim to remove the burden this may present in the clinic by training themselves autonomously during normal use but have only been developed for continuous control. Here we address the problem for discrete decoding (classifiers). APPROACH We recorded threshold crossings from 96-electrode arrays implanted in the motor cortex of two rhesus macaques performing center-out reaches in 7 directions over 41 and 36 separate days spanning 48 and 58 days in total for offline analysis. MAIN RESULTS We show that for the purposes of developing a self-recalibrating classifier, tuning parameters can be considered as fixed within days and that parameters on the same electrode move up and down together between days. Further, drift is constrained across time, which is reflected in the performance of a standard classifier which does not progressively worsen if it is not retrained daily, though overall performance is reduced by more than 10% compared to a daily retrained classifier. Two novel self-recalibrating classifiers produce a ~15% increase in classification accuracy over that achieved by the non-retrained classifier to nearly recover the performance of the daily retrained classifier. SIGNIFICANCE We believe that the development of classifiers that require no daily retraining will accelerate the clinical translation of BCI systems. Future work should test these results in a closed-loop setting.
Collapse
|
86
|
Kaufman MT, Churchland MM, Ryu SI, Shenoy KV. Cortical activity in the null space: permitting preparation without movement. Nat Neurosci 2014; 17:440-8. [PMID: 24487233 PMCID: PMC3955357 DOI: 10.1038/nn.3643] [Citation(s) in RCA: 390] [Impact Index Per Article: 39.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2013] [Accepted: 01/06/2014] [Indexed: 12/19/2022]
Abstract
Neural circuits must perform computations and then selectively output the results to other circuits. Yet synapses do not change radically at millisecond timescales. A key question then is: how is communication between neural circuits controlled? In motor control, brain areas directly involved in driving movement are active well before movement begins. Muscle activity is some readout of neural activity, yet remains largely unchanged during preparation. Here we find that during preparation, while the monkey holds still, changes in motor cortical activity cancel out at the level of these population readouts. Motor cortex can thereby prepare the movement without prematurely causing it. Further, we found evidence that this mechanism also operates in dorsal premotor cortex (PMd), largely accounting for how preparatory activity is attenuated in primary motor cortex (M1). Selective use of “output-null” vs. “output-potent” patterns of activity may thus help control communication to the muscles and between these brain areas.
Collapse
|
87
|
Cowley BR, Kaufman MT, Butler ZS, Churchland MM, Ryu SI, Shenoy KV, Yu BM. DataHigh: graphical user interface for visualizing and interacting with high-dimensional neural activity. J Neural Eng 2013; 10:066012. [PMID: 24216250 DOI: 10.1088/1741-2560/10/6/066012] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Analyzing and interpreting the activity of a heterogeneous population of neurons can be challenging, especially as the number of neurons, experimental trials, and experimental conditions increases. One approach is to extract a set of latent variables that succinctly captures the prominent co-fluctuation patterns across the neural population. A key problem is that the number of latent variables needed to adequately describe the population activity is often greater than 3, thereby preventing direct visualization of the latent space. By visualizing a small number of 2-d projections of the latent space or each latent variable individually, it is easy to miss salient features of the population activity. APPROACH To address this limitation, we developed a Matlab graphical user interface (called DataHigh) that allows the user to quickly and smoothly navigate through a continuum of different 2-d projections of the latent space. We also implemented a suite of additional visualization tools (including playing out population activity timecourses as a movie and displaying summary statistics, such as covariance ellipses and average timecourses) and an optional tool for performing dimensionality reduction. MAIN RESULTS To demonstrate the utility and versatility of DataHigh, we used it to analyze single-trial spike count and single-trial timecourse population activity recorded using a multi-electrode array, as well as trial-averaged population activity recorded using single electrodes. SIGNIFICANCE DataHigh was developed to fulfil a need for visualization in exploratory neural data analysis, which can provide intuition that is critical for building scientific hypotheses and models of population activity.
Collapse
|
88
|
Foster JD, Nuyujukian P, Freifeld O, Ryu SI, Black MJ, Shenoy KV. A framework for relating neural activity to freely moving behavior. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2013; 2012:2736-9. [PMID: 23366491 DOI: 10.1109/embc.2012.6346530] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Two research communities, motor systems neuroscience and motor prosthetics, examine the relationship between neural activity in the motor cortex and movement. The former community aims to understand how the brain controls and generates movement; the latter community focuses on how to decode neural activity as control signals for a prosthetic cursor or limb. Both have made progress toward understanding the relationship between neural activity in the motor cortex and behavior. However, these findings are tested using animal models in an environment that constrains behavior to simple, limited movements. These experiments show that, in constrained settings, simple reaching motions can be decoded from small populations of spiking neurons. It is unclear whether these findings hold for more complex, full-body behaviors in unconstrained settings. Here we present the results of freely-moving behavioral experiments from a monkey with simultaneous intracortical recording. We investigated neural firing rates while the monkey performed various tasks such as walking on a treadmill, reaching for food, and sitting idly. We show that even in such an unconstrained and varied context, neural firing rates are well tuned to behavior, supporting findings of basic neuroscience. Further, we demonstrate that the various behavioral tasks can be reliably classified with over 95% accuracy, illustrating the viability of decoding techniques despite significant variation and environmental distractions associated with unconstrained behavior. Such encouraging results hint at potential utility of the freely-moving experimental paradigm.
Collapse
|
89
|
Gilja V, Nuyujukian P, Chestek CA, Cunningham JP, Yu BM, Fan JM, Ryu SI, Shenoy KV. A brain machine interface control algorithm designed from a feedback control perspective. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2013; 2012:1318-22. [PMID: 23366141 DOI: 10.1109/embc.2012.6346180] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
We present a novel brain machine interface (BMI) control algorithm, the recalibrated feedback intention-trained Kalman filter (ReFIT-KF). The design of ReFIT-KF is motivated from a feedback control perspective applied to existing BMI control algorithms. The result is two design innovations that alter the modeling assumptions made by these algorithms and the methods by which these algorithms are trained. In online neural control experiments recording from a 96-electrode array implanted in M1 of a macaque monkey, the ReFIT-KF control algorithm demonstrates large performance improvements over the current state of the art velocity Kalman filter, reducing target acquisition time by a factor of two, while maintaining a 500 ms hold period, thereby increasing the clinical viability of BMI systems.
Collapse
|
90
|
Henderson JM, Gilja V, Pandarinath C, Blabe C, Hochberg LR, Shenoy KV. 194 High Performance Computer Cursor Control Using Neuronal Ensemble Recordings From the Motor Cortex of a Person With ALS. Neurosurgery 2013. [DOI: 10.1227/01.neu.0000432784.58847.74] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
|
91
|
Cowley BR, Kaufman MT, Churchland MM, Ryu SI, Shenoy KV, Yu BM. DataHigh: graphical user interface for visualizing and interacting with high-dimensional neural activity. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2013; 2012:4607-10. [PMID: 23366954 DOI: 10.1109/embc.2012.6346993] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The activity of tens to hundreds of neurons can be succinctly summarized by a smaller number of latent variables extracted using dimensionality reduction methods. These latent variables define a reduced-dimensional space in which we can study how population activity varies over time, across trials, and across experimental conditions. Ideally, we would like to visualize the population activity directly in the reduced-dimensional space, whose optimal dimensionality (as determined from the data) is typically greater than 3. However, direct plotting can only provide a 2D or 3D view. To address this limitation, we developed a Matlab graphical user interface (GUI) that allows the user to quickly navigate through a continuum of different 2D projections of the reduced-dimensional space. To demonstrate the utility and versatility of this GUI, we applied it to visualize population activity recorded in premotor and motor cortices during reaching tasks. Examples include single-trial population activity recorded using a multi-electrode array, as well as trial-averaged population activity recorded sequentially using single electrodes. Because any single 2D projection may provide a misleading impression of the data, being able to see a large number of 2D projections is critical for intuition-and hypothesis-building during exploratory data analysis. The GUI includes a suite of additional interactive tools, including playing out population activity timecourses as a movie and displaying summary statistics, such as covariance ellipses and average timecourses. The use of visualization tools like the GUI developed here, in tandem with dimensionality reduction methods, has the potential to further our understanding of neural population activity.
Collapse
|
92
|
Ozden I, Wang J, Lu Y, May T, Lee J, Goo W, O'Shea DJ, Kalanithi P, Diester I, Diagne M, Deisseroth K, Shenoy KV, Nurmikko AV. A coaxial optrode as multifunction write-read probe for optogenetic studies in non-human primates. J Neurosci Methods 2013; 219:142-54. [PMID: 23867081 DOI: 10.1016/j.jneumeth.2013.06.011] [Citation(s) in RCA: 81] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2013] [Revised: 06/22/2013] [Accepted: 06/28/2013] [Indexed: 01/12/2023]
Abstract
BACKGROUND Advances in optogenetics have led to first reports of expression of light-gated ion-channels in non-human primates (NHPs). However, a major obstacle preventing effective application of optogenetics in NHPs and translation to optogenetic therapeutics is the absence of compatible multifunction optoelectronic probes for (1) precision light delivery, (2) low-interference electrophysiology, (3) protein fluorescence detection, and (4) repeated insertion with minimal brain trauma. NEW METHOD Here we describe a novel brain probe device, a "coaxial optrode", designed to minimize brain tissue damage while microfabricated to perform simultaneous electrophysiology, light delivery and fluorescence measurements in the NHP brain. The device consists of a tapered, gold-coated optical fiber inserted in a polyamide tube. A portion of the gold coating is exposed at the fiber tip to allow electrophysiological recordings in addition to light delivery/collection at the tip. RESULTS Coaxial optrode performance was demonstrated by experiments in rodents and NHPs, and characterized by computational models. The device mapped opsin expression in the brain and achieved precisely targeted optical stimulation and electrophysiology with minimal cortical damage. COMPARISON WITH EXISTING METHODS Overall, combined electrical, optical and mechanical features of the coaxial optrode allowed a performance for NHP studies which was not possible with previously existing devices. CONCLUSIONS Coaxial optrode is currently being used in two NHP laboratories as a major tool to study brain function by inducing light modulated neural activity and behavior. By virtue of its design, the coaxial optrode can be extended for use as a chronic implant and multisite neural stimulation/recording.
Collapse
|
93
|
Shenoy KV, Sahani M, Churchland MM. Cortical control of arm movements: a dynamical systems perspective. Annu Rev Neurosci 2013; 36:337-59. [PMID: 23725001 DOI: 10.1146/annurev-neuro-062111-150509] [Citation(s) in RCA: 431] [Impact Index Per Article: 39.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Our ability to move is central to everyday life. Investigating the neural control of movement in general, and the cortical control of volitional arm movements in particular, has been a major research focus in recent decades. Studies have involved primarily either attempts to account for single-neuron responses in terms of tuning for movement parameters or attempts to decode movement parameters from populations of tuned neurons. Even though this focus on encoding and decoding has led to many seminal advances, it has not produced an agreed-upon conceptual framework. Interest in understanding the underlying neural dynamics has recently increased, leading to questions such as how does the current population response determine the future population response, and to what purpose? We review how a dynamical systems perspective may help us understand why neural activity evolves the way it does, how neural activity relates to movement parameters, and how a unified conceptual framework may result.
Collapse
|
94
|
Kaufman MT, Churchland MM, Shenoy KV. The roles of monkey M1 neuron classes in movement preparation and execution. J Neurophysiol 2013; 110:817-25. [PMID: 23699057 PMCID: PMC3742981 DOI: 10.1152/jn.00892.2011] [Citation(s) in RCA: 65] [Impact Index Per Article: 5.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023] Open
Abstract
The motor cortices exhibit substantial activity while preparing movements, yet the arm remains still during preparation. We investigated whether a subpopulation of presumed inhibitory neurons in primary motor cortex (M1) might be involved in “gating” motor output during preparation, while permitting output during movement. This hypothesis predicts a release of inhibition just before movement onset. In data from M1 of two monkeys, we did not find evidence for this hypothesis: few neurons exhibited a clear pause during movement, and these were at the tail end of a broad distribution. We then identified a subpopulation likely to be enriched for inhibitory interneurons, using their waveform shapes. We found that the firing rates of this subpopulation tended to increase during movement instead of decreasing as predicted by the M1 gating model. No clear subset that might implement an inhibitory gate was observed. Together with previous evidence against upstream inhibitory mechanisms in premotor cortex, this provides evidence against an inhibitory “gate” for motor output in cortex. Instead, it appears that some other mechanism must likely exist.
Collapse
|
95
|
Dethier J, Nuyujukian P, Ryu SI, Shenoy KV, Boahen K. Design and validation of a real-time spiking-neural-network decoder for brain-machine interfaces. J Neural Eng 2013; 10:036008. [PMID: 23574919 DOI: 10.1088/1741-2560/10/3/036008] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Cortically-controlled motor prostheses aim to restore functions lost to neurological disease and injury. Several proof of concept demonstrations have shown encouraging results, but barriers to clinical translation still remain. In particular, intracortical prostheses must satisfy stringent power dissipation constraints so as not to damage cortex. APPROACH One possible solution is to use ultra-low power neuromorphic chips to decode neural signals for these intracortical implants. The first step is to explore in simulation the feasibility of translating decoding algorithms for brain-machine interface (BMI) applications into spiking neural networks (SNNs). MAIN RESULTS Here we demonstrate the validity of the approach by implementing an existing Kalman-filter-based decoder in a simulated SNN using the Neural Engineering Framework (NEF), a general method for mapping control algorithms onto SNNs. To measure this system's robustness and generalization, we tested it online in closed-loop BMI experiments with two rhesus monkeys. Across both monkeys, a Kalman filter implemented using a 2000-neuron SNN has comparable performance to that of a Kalman filter implemented using standard floating point techniques. SIGNIFICANCE These results demonstrate the tractability of SNN implementations of statistical signal processing algorithms on different monkeys and for several tasks, suggesting that a SNN decoder, implemented on a neuromorphic chip, may be a feasible computational platform for low-power fully-implanted prostheses. The validation of this closed-loop decoder system and the demonstration of its robustness and generalization hold promise for SNN implementations on an ultra-low power neuromorphic chip using the NEF.
Collapse
|
96
|
Chestek CA, Gilja V, Blabe CH, Foster BL, Shenoy KV, Parvizi J, Henderson JM. Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas. J Neural Eng 2013; 10:026002. [PMID: 23369953 DOI: 10.1088/1741-2560/10/2/026002] [Citation(s) in RCA: 99] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Brain-machine interface systems translate recorded neural signals into command signals for assistive technology. In individuals with upper limb amputation or cervical spinal cord injury, the restoration of a useful hand grasp could significantly improve daily function. We sought to determine if electrocorticographic (ECoG) signals contain sufficient information to select among multiple hand postures for a prosthetic hand, orthotic, or functional electrical stimulation system. APPROACH We recorded ECoG signals from subdural macro- and microelectrodes implanted in motor areas of three participants who were undergoing inpatient monitoring for diagnosis and treatment of intractable epilepsy. Participants performed five distinct isometric hand postures, as well as four distinct finger movements. Several control experiments were attempted in order to remove sensory information from the classification results. Online experiments were performed with two participants. MAIN RESULTS Classification rates were 68%, 84% and 81% for correct identification of 5 isometric hand postures offline. Using 3 potential controls for removing sensory signals, error rates were approximately doubled on average (2.1×). A similar increase in errors (2.6×) was noted when the participant was asked to make simultaneous wrist movements along with the hand postures. In online experiments, fist versus rest was successfully classified on 97% of trials; the classification output drove a prosthetic hand. Online classification performance for a larger number of hand postures remained above chance, but substantially below offline performance. In addition, the long integration windows used would preclude the use of decoded signals for control of a BCI system. SIGNIFICANCE These results suggest that ECoG is a plausible source of command signals for prosthetic grasp selection. Overall, avenues remain for improvement through better electrode designs and placement, better participant training, and characterization of non-stationarities such that ECoG could be a viable signal source for grasp control for amputees or individuals with paralysis.
Collapse
|
97
|
Kao JC, Nuyujukian P, Stavisky S, Ryu SI, Ganguli S, Shenoy KV. Investigating the role of firing-rate normalization and dimensionality reduction in brain-machine interface robustness. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2013; 2013:293-298. [PMID: 24109682 DOI: 10.1109/embc.2013.6609495] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
The intraday robustness of brain-machine interfaces (BMIs) is important to their clinical viability. In particular, BMIs must be robust to intraday perturbations in neuron firing rates, which may arise from several factors including recording loss and external noise. Using a state-of-the-art decode algorithm, the Recalibrated Feedback Intention Trained Kalman filter (ReFIT-KF) we introduce two novel modifications: (1) a normalization of the firing rates, and (2) a reduction of the dimensionality of the data via principal component analysis (PCA). We demonstrate in online studies that a ReFIT-KF equipped with normalization and PCA (NPC-ReFIT-KF) (1) achieves comparable performance to a standard ReFIT-KF when at least 60% of the neural variance is captured, and (2) is more robust to the undetected loss of channels. We present intuition as to how both modifications may increase the robustness of BMIs, and investigate the contribution of each modification to robustness. These advances, which lead to a decoder achieving state-of-the-art performance with improved robustness, are important for the clinical viability of BMI systems.
Collapse
|
98
|
Churchland MM, Cunningham JP, Kaufman MT, Foster JD, Nuyujukian P, Ryu SI, Shenoy KV. Neural population dynamics during reaching. Nature 2012; 487:51-6. [PMID: 22722855 PMCID: PMC3393826 DOI: 10.1038/nature11129] [Citation(s) in RCA: 789] [Impact Index Per Article: 65.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2011] [Accepted: 04/05/2012] [Indexed: 11/25/2022]
Abstract
Most theories of motor cortex have assumed that neural activity represents movement parameters. This view derives from what is known about primary visual cortex, where neural activity represents patterns of light. Yet it is unclear how well the analogy between motor and visual cortex holds. Single-neuron responses in motor cortex are complex, and there is marked disagreement regarding which movement parameters are represented. A better analogy might be with other motor systems, where a common principle is rhythmic neural activity. Here we find that motor cortex responses during reaching contain a brief but strong oscillatory component, something quite unexpected for a non-periodic behaviour. Oscillation amplitude and phase followed naturally from the preparatory state, suggesting a mechanistic role for preparatory neural activity. These results demonstrate an unexpected yet surprisingly simple structure in the population response. This underlying structure explains many of the confusing features of individual neural responses.
Collapse
|
99
|
Nuyujukian P, Fan JM, Gilja V, Kalanithi PS, Chestek CA, Shenoy KV. Monkey models for brain-machine interfaces: the need for maintaining diversity. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2012; 2011:1301-5. [PMID: 22254555 DOI: 10.1109/iembs.2011.6090306] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Brain-machine interfaces (BMIs) aim to help disabled patients by translating neural signals from the brain into control signals for guiding prosthetic arms, computer cursors, and other assistive devices. Animal models are central to the development of these systems and have helped enable the successful translation of the first generation of BMIs. As we move toward next-generation systems, we face the question of which animal models will aid broader patient populations and achieve even higher performance, robustness, and functionality. We review here four general types of rhesus monkey models employed in BMI research, and describe two additional, complementary models. Given the physiological diversity of neurological injury and disease, we suggest a need to maintain the current diversity of animal models and to explore additional alternatives, as each mimic different aspects of injury or disease.
Collapse
|
100
|
Afshar A, Santhanam G, Yu BM, Ryu SI, Sahani M, Shenoy KV. Single-trial neural correlates of arm movement preparation. Neuron 2011; 71:555-64. [PMID: 21835350 DOI: 10.1016/j.neuron.2011.05.047] [Citation(s) in RCA: 146] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/25/2011] [Indexed: 11/16/2022]
Abstract
The process by which neural circuitry in the brain plans and executes movements is not well understood. Until recently, most available data were limited either to single-neuron electrophysiological recordings or to measures of aggregate field or metabolism. Neither approach reveals how individual neurons' activities are coordinated within the population, and thus inferences about how the neural circuit forms a motor plan for an upcoming movement have been indirect. Here we build on recent advances in the measurement and description of population activity to frame and test an "initial condition hypothesis" of arm movement preparation and initiation. This hypothesis leads to a model in which the timing of movements may be predicted on each trial using neurons' moment-by-moment firing rates and rates of change of those rates. Using simultaneous microelectrode array recordings from premotor cortex of monkeys performing delayed-reach movements, we compare such single-trial predictions to those of other theories. We show that our model can explain approximately 4-fold more arm-movement reaction-time variance than the best alternative method. Thus, the initial condition hypothesis elucidates a view of the relationship between single-trial preparatory neural population dynamics and single-trial behavior.
Collapse
|