1
|
Churchland MM, Shenoy KV. Preparatory activity and the expansive null-space. Nat Rev Neurosci 2024; 25:213-236. [PMID: 38443626 DOI: 10.1038/s41583-024-00796-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/26/2024] [Indexed: 03/07/2024]
Abstract
The study of the cortical control of movement experienced a conceptual shift over recent decades, as the basic currency of understanding shifted from single-neuron tuning towards population-level factors and their dynamics. This transition was informed by a maturing understanding of recurrent networks, where mechanism is often characterized in terms of population-level factors. By estimating factors from data, experimenters could test network-inspired hypotheses. Central to such hypotheses are 'output-null' factors that do not directly drive motor outputs yet are essential to the overall computation. In this Review, we highlight how the hypothesis of output-null factors was motivated by the venerable observation that motor-cortex neurons are active during movement preparation, well before movement begins. We discuss how output-null factors then became similarly central to understanding neural activity during movement. We discuss how this conceptual framework provided key analysis tools, making it possible for experimenters to address long-standing questions regarding motor control. We highlight an intriguing trend: as experimental and theoretical discoveries accumulate, the range of computational roles hypothesized to be subserved by output-null factors continues to expand.
Collapse
Affiliation(s)
- Mark M Churchland
- Department of Neuroscience, Columbia University, New York, NY, USA.
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| |
Collapse
|
2
|
Deo DR, Willett FR, Avansino DT, Hochberg LR, Henderson JM, Shenoy KV. Brain control of bimanual movement enabled by recurrent neural networks. Sci Rep 2024; 14:1598. [PMID: 38238386 PMCID: PMC10796685 DOI: 10.1038/s41598-024-51617-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 01/07/2024] [Indexed: 01/22/2024] Open
Abstract
Brain-computer interfaces have so far focused largely on enabling the control of a single effector, for example a single computer cursor or robotic arm. Restoring multi-effector motion could unlock greater functionality for people with paralysis (e.g., bimanual movement). However, it may prove challenging to decode the simultaneous motion of multiple effectors, as we recently found that a compositional neural code links movements across all limbs and that neural tuning changes nonlinearly during dual-effector motion. Here, we demonstrate the feasibility of high-quality bimanual control of two cursors via neural network (NN) decoders. Through simulations, we show that NNs leverage a neural 'laterality' dimension to distinguish between left and right-hand movements as neural tuning to both hands become increasingly correlated. In training recurrent neural networks (RNNs) for two-cursor control, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously. Our results suggest that neural network decoders may be advantageous for multi-effector decoding, provided they are designed to transfer to the online setting.
Collapse
Affiliation(s)
- Darrel R Deo
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
| | - Francis R Willett
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | - Donald T Avansino
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- School of Engineering, Brown University, Providence, RI, USA
- Carney Institute for Brain Science, Brown University, Providence, RI, USA
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| |
Collapse
|
3
|
Fan C, Hahn N, Kamdar F, Avansino D, Wilson GH, Hochberg L, Shenoy KV, Henderson JM, Willett FR. Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication. Adv Neural Inf Process Syst 2023; 36:42258-42270. [PMID: 38738213 PMCID: PMC11086983] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/14/2024]
Abstract
Intracortical brain-computer interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. This requires iBCI users to stop using the iBCI and engage in supervised data collection, making the iBCI system hard to use. In this paper, we propose a method that enables self-recalibration of communication iBCIs without interrupting the user. Our method leverages large language models (LMs) to automatically correct errors in iBCI outputs. The self-recalibration process uses these corrected outputs ("pseudo-labels") to continually update the iBCI decoder online. Over a period of more than one year (403 days), we evaluated our Continual Online Recalibration with Pseudo-labels (CORP) framework with one clinical trial participant. CORP achieved a stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods. Notably, this is the longest-running iBCI stability demonstration involving a human participant. Our results provide the first evidence for long-term stabilization of a plug-and-play, high-performance communication iBCI, addressing a major barrier for the clinical translation of iBCIs.
Collapse
Affiliation(s)
- Chaofei Fan
- Department of Computer Science, Stanford University
| | - Nick Hahn
- Department of Neurosurgery, Stanford University
| | | | | | | | - Leigh Hochberg
- School of Engineering and Carney Institute for Brain Science, Brown University
- VA RR&D Center for Neurorestoration and Neurotechnology, VA Providence Healthcare System
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School
| | - Krishna V. Shenoy
- Bio-X Program, Stanford University
- Department of Neurobiology, Stanford University
- Department of Bioengineering, Stanford University
- Wu Tsai Neurosciences Institute, Stanford University
- Howard Hughes Medical Institute at Stanford University
- Department of Electrical Engineering, Stanford University
| | - Jaimie M. Henderson
- Department of Neurosurgery, Stanford University
- Wu Tsai Neurosciences Institute, Stanford University
| | | |
Collapse
|
4
|
Fan C, Hahn N, Kamdar F, Avansino D, Wilson GH, Hochberg L, Shenoy KV, Henderson JM, Willett FR. Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication. ArXiv 2023:arXiv:2311.03611v1. [PMID: 37986728 PMCID: PMC10659441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Subscribe] [Scholar Register] [Indexed: 11/22/2023]
Abstract
Intracortical brain-computer interfaces (iBCIs) have shown promise for restoring rapid communication to people with neurological disorders such as amyotrophic lateral sclerosis (ALS). However, to maintain high performance over time, iBCIs typically need frequent recalibration to combat changes in the neural recordings that accrue over days. This requires iBCI users to stop using the iBCI and engage in supervised data collection, making the iBCI system hard to use. In this paper, we propose a method that enables self-recalibration of communication iBCIs without interrupting the user. Our method leverages large language models (LMs) to automatically correct errors in iBCI outputs. The self-recalibration process uses these corrected outputs ("pseudo-labels") to continually update the iBCI decoder online. Over a period of more than one year (403 days), we evaluated our Continual Online Recalibration with Pseudo-labels (CORP) framework with one clinical trial participant. CORP achieved a stable decoding accuracy of 93.84% in an online handwriting iBCI task, significantly outperforming other baseline methods. Notably, this is the longest-running iBCI stability demonstration involving a human participant. Our results provide the first evidence for long-term stabilization of a plug-and-play, high-performance communication iBCI, addressing a major barrier for the clinical translation of iBCIs.
Collapse
|
5
|
Verhein JR, Vyas S, Shenoy KV. Methylphenidate modulates motor cortical dynamics and behavior. bioRxiv 2023:2023.10.15.562405. [PMID: 37905157 PMCID: PMC10614820 DOI: 10.1101/2023.10.15.562405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2023]
Abstract
Methylphenidate (MPH, brand: Ritalin) is a common stimulant used both medically and non-medically. Though typically prescribed for its cognitive effects, MPH also affects movement. While it is known that MPH noncompetitively blocks the reuptake of catecholamines through inhibition of dopamine and norepinephrine transporters, a critical step in exploring how it affects behavior is to understand how MPH directly affects neural activity. This would establish an electrophysiological mechanism of action for MPH. Since we now have biologically-grounded network-level hypotheses regarding how populations of motor cortical neurons plan and execute movements, there is a unique opportunity to make testable predictions regarding how systemic MPH administration - a pharmacological perturbation - might affect neural activity in motor cortex. To that end, we administered clinically-relevant doses of MPH to Rhesus monkeys as they performed an instructed-delay reaching task. Concomitantly, we measured neural activity from dorsal premotor and primary motor cortex. Consistent with our predictions, we found dose-dependent and significant effects on reaction time, trial-by-trial variability, and movement speed. We confirmed our hypotheses that changes in reaction time and variability were accompanied by previously established population-level changes in motor cortical preparatory activity and the condition-independent signal that precedes movements. We expected changes in speed to be a result of changes in the amplitude of motor cortical dynamics and/or a translation of those dynamics in activity space. Instead, our data are consistent with a mechanism whereby the neuromodulatory effect of MPH is to increase the gain and/or the signal-to-noise of motor cortical dynamics during reaching. Continued work in this domain to better understand the brain-wide electrophysiological mechanism of action of MPH and other psychoactive drugs could facilitate more targeted treatments for a host of cognitive-motor disorders.
Collapse
Affiliation(s)
- Jessica R Verhein
- Medical Scientist Training Program, Stanford School of Medicine, Stanford University, Stanford, CA
- Neurosciences Graduate Program, Stanford School of Medicine, Stanford University, Stanford, CA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA
- Current affiliations: Psychiatry Research Residency Training Program, University of California, San Francisco, San Francisco, CA
| | - Saurabh Vyas
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA
- Department of Bioengineering, Stanford University, Stanford, CA
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY
| | - Krishna V Shenoy
- Neurosciences Graduate Program, Stanford School of Medicine, Stanford University, Stanford, CA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA
- Department of Bioengineering, Stanford University, Stanford, CA
- Department of Electrical Engineering, Stanford University, Stanford, CA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA
- Department of Neurobiology, Stanford University, Stanford, CA
- Bio-X Program, Stanford University, Stanford, CA
| |
Collapse
|
6
|
Boucher PO, Wang T, Carceroni L, Kane G, Shenoy KV, Chandrasekaran C. Initial conditions combine with sensory evidence to induce decision-related dynamics in premotor cortex. Nat Commun 2023; 14:6510. [PMID: 37845221 PMCID: PMC10579235 DOI: 10.1038/s41467-023-41752-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Accepted: 09/18/2023] [Indexed: 10/18/2023] Open
Abstract
We used a dynamical systems perspective to understand decision-related neural activity, a fundamentally unresolved problem. This perspective posits that time-varying neural activity is described by a state equation with an initial condition and evolves in time by combining at each time step, recurrent activity and inputs. We hypothesized various dynamical mechanisms of decisions, simulated them in models to derive predictions, and evaluated these predictions by examining firing rates of neurons in the dorsal premotor cortex (PMd) of monkeys performing a perceptual decision-making task. Prestimulus neural activity (i.e., the initial condition) predicted poststimulus neural trajectories, covaried with RT and the outcome of the previous trial, but not with choice. Poststimulus dynamics depended on both the sensory evidence and initial condition, with easier stimuli and fast initial conditions leading to the fastest choice-related dynamics. Together, these results suggest that initial conditions combine with sensory evidence to induce decision-related dynamics in PMd.
Collapse
Affiliation(s)
- Pierre O Boucher
- Department of Biomedical Engineering, Boston University, Boston, 02115, MA, USA
| | - Tian Wang
- Department of Biomedical Engineering, Boston University, Boston, 02115, MA, USA
| | - Laura Carceroni
- Undergraduate Program in Neuroscience, Boston University, Boston, 02115, MA, USA
| | - Gary Kane
- Department of Psychological and Brain Sciences, Boston University, Boston, 02115, MA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, 94305, CA, USA
- Department of Neurobiology, Stanford University, Stanford, 94305, CA, USA
- Howard Hughes Medical Institute, HHMI, Chevy Chase, 20815-6789, MD, USA
- Department of Bioengineering, Stanford University, Stanford, 94305, CA, USA
- Stanford Neurosciences Institute, Stanford University, Stanford, 94305, CA, USA
- Bio-X Program, Stanford University, Stanford, 94305, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, 94305, CA, USA
| | - Chandramouli Chandrasekaran
- Department of Biomedical Engineering, Boston University, Boston, 02115, MA, USA.
- Department of Psychological and Brain Sciences, Boston University, Boston, 02115, MA, USA.
- Center for Systems Neuroscience, Boston University, Boston, 02115, MA, USA.
- Department of Anatomy & Neurobiology, Boston University, Boston, 02118, MA, USA.
| |
Collapse
|
7
|
Coughlin B, Muñoz W, Kfir Y, Young MJ, Meszéna D, Jamali M, Caprara I, Hardstone R, Khanna A, Mustroph ML, Trautmann EM, Windolf C, Varol E, Soper DJ, Stavisky SD, Welkenhuysen M, Dutta B, Shenoy KV, Hochberg LR, Mark Richardson R, Williams ZM, Cash SS, Paulk AC. Modified Neuropixels probes for recording human neurophysiology in the operating room. Nat Protoc 2023; 18:2927-2953. [PMID: 37697108 DOI: 10.1038/s41596-023-00871-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 06/08/2023] [Indexed: 09/13/2023]
Abstract
Neuropixels are silicon-based electrophysiology-recording probes with high channel count and recording-site density. These probes offer a turnkey platform for measuring neural activity with single-cell resolution and at a scale that is beyond the capabilities of current clinically approved devices. Our team demonstrated the first-in-human use of these probes during resection surgery for epilepsy or tumors and deep brain stimulation electrode placement in patients with Parkinson's disease. Here, we provide a better understanding of the capabilities and challenges of using Neuropixels as a research tool to study human neurophysiology, with the hope that this information may inform future efforts toward regulatory approval of Neuropixels probes as research devices. In perioperative procedures, the major concerns are the initial sterility of the device, maintaining a sterile field during surgery, having multiple referencing and grounding schemes available to de-noise recordings (if necessary), protecting the silicon probe from accidental contact before insertion and obtaining high-quality action potential and local field potential recordings. The research team ensures that the device is fully operational while coordinating with the surgical team to remove sources of electrical noise that could otherwise substantially affect the signals recorded by the sensitive hardware. Prior preparation using the equipment and training in human clinical research and working in operating rooms maximize effective communication within and between the teams, ensuring high recording quality and minimizing the time added to the surgery. The perioperative procedure requires ~4 h, and the entire protocol requires multiple weeks.
Collapse
Affiliation(s)
- Brian Coughlin
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - William Muñoz
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Yoav Kfir
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Michael J Young
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Domokos Meszéna
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Mohsen Jamali
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Irene Caprara
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Richard Hardstone
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Arjun Khanna
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Martina L Mustroph
- Department of Neurosurgery, Harvard Medical School and Brigham & Women's Hospital, Boston, MA, USA
| | - Eric M Trautmann
- Department of Neuroscience, Columbia University Medical Center, New York, NY, USA
- Zuckerman Institute, Columbia University, New York, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University Medical Center, New York, NY, USA
| | - Charlie Windolf
- Department of Statistics, Zuckerman Institute, Columbia University, New York, NY, USA
| | - Erdem Varol
- Department of Statistics, Zuckerman Institute, Columbia University, New York, NY, USA
- Department of Computer Science and Engineering, Zuckerman Institute, Columbia University, New York, NY, USA
| | - Dan J Soper
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
| | - Sergey D Stavisky
- Department of Neurological Surgery, University of California Davis, Davis, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | | | | | - Krishna V Shenoy
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - R Mark Richardson
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Ziv M Williams
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA.
| | - Sydney S Cash
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurology, Harvard Medical School, Boston, MA, USA.
| | - Angelique C Paulk
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurology, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
8
|
Willett FR, Kunz EM, Fan C, Avansino DT, Wilson GH, Choi EY, Kamdar F, Glasser MF, Hochberg LR, Druckmann S, Shenoy KV, Henderson JM. A high-performance speech neuroprosthesis. Nature 2023; 620:1031-1036. [PMID: 37612500 PMCID: PMC10468393 DOI: 10.1038/s41586-023-06377-x] [Citation(s) in RCA: 35] [Impact Index Per Article: 35.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2023] [Accepted: 06/27/2023] [Indexed: 08/25/2023]
Abstract
Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text1,2 or sound3,4. Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary1-7. Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant-who can no longer speak intelligibly owing to amyotrophic lateral sclerosis-achieved a 9.1% word error rate on a 50-word vocabulary (2.7 times fewer errors than the previous state-of-the-art speech BCI2) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant's attempted speech was decoded at 62 words per minute, which is 3.4 times as fast as the previous record8 and begins to approach the speed of natural conversation (160 words per minute9). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.
Collapse
Affiliation(s)
- Francis R Willett
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA.
| | - Erin M Kunz
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Chaofei Fan
- Department of Computer Science, Stanford University, Stanford, CA, USA
| | - Donald T Avansino
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | - Guy H Wilson
- Department of Neuroscience, Stanford University, Stanford, CA, USA
| | - Eun Young Choi
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Foram Kamdar
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Matthew F Glasser
- Department of Neuroscience, Washington University in St. Louis, St. Louis, MO, USA
- Department of Radiology, Washington University in St. Louis, St. Louis, MO, USA
| | - Leigh R Hochberg
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Shaul Druckmann
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Bio-X Program, Stanford University, Stanford, CA, USA
| | - Jaimie M Henderson
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| |
Collapse
|
9
|
Genkin M, Shenoy KV, Chandrasekaran C, Engel TA. The dynamics and geometry of choice in premotor cortex. bioRxiv 2023:2023.07.22.550183. [PMID: 37546748 PMCID: PMC10401920 DOI: 10.1101/2023.07.22.550183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
The brain represents sensory variables in the coordinated activity of neural populations, in which tuning curves of single neurons define the geometry of the population code. Whether the same coding principle holds for dynamic cognitive variables remains unknown because internal cognitive processes unfold with a unique time course on single trials observed only in the irregular spiking of heterogeneous neural populations. Here we show the existence of such a population code for the dynamics of choice formation in the primate premotor cortex. We developed an approach to simultaneously infer population dynamics and tuning functions of single neurons to the population state. Applied to spike data recorded during decision-making, our model revealed that populations of neurons encoded the same dynamic variable predicting choices, and heterogeneous firing rates resulted from the diverse tuning of single neurons to this decision variable. The inferred dynamics indicated an attractor mechanism for decision computation. Our results reveal a common geometric principle for neural encoding of sensory and dynamic cognitive variables.
Collapse
Affiliation(s)
| | - Krishna V Shenoy
- Howard Hughes Medical Institute, Stanford University, Stanford, CA
- Department of Electrical Engineering, Stanford University, Stanford, CA
| | - Chandramouli Chandrasekaran
- Department of Anatomy & Neurobiology, Boston University, Boston, MA
- Department of Psychological and Brain Sciences, Boston University, Boston, MA
- Center for Systems Neuroscience, Boston University, Boston, MA
| | - Tatiana A Engel
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ
| |
Collapse
|
10
|
Williams AH, Poole B, Maheswaranathan N, Dhawale AK, Fisher T, Wilson CD, Brann DH, Trautmann EM, Ryu S, Shusterman R, Rinberg D, Ölveczky BP, Shenoy KV, Gangul S. Discovering Precise Temporal Patterns in Large-Scale Neural Recordings through Robust and Interpretable Time Warping. Neuron 2023; 111:1685. [PMID: 37201504 DOI: 10.1016/j.neuron.2023.04.030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
|
11
|
Deo DR, Willett FR, Avansino DT, Hochberg LR, Henderson JM, Shenoy KV. Translating deep learning to neuroprosthetic control. bioRxiv 2023:2023.04.21.537581. [PMID: 37131830 PMCID: PMC10153231 DOI: 10.1101/2023.04.21.537581] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Advances in deep learning have given rise to neural network models of the relationship between movement and brain activity that appear to far outperform prior approaches. Brain-computer interfaces (BCIs) that enable people with paralysis to control external devices, such as robotic arms or computer cursors, might stand to benefit greatly from these advances. We tested recurrent neural networks (RNNs) on a challenging nonlinear BCI problem: decoding continuous bimanual movement of two computer cursors. Surprisingly, we found that although RNNs appeared to perform well in offline settings, they did so by overfitting to the temporal structure of the training data and failed to generalize to real-time neuroprosthetic control. In response, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously, far outperforming standard linear methods. Our results provide evidence that preventing models from overfitting to temporal structure in training data may, in principle, aid in translating deep learning advances to the BCI setting, unlocking improved performance for challenging applications.
Collapse
|
12
|
Shah NP, Willsey MS, Hahn N, Kamdar F, Avansino DT, Hochberg LR, Shenoy KV, Henderson JM. A brain-computer typing interface using finger movements. Int IEEE EMBS Conf Neural Eng 2023; 2023:10.1109/ner52421.2023.10123912. [PMID: 37465143 PMCID: PMC10353344 DOI: 10.1109/ner52421.2023.10123912] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2023]
Abstract
Intracortical brain computer interfaces (iBCIs) decode neural activity from the cortex and enable motor and communication prostheses, such as cursor control, handwriting and speech, for people with paralysis. This paper introduces a new iBCI communication prosthesis using a 3D keyboard interface for typing using continuous, closed loop movement of multiple fingers. A participant-specific BCI keyboard prototype was developed for a BrainGate2 clinical trial participant (T5) using neural recordings from the hand-knob area of the left premotor cortex. We assessed the relative decoding accuracy of flexion/extension movements of individual single fingers (5 degrees of freedom (DOF)) vs. three groups of fingers (thumb, index-middle, and ring-small fingers, 3 DOF). Neural decoding using 3 independent DOF was more accurate (95%) than that using 5 DOF (76%). A virtual keyboard was then developed where each finger group moved along a flexion-extension arc to acquire targets that corresponded to English letters and symbols. The locations of these letter/symbols were optimized using natural language statistics, resulting in an approximately a 2× reduction in distance traveled by fingers on average compared to a random keyboard layout. This keyboard was tested using a simple real-time closed loop decoder enabling T5 to type with 31 symbols at 90% accuracy and approximately 2.3 sec/symbol (excluding a 2 second hold time) on average.
Collapse
Affiliation(s)
| | | | | | | | | | - Leigh R Hochberg
- Neurol., Mass. Gen. Hosp; Boston, MA; Brown Univ./VA Medical Center, Providence, RI
| | | | | |
Collapse
|
13
|
Rubin DB, Ajiboye AB, Barefoot L, Bowker M, Cash SS, Chen D, Donoghue JP, Eskandar EN, Friehs G, Grant C, Henderson JM, Kirsch RF, Marujo R, Masood M, Mernoff ST, Miller JP, Mukand JA, Penn RD, Shefner J, Shenoy KV, Simeral JD, Sweet JA, Walter BL, Williams ZM, Hochberg LR. Interim Safety Profile From the Feasibility Study of the BrainGate Neural Interface System. Neurology 2023; 100:e1177-e1192. [PMID: 36639237 PMCID: PMC10074470 DOI: 10.1212/wnl.0000000000201707] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2022] [Accepted: 11/03/2022] [Indexed: 01/15/2023] Open
Abstract
BACKGROUND AND OBJECTIVES Brain-computer interfaces (BCIs) are being developed to restore mobility, communication, and functional independence to people with paralysis. Though supported by decades of preclinical data, the safety of chronically implanted microelectrode array BCIs in humans is unknown. We report safety results from the prospective, open-label, nonrandomized BrainGate feasibility study (NCT00912041), the largest and longest-running clinical trial of an implanted BCI. METHODS Adults aged 18-75 years with quadriparesis from spinal cord injury, brainstem stroke, or motor neuron disease were enrolled through 7 clinical sites in the United States. Participants underwent surgical implantation of 1 or 2 microelectrode arrays in the motor cortex of the dominant cerebral hemisphere. The primary safety outcome was device-related serious adverse events (SAEs) requiring device explantation or resulting in death or permanently increased disability during the 1-year postimplant evaluation period. The secondary outcomes included the type and frequency of other adverse events and the feasibility of the BrainGate system for controlling a computer or other assistive technologies. RESULTS From 2004 to 2021, 14 adults enrolled in the BrainGate trial had devices surgically implanted. The average duration of device implantation was 872 days, yielding 12,203 days of safety experience. There were 68 device-related adverse events, including 6 device-related SAEs. The most common device-related adverse event was skin irritation around the percutaneous pedestal. There were no safety events that required device explantation, no unanticipated adverse device events, no intracranial infections, and no participant deaths or adverse events resulting in permanently increased disability related to the investigational device. DISCUSSION The BrainGate Neural Interface system has a safety record comparable with other chronically implanted medical devices. Given rapid recent advances in this technology and continued performance gains, these data suggest a favorable risk/benefit ratio in appropriately selected individuals to support ongoing research and development. TRIAL REGISTRATION INFORMATION ClinicalTrials.gov Identifier: NCT00912041. CLASSIFICATION OF EVIDENCE This study provides Class IV evidence that the neurosurgically placed BrainGate Neural Interface system is associated with a low rate of SAEs defined as those requiring device explantation, resulting in death, or resulting in permanently increased disability during the 1-year postimplant period.
Collapse
Affiliation(s)
- Daniel B Rubin
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA.
| | - A Bolu Ajiboye
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Laurie Barefoot
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Marguerite Bowker
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Sydney S Cash
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - David Chen
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - John P Donoghue
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Emad N Eskandar
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Gerhard Friehs
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Carol Grant
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Jaimie M Henderson
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Robert F Kirsch
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Rose Marujo
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Maryam Masood
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Stephen T Mernoff
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Jonathan P Miller
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Jon A Mukand
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Richard D Penn
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Jeremy Shefner
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Krishna V Shenoy
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - John D Simeral
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Jennifer A Sweet
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Benjamin L Walter
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Ziv M Williams
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| | - Leigh R Hochberg
- From the Center for Neurotechnology and Neurorecovery (CNTR) (D.B.R., L.B., S.S.C., C.G., R.M., M.M., L.R.H.), Department of Neurology, and Department of Neurosurgery (Z.M.W.), Massachusetts General Hospital, Boston; Harvard Medical School (D.B.R., S.S.C., L.R.H.), Boston, MA; Department of Biomedical Engineering (A.B.A., R.F.K.), Case Western Reserve University, Cleveland, OH; FES Center of Excellence, Rehab. R&D Service (A.B.A., R.F.K., J.P.M., J.A.S., B.L.W.), Louis Stokes Cleveland Department of Veterans Affairs Medical Center, OH; Center for Neurorestoration and Neurotechnology (CfNN) (M.B., J.P.D., J.D.S., L.R.H.), Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI; Legs and Walking Lab (D.C.), Shirley Ryan AbilityLab, Chicago, IL; Department of Physical Medicine and Rehabilitation (D.C.), Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, IL; Department of Neuroscience (J.P.D.), Robert J. and Nancy D. Carney Institute for Brain Science (J.P.D., J.D.S., L.R.H.), School of Engineering (J.P.D., J.D.S., L.R.H.), and Department of Rehabilitation Medicine (J.A.M.), Brown University, Providence, RI; Department of Neurological Surgery (E.N.E.), Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY; European University of Cyprus (G.F.), Nicosia, Cyprus; Department of Neurosurgery (J.M.H.), Stanford University School of Medicine, CA; Wu Tsai Neurosciences Institute (J.M.H., K.V.S.), Bio-X Institute (J.M.H., K.V.S.), and Departments of Neurobiology (K.V.S.), Electrical Engineering (K.V.S.), and Bioengineering (K.V.S.), Stanford University, CA; Department of Neurological Surgery (R.F.K., J.P.M., J.A.S.), University Hospitals Case Medical Center, Cleveland, OH; Neurology Section (S.T.M.), VA Providence Health Care System, Providence, RI; Department of Neurology (S.T.M.), Alpert Medical School of Brown University, Providence, RI; Sargent Rehabilitation Center (J.A.M.), Warwick, RI; Section of Neurosurgery (R.D.P.), Department of Surgery, University of Chicago; Department of Neurosurgery (R.D.P.), Rush University Medical Center, Chicago, IL; Department of Neurology (J.S.), Barrow Neurological Institute, Phoenix, AZ; Howard Hughes Medical Institute at Stanford University (K.V.S.); Center for Neurological Restoration (B.L.W.), Cleveland Clinic, OH; and Program in Neuroscience (Z.M.W.), Harvard-MIT Program in Health Sciences and Technology, Harvard Medical School, Boston, MA
| |
Collapse
|
14
|
Wilson GH, Willett FR, Stein EA, Kamdar F, Avansino DT, Hochberg LR, Shenoy KV, Druckmann S, Henderson JM. Long-term unsupervised recalibration of cursor BCIs. bioRxiv 2023:2023.02.03.527022. [PMID: 36778458 PMCID: PMC9915729 DOI: 10.1101/2023.02.03.527022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Intracortical brain-computer interfaces (iBCIs) require frequent recalibration to maintain robust performance due to changes in neural activity that accumulate over time. Compensating for this nonstationarity would enable consistently high performance without the need for supervised recalibration periods, where users cannot engage in free use of their device. Here we introduce a hidden Markov model (HMM) to infer what targets users are moving toward during iBCI use. We then retrain the system using these inferred targets, enabling unsupervised adaptation to changing neural activity. Our approach outperforms the state of the art in large-scale, closed-loop simulations over two months and in closed-loop with a human iBCI user over one month. Leveraging an offline dataset spanning five years of iBCI recordings, we further show how recently proposed data distribution-matching approaches to recalibration fail over long time scales; only target-inference methods appear capable of enabling long-term unsupervised recalibration. Our results demonstrate how task structure can be used to bootstrap a noisy decoder into a highly-performant one, thereby overcoming one of the major barriers to clinically translating BCIs.
Collapse
|
15
|
Sylwestrak EL, Jo Y, Vesuna S, Wang X, Holcomb B, Tien RH, Kim DK, Fenno L, Ramakrishnan C, Allen WE, Chen R, Shenoy KV, Sussillo D, Deisseroth K. Cell-type-specific population dynamics of diverse reward computations. Cell 2022; 185:3568-3587.e27. [PMID: 36113428 PMCID: PMC10387374 DOI: 10.1016/j.cell.2022.08.019] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2021] [Revised: 06/16/2022] [Accepted: 08/17/2022] [Indexed: 01/26/2023]
Abstract
Computational analysis of cellular activity has developed largely independently of modern transcriptomic cell typology, but integrating these approaches may be essential for full insight into cellular-level mechanisms underlying brain function and dysfunction. Applying this approach to the habenula (a structure with diverse, intermingled molecular, anatomical, and computational features), we identified encoding of reward-predictive cues and reward outcomes in distinct genetically defined neural populations, including TH+ cells and Tac1+ cells. Data from genetically targeted recordings were used to train an optimized nonlinear dynamical systems model and revealed activity dynamics consistent with a line attractor. High-density, cell-type-specific electrophysiological recordings and optogenetic perturbation provided supporting evidence for this model. Reverse-engineering predicted how Tac1+ cells might integrate reward history, which was complemented by in vivo experimentation. This integrated approach describes a process by which data-driven computational models of population activity can generate and frame actionable hypotheses for cell-type-specific investigation in biological systems.
Collapse
Affiliation(s)
- Emily L Sylwestrak
- Department of Biology, University of Oregon, Eugene, OR 97403, USA; Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA.
| | - YoungJu Jo
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Department of Applied Physics, Stanford University, Stanford, CA 94305, USA
| | - Sam Vesuna
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
| | - Xiao Wang
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
| | - Blake Holcomb
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA
| | - Rebecca H Tien
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
| | - Doo Kyung Kim
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
| | - Lief Fenno
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA 94305, USA
| | - Charu Ramakrishnan
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
| | - William E Allen
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Neurosciences Interdepartmental Program, Stanford University, Stanford, CA 94303, USA
| | - Ritchie Chen
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
| | - Krishna V Shenoy
- Department of Neurobiology, Stanford University, Stanford, CA 94303, USA; Department of Electrical Engineering, Stanford University, Stanford, CA, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Karl Deisseroth
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA 94305, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA.
| |
Collapse
|
16
|
Paulk AC, Kfir Y, Khanna AR, Mustroph ML, Trautmann EM, Soper DJ, Stavisky SD, Welkenhuysen M, Dutta B, Shenoy KV, Hochberg LR, Richardson RM, Williams ZM, Cash SS. Large-scale neural recordings with single neuron resolution using Neuropixels probes in human cortex. Nat Neurosci 2022; 25:252-263. [PMID: 35102333 DOI: 10.1038/s41593-021-00997-0] [Citation(s) in RCA: 70] [Impact Index Per Article: 35.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Accepted: 12/07/2021] [Indexed: 12/20/2022]
Abstract
Recent advances in multi-electrode array technology have made it possible to monitor large neuronal ensembles at cellular resolution in animal models. In humans, however, current approaches restrict recordings to a few neurons per penetrating electrode or combine the signals of thousands of neurons in local field potential (LFP) recordings. Here we describe a new probe variant and set of techniques that enable simultaneous recording from over 200 well-isolated cortical single units in human participants during intraoperative neurosurgical procedures using silicon Neuropixels probes. We characterized a diversity of extracellular waveforms with eight separable single-unit classes, with differing firing rates, locations along the length of the electrode array, waveform spatial spread and modulation by LFP events such as inter-ictal discharges and burst suppression. Although some challenges remain in creating a turnkey recording system, high-density silicon arrays provide a path for studying human-specific cognitive processes and their dysfunction at unprecedented spatiotemporal resolution.
Collapse
Affiliation(s)
- Angelique C Paulk
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurology, Harvard Medical School, Boston, MA, USA.
| | - Yoav Kfir
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Arjun R Khanna
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Martina L Mustroph
- Department of Neurosurgery, Harvard Medical School and Brigham & Women's Hospital, Boston, MA, USA
| | - Eric M Trautmann
- Department of Neuroscience, Columbia University Medical Center, New York City, NY, USA
- Zuckerman Institute, Columbia University, New York City, NY, USA
- Grossman Center for the Statistics of Mind, Columbia University Medical Center, New York City, NY, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Columbia University, New York City, NY, USA
| | - Dan J Soper
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
| | - Sergey D Stavisky
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Department of Neurological Surgery, University of California at Davis, Davis, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | | | | | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - R Mark Richardson
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA
| | - Ziv M Williams
- Department of Neurosurgery, Harvard Medical School and Massachusetts General Hospital, Boston, MA, USA.
| | - Sydney S Cash
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurology, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
17
|
Sun X, O'Shea DJ, Golub MD, Trautmann EM, Vyas S, Ryu SI, Shenoy KV. Cortical preparatory activity indexes learned motor memories. Nature 2022; 602:274-279. [PMID: 35082444 PMCID: PMC9851374 DOI: 10.1038/s41586-021-04329-x] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2020] [Accepted: 12/09/2021] [Indexed: 01/21/2023]
Abstract
The brain's remarkable ability to learn and execute various motor behaviours harnesses the capacity of neural populations to generate a variety of activity patterns. Here we explore systematic changes in preparatory activity in motor cortex that accompany motor learning. We trained rhesus monkeys to learn an arm-reaching task1 in a curl force field that elicited new muscle forces for some, but not all, movement directions2,3. We found that in a neural subspace predictive of hand forces, changes in preparatory activity tracked the learned behavioural modifications and reassociated4 existing activity patterns with updated movements. Along a neural population dimension orthogonal to the force-predictive subspace, we discovered that preparatory activity shifted uniformly for all movement directions, including those unaltered by learning. During a washout period when the curl field was removed, preparatory activity gradually reverted in the force-predictive subspace, but the uniform shift persisted. These persistent preparatory activity patterns may retain a motor memory of the learned field5,6 and support accelerated relearning of the same curl field. When a set of distinct curl fields was learned in sequence, we observed a corresponding set of field-specific uniform shifts which separated the associated motor memories in the neural state space7-9. The precise geometry of these uniform shifts in preparatory activity could serve to index motor memories, facilitating the acquisition, retention and retrieval of a broad motor repertoire.
Collapse
Affiliation(s)
- Xulu Sun
- Department of Biology, Stanford University, Stanford, CA, USA.
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
| | - Daniel J O'Shea
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Matthew D Golub
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Eric M Trautmann
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Saurabh Vyas
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
- Department of Bioengineering, Stanford University, Stanford, CA, USA.
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.
- Department of Neurobiology, Stanford University, Stanford, CA, USA.
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.
| |
Collapse
|
18
|
Deo DR, Rezaii P, Hochberg LR, M Okamura A, Shenoy KV, Henderson JM. Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex. IEEE Trans Haptics 2021; 14:762-775. [PMID: 33844633 PMCID: PMC8745032 DOI: 10.1109/toh.2021.3072615] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.
Collapse
|
19
|
Lee EK, Balasubramanian H, Tsolias A, Anakwe SU, Medalla M, Shenoy KV, Chandrasekaran C. Non-linear dimensionality reduction on extracellular waveforms reveals cell type diversity in premotor cortex. eLife 2021; 10:e67490. [PMID: 34355695 PMCID: PMC8452311 DOI: 10.7554/elife.67490] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Accepted: 08/04/2021] [Indexed: 11/13/2022] Open
Abstract
Cortical circuits are thought to contain a large number of cell types that coordinate to produce behavior. Current in vivo methods rely on clustering of specified features of extracellular waveforms to identify putative cell types, but these capture only a small amount of variation. Here, we develop a new method (WaveMAP) that combines non-linear dimensionality reduction with graph clustering to identify putative cell types. We apply WaveMAP to extracellular waveforms recorded from dorsal premotor cortex of macaque monkeys performing a decision-making task. Using WaveMAP, we robustly establish eight waveform clusters and show that these clusters recapitulate previously identified narrow- and broad-spiking types while revealing previously unknown diversity within these subtypes. The eight clusters exhibited distinct laminar distributions, characteristic firing rate patterns, and decision-related dynamics. Such insights were weaker when using feature-based approaches. WaveMAP therefore provides a more nuanced understanding of the dynamics of cell types in cortical circuits.
Collapse
Affiliation(s)
- Eric Kenji Lee
- Psychological and Brain Sciences, Boston UniversityBostonUnited States
| | - Hymavathy Balasubramanian
- Bernstein Center for Computational Neuroscience, Bernstein Center for Computational NeuroscienceBerlinGermany
| | - Alexandra Tsolias
- Department of Anatomy and Neurobiology, Boston UniversityBostonUnited States
| | | | - Maria Medalla
- Department of Anatomy and Neurobiology, Boston UniversityBostonUnited States
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford UniversityStanfordUnited States
- Department of Bioengineering, Stanford UniversityStanfordUnited States
- Department of Neurobiology, Stanford UniversityStanfordUnited States
- Wu Tsai Neurosciences Institute, Stanford UniversityStanfordUnited States
- Bio-X Institute, Stanford UniversityStanfordUnited States
- Howard Hughes Medical Institute, Stanford UniversityStanfordUnited States
| | - Chandramouli Chandrasekaran
- Psychological and Brain Sciences, Boston UniversityBostonUnited States
- Department of Anatomy and Neurobiology, Boston UniversityBostonUnited States
- Center for Systems Neuroscience, Boston UniversityBostonUnited States
- Department of Biomedical Engineering, Boston UniversityBostonUnited States
| |
Collapse
|
20
|
Abstract
Significant experimental, computational, and theoretical work has identified rich structure within the coordinated activity of interconnected neural populations. An emerging challenge now is to uncover the nature of the associated computations, how they are implemented, and what role they play in driving behavior. We term this computation through neural population dynamics. If successful, this framework will reveal general motifs of neural population activity and quantitatively describe how neural population dynamics implement computations necessary for driving goal-directed behavior. Here, we start with a mathematical primer on dynamical systems theory and analytical tools necessary to apply this perspective to experimental data. Next, we highlight some recent discoveries resulting from successful application of dynamical systems. We focus on studies spanning motor control, timing, decision-making, and working memory. Finally, we briefly discuss promising recent lines of investigation and future directions for the computation through neural population dynamics framework.
Collapse
Affiliation(s)
- Saurabh Vyas
- Department of Bioengineering, Stanford University, Stanford, California 94305, USA; .,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA
| | - Matthew D Golub
- Department of Electrical Engineering, Stanford University, Stanford, California 94305, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, California 94305, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA.,Google AI, Google Inc., Mountain View, California 94305, USA
| | - Krishna V Shenoy
- Department of Bioengineering, Stanford University, Stanford, California 94305, USA; .,Department of Electrical Engineering, Stanford University, Stanford, California 94305, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, California 94305, USA.,Department of Neurobiology, Bio-X Institute, Neurosciences Program, and Howard Hughes Medical Institute, Stanford University, Stanford, California 94305, USA
| |
Collapse
|
21
|
Simeral JD, Hosman T, Saab J, Flesher SN, Vilela M, Franco B, Kelemen J, Brandman DM, Ciancibello JG, Rezaii PG, Eskandar EN, Rosler DM, Shenoy KV, Henderson JM, Nurmikko AV, Hochberg LR. Home Use of a Percutaneous Wireless Intracortical Brain-Computer Interface by Individuals With Tetraplegia. IEEE Trans Biomed Eng 2021; 68:2313-2325. [PMID: 33784612 PMCID: PMC8218873 DOI: 10.1109/tbme.2021.3069119] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
OBJECTIVE Individuals with neurological disease or injury such as amyotrophic lateral sclerosis, spinal cord injury or stroke may become tetraplegic, unable to speak or even locked-in. For people with these conditions, current assistive technologies are often ineffective. Brain-computer interfaces are being developed to enhance independence and restore communication in the absence of physical movement. Over the past decade, individuals with tetraplegia have achieved rapid on-screen typing and point-and-click control of tablet apps using intracortical brain-computer interfaces (iBCIs) that decode intended arm and hand movements from neural signals recorded by implanted microelectrode arrays. However, cables used to convey neural signals from the brain tether participants to amplifiers and decoding computers and require expert oversight, severely limiting when and where iBCIs could be available for use. Here, we demonstrate the first human use of a wireless broadband iBCI. METHODS Based on a prototype system previously used in pre-clinical research, we replaced the external cables of a 192-electrode iBCI with wireless transmitters and achieved high-resolution recording and decoding of broadband field potentials and spiking activity from people with paralysis. Two participants in an ongoing pilot clinical trial completed on-screen item selection tasks to assess iBCI-enabled cursor control. RESULTS Communication bitrates were equivalent between cabled and wireless configurations. Participants also used the wireless iBCI to control a standard commercial tablet computer to browse the web and use several mobile applications. Within-day comparison of cabled and wireless interfaces evaluated bit error rate, packet loss, and the recovery of spike rates and spike waveforms from the recorded neural signals. In a representative use case, the wireless system recorded intracortical signals from two arrays in one participant continuously through a 24-hour period at home. SIGNIFICANCE Wireless multi-electrode recording of broadband neural signals over extended periods introduces a valuable tool for human neuroscience research and is an important step toward practical deployment of iBCI technology for independent use by individuals with paralysis. On-demand access to high-performance iBCI technology in the home promises to enhance independence and restore communication and mobility for individuals with severe motor impairment.
Collapse
Affiliation(s)
| | - Thomas Hosman
- CfNN and the School of Engineering, Brown University
| | - Jad Saab
- CfNN and the School of Engineering, Brown University. He is now with Insight Data Science, New York City, NY
| | - Sharlene N. Flesher
- Department of Electrical Engineering, Department of Neurosurgery, and Howard Hughes Medical Institute, Stanford University. She is now with Apple Inc., Cupertino, CA
| | - Marco Vilela
- School of Engineering, Brown University. He is now with Takeda, Cambridge, MA
| | - Brian Franco
- Department of Neurology, Massachusetts General Hospital, Boston, MA. He is now with NeuroPace Inc., Mountain View, CA
| | - Jessica Kelemen
- Department of Neurology, Massachusetts General Hospital, Boston
| | - David M. Brandman
- School of Engineering, Brown University. He is now with the Department of Neurosurgery, Emory University, Atlanta, GA
| | - John G. Ciancibello
- School of Engineering, Brown University. He is now with the Center for Bioelectronic Medicine at the Feinstein Institute for Medical Research, Manhasset, NY
| | - Paymon G. Rezaii
- Department of Neurosurgery, Stanford University. He is now with the School of Medicine, Tulane University
| | - Emad N. Eskandar
- Department of Neurosurgery, Massachusetts General Hospital. He is now with the Department of Neurosurgery, Albert Einstein College of Medicine, Montefiore Medical Center, NY
| | - David M. Rosler
- CfNN and the School of Engineering and the Robert J. & Nancy D. Carney Institute for Brain Science, Brown University, and also with the Department of Neurology, Massachusetts General Hospital
| | - Krishna V. Shenoy
- Departments of Electrical Engineering, Bioengineering and Neurobiology, Wu Tsai Neurosciences Institute, and the Bio-X Institute, Stanford, and also with the Howard Hughes Medical Institute, Stanford University
| | - Jaimie M. Henderson
- Department of Neurosurgery and Wu Tsai Neurosciences Institute and the Bio-X Institute, Stanford University
| | - Arto V. Nurmikko
- School of Engineering, Department of Physics, and the Robert J. & Nancy D. Carney Institute for Brain Science, Brown University
| | - Leigh R. Hochberg
- CfNN, and the School of Engineering and the Robert J. & Nancy D. Carney Institute for Brain Science, Brown University, and the Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School
| |
Collapse
|
22
|
Trautmann EM, O'Shea DJ, Sun X, Marshel JH, Crow A, Hsueh B, Vesuna S, Cofer L, Bohner G, Allen W, Kauvar I, Quirin S, MacDougall M, Chen Y, Whitmire MP, Ramakrishnan C, Sahani M, Seidemann E, Ryu SI, Deisseroth K, Shenoy KV. Dendritic calcium signals in rhesus macaque motor cortex drive an optical brain-computer interface. Nat Commun 2021; 12:3689. [PMID: 34140486 PMCID: PMC8211867 DOI: 10.1038/s41467-021-23884-5] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 05/19/2021] [Indexed: 02/05/2023] Open
Abstract
Calcium imaging is a powerful tool for recording from large populations of neurons in vivo. Imaging in rhesus macaque motor cortex can enable the discovery of fundamental principles of motor cortical function and can inform the design of next generation brain-computer interfaces (BCIs). Surface two-photon imaging, however, cannot presently access somatic calcium signals of neurons from all layers of macaque motor cortex due to photon scattering. Here, we demonstrate an implant and imaging system capable of chronic, motion-stabilized two-photon imaging of neuronal calcium signals from macaques engaged in a motor task. By imaging apical dendrites, we achieved optical access to large populations of deep and superficial cortical neurons across dorsal premotor (PMd) and gyral primary motor (M1) cortices. Dendritic signals from individual neurons displayed tuning for different directions of arm movement. Combining several technical advances, we developed an optical BCI (oBCI) driven by these dendritic signalswhich successfully decoded movement direction online. By fusing two-photon functional imaging with CLARITY volumetric imaging, we verified that many imaged dendrites which contributed to oBCI decoding originated from layer 5 output neurons, including a putative Betz cell. This approach establishes new opportunities for studying motor control and designing BCIs via two photon imaging.
Collapse
Affiliation(s)
- Eric M Trautmann
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA.
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Daniel J O'Shea
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA.
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Xulu Sun
- Department of Biology, Stanford University, Stanford, CA, USA.
| | - James H Marshel
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Ailey Crow
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Brian Hsueh
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Sam Vesuna
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Lucas Cofer
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Gergő Bohner
- Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Will Allen
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Isaac Kauvar
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Sean Quirin
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | | | - Yuzhi Chen
- Center for Perceptual Systems, University of Texas, Austin, TX, USA
- Department of Psychology, University of Texas, Austin, TX, USA
- Department of Neuroscience, University of Texas, Austin, TX, USA
| | - Matthew P Whitmire
- Center for Perceptual Systems, University of Texas, Austin, TX, USA
- Department of Psychology, University of Texas, Austin, TX, USA
- Department of Neuroscience, University of Texas, Austin, TX, USA
| | | | - Maneesh Sahani
- Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Eyal Seidemann
- Center for Perceptual Systems, University of Texas, Austin, TX, USA
- Department of Psychology, University of Texas, Austin, TX, USA
- Department of Neuroscience, University of Texas, Austin, TX, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA, USA
| | - Karl Deisseroth
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA.
- Department of Bioengineering, Stanford University, Stanford, CA, USA.
- Department of Psychiatry and Behavioral Science, Stanford University, Stanford, CA, USA.
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.
- Wu Tsai Neuroscience Institute, Stanford University, Stanford, CA, USA.
| | - Krishna V Shenoy
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA.
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
- Department of Bioengineering, Stanford University, Stanford, CA, USA.
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.
- Wu Tsai Neuroscience Institute, Stanford University, Stanford, CA, USA.
- Department of Neurobiology, Stanford University, Stanford, CA, USA.
| |
Collapse
|
23
|
Peixoto D, Verhein JR, Kiani R, Kao JC, Nuyujukian P, Chandrasekaran C, Brown J, Fong S, Ryu SI, Shenoy KV, Newsome WT. Decoding and perturbing decision states in real time. Nature 2021; 591:604-609. [PMID: 33473215 DOI: 10.1038/s41586-020-03181-9] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2019] [Accepted: 12/09/2020] [Indexed: 01/01/2023]
Abstract
In dynamic environments, subjects often integrate multiple samples of a signal and combine them to reach a categorical judgment1. The process of deliberation can be described by a time-varying decision variable (DV), decoded from neural population activity, that predicts a subject's upcoming decision2. Within single trials, however, there are large moment-to-moment fluctuations in the DV, the behavioural significance of which is unclear. Here, using real-time, neural feedback control of stimulus duration, we show that within-trial DV fluctuations, decoded from motor cortex, are tightly linked to decision state in macaques, predicting behavioural choices substantially better than the condition-averaged DV or the visual stimulus alone. Furthermore, robust changes in DV sign have the statistical regularities expected from behavioural studies of changes of mind3. Probing the decision process on single trials with weak stimulus pulses, we find evidence for time-varying absorbing decision bounds, enabling us to distinguish between specific models of decision making.
Collapse
Affiliation(s)
- Diogo Peixoto
- Neurobiology Department, Stanford University, Stanford, CA, USA. .,Champalimaud Neuroscience Programme, Lisbon, Portugal. .,Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
| | - Jessica R Verhein
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA. .,Neurosciences Graduate Program, Stanford University, Stanford, CA, USA. .,Medical Scientist Training Program, Stanford University School of Medicine, Stanford, CA, USA.
| | - Roozbeh Kiani
- Center for Neural Science, New York University, New York, NY, USA
| | - Jonathan C Kao
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.,Electrical Engineering Department, Stanford University, Stanford, CA, USA.,Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA.,Neurosciences Program, University of California, Los Angeles, Los Angeles, CA, USA
| | - Paul Nuyujukian
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.,Electrical Engineering Department, Stanford University, Stanford, CA, USA.,Bioengineering Department, Stanford University, Stanford, CA, USA.,Neurosurgery Department, Stanford University, Stanford, CA, USA.,Bio-X Institute, Stanford University, Stanford, CA, USA
| | - Chandramouli Chandrasekaran
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.,Electrical Engineering Department, Stanford University, Stanford, CA, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.,Department of Psychological and Brain Sciences, Boston University, Boston, MA, USA.,Department of Anatomy and Neurobiology, Boston University School of Medicine, Boston, MA, USA
| | - Julian Brown
- Neurobiology Department, Stanford University, Stanford, CA, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Sania Fong
- Neurobiology Department, Stanford University, Stanford, CA, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Stephen I Ryu
- Electrical Engineering Department, Stanford University, Stanford, CA, USA.,Neurosurgery Department, Palo Alto Medical Foundation, Palo Alto, CA, USA
| | - Krishna V Shenoy
- Neurobiology Department, Stanford University, Stanford, CA, USA.,Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.,Electrical Engineering Department, Stanford University, Stanford, CA, USA.,Bioengineering Department, Stanford University, Stanford, CA, USA.,Bio-X Institute, Stanford University, Stanford, CA, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| | - William T Newsome
- Neurobiology Department, Stanford University, Stanford, CA, USA. .,Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA. .,Bio-X Institute, Stanford University, Stanford, CA, USA.
| |
Collapse
|
24
|
Tremblay S, Acker L, Afraz A, Albaugh DL, Amita H, Andrei AR, Angelucci A, Aschner A, Balan PF, Basso MA, Benvenuti G, Bohlen MO, Caiola MJ, Calcedo R, Cavanaugh J, Chen Y, Chen S, Chernov MM, Clark AM, Dai J, Debes SR, Deisseroth K, Desimone R, Dragoi V, Egger SW, Eldridge MAG, El-Nahal HG, Fabbrini F, Federer F, Fetsch CR, Fortuna MG, Friedman RM, Fujii N, Gail A, Galvan A, Ghosh S, Gieselmann MA, Gulli RA, Hikosaka O, Hosseini EA, Hu X, Hüer J, Inoue KI, Janz R, Jazayeri M, Jiang R, Ju N, Kar K, Klein C, Kohn A, Komatsu M, Maeda K, Martinez-Trujillo JC, Matsumoto M, Maunsell JHR, Mendoza-Halliday D, Monosov IE, Muers RS, Nurminen L, Ortiz-Rios M, O'Shea DJ, Palfi S, Petkov CI, Pojoga S, Rajalingham R, Ramakrishnan C, Remington ED, Revsine C, Roe AW, Sabes PN, Saunders RC, Scherberger H, Schmid MC, Schultz W, Seidemann E, Senova YS, Shadlen MN, Sheinberg DL, Siu C, Smith Y, Solomon SS, Sommer MA, Spudich JL, Stauffer WR, Takada M, Tang S, Thiele A, Treue S, Vanduffel W, Vogels R, Whitmire MP, Wichmann T, Wurtz RH, Xu H, Yazdan-Shahmorad A, Shenoy KV, DiCarlo JJ, Platt ML. An Open Resource for Non-human Primate Optogenetics. Neuron 2020; 108:1075-1090.e6. [PMID: 33080229 PMCID: PMC7962465 DOI: 10.1016/j.neuron.2020.09.027] [Citation(s) in RCA: 57] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2020] [Revised: 07/28/2020] [Accepted: 09/21/2020] [Indexed: 12/26/2022]
Abstract
Optogenetics has revolutionized neuroscience in small laboratory animals, but its effect on animal models more closely related to humans, such as non-human primates (NHPs), has been mixed. To make evidence-based decisions in primate optogenetics, the scientific community would benefit from a centralized database listing all attempts, successful and unsuccessful, of using optogenetics in the primate brain. We contacted members of the community to ask for their contributions to an open science initiative. As of this writing, 45 laboratories around the world contributed more than 1,000 injection experiments, including precise details regarding their methods and outcomes. Of those entries, more than half had not been published. The resource is free for everyone to consult and contribute to on the Open Science Framework website. Here we review some of the insights from this initial release of the database and discuss methodological considerations to improve the success of optogenetic experiments in NHPs.
Collapse
Affiliation(s)
- Sébastien Tremblay
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104, USA.
| | - Leah Acker
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Arash Afraz
- National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, USA
| | - Daniel L Albaugh
- Yerkes National Primate Research Center, Morris K. Udall Center of Excellence for Parkinson's Disease, Department of Neurology, Emory University, GA 30329, USA
| | - Hidetoshi Amita
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Ariana R Andrei
- Department of Neurobiology and Anatomy, McGovern Medical School, University of Texas-Houston, Houston, TX 77030, USA
| | - Alessandra Angelucci
- Department of Ophthalmology, Moran Eye Institute, University of Utah, Salt Lake City, UT 84132, USA
| | - Amir Aschner
- Dominik P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Puiu F Balan
- Laboratory of Neuro- and Psychophysiology, KU Leuven, 3000 Leuven, Belgium
| | - Michele A Basso
- Departments of Psychiatry and Biobehavioral Sciences and Neurobiology, UCLA, Los Angeles, CA 90095, USA
| | - Giacomo Benvenuti
- Departments of Psychology and Neuroscience, Center for Perceptual Systems, University of Texas, Austin, TX 78712, USA
| | - Martin O Bohlen
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Michael J Caiola
- Yerkes National Primate Research Center, Morris K. Udall Center of Excellence for Parkinson's Disease, Department of Neurology, Emory University, GA 30329, USA
| | - Roberto Calcedo
- Gene Therapy Program, Department of Medicine, University of Pennsylvania, Philadelphia, PA 19014, USA
| | - James Cavanaugh
- Laboratory of Sensorimotor Research, National Eye Institute, NIH, Bethesda, MD 20982, USA
| | - Yuzhi Chen
- Departments of Psychology and Neuroscience, Center for Perceptual Systems, University of Texas, Austin, TX 78712, USA
| | - Spencer Chen
- Departments of Psychology and Neuroscience, Center for Perceptual Systems, University of Texas, Austin, TX 78712, USA
| | - Mykyta M Chernov
- Division of Neuroscience, Oregon National Primate Resource Center, Oregon Health and Science University, Beaverton, OR 97006, USA
| | - Andrew M Clark
- Department of Ophthalmology, Moran Eye Institute, University of Utah, Salt Lake City, UT 84132, USA
| | - Ji Dai
- CAS Key Laboratory of Brain Connectome and Manipulation, The Brain Cognition and Brain Disease Institute, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen-Hong Kong Institute of Brain Science, Shenzhen 518055, China
| | - Samantha R Debes
- Department of Neurobiology and Anatomy, McGovern Medical School, University of Texas-Houston, Houston, TX 77030, USA
| | - Karl Deisseroth
- Neuroscience Program, Departments of Bioengineering, Psychiatry, and Behavioral Science, Wu Tsai Neurosciences Institute, Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Robert Desimone
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Valentin Dragoi
- Department of Neurobiology and Anatomy, McGovern Medical School, University of Texas-Houston, Houston, TX 77030, USA
| | - Seth W Egger
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Mark A G Eldridge
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Department of Health and Human Services, Bethesda, MD 20892, USA
| | - Hala G El-Nahal
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Francesco Fabbrini
- Laboratory of Neuro- and Psychophysiology, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium
| | - Frederick Federer
- Department of Ophthalmology, Moran Eye Institute, University of Utah, Salt Lake City, UT 84132, USA
| | - Christopher R Fetsch
- The Solomon H. Snyder Department of Neuroscience & Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Michal G Fortuna
- German Primate Center - Leibniz Institute for Primate Research, 37077 Göttingen, Germany
| | - Robert M Friedman
- Division of Neuroscience, Oregon National Primate Resource Center, Oregon Health and Science University, Beaverton, OR 97006, USA
| | - Naotaka Fujii
- Laboratory for Adaptive Intelligence, RIKEN Brain Science Institute, Wako, Saitama 351-0198, Japan
| | - Alexander Gail
- German Primate Center - Leibniz Institute for Primate Research, 37077 Göttingen, Germany; Bernstein Center for Computational Neuroscience, Göttingen, Germany; Faculty for Biology and Psychology, University of Göttingen, Göttingen, Germany; Leibniz Science Campus Primate Cognition, Göttingen, Germany
| | - Adriana Galvan
- Yerkes National Primate Research Center, Morris K. Udall Center of Excellence for Parkinson's Disease, Department of Neurology, Emory University, GA 30329, USA
| | - Supriya Ghosh
- Department of Neurobiology and Grossman Institute for Neuroscience, Quantitative Biology and Human Behavior, University of Chicago, Chicago, IL 60637, USA
| | - Marc Alwin Gieselmann
- Biosciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle NE2 4HH, UK
| | - Roberto A Gulli
- Zuckerman Institute, Columbia University, New York, NY 10027, USA; Center for Theoretical Neuroscience, Columbia University, New York, NY 10027, USA
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Eghbal A Hosseini
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Xing Hu
- Yerkes National Primate Research Center, Morris K. Udall Center of Excellence for Parkinson's Disease, Department of Neurology, Emory University, GA 30329, USA
| | - Janina Hüer
- German Primate Center - Leibniz Institute for Primate Research, 37077 Göttingen, Germany
| | - Ken-Ichi Inoue
- Systems Neuroscience Section, Primate Research Institute, Kyoto University, Inuyama, Aichi 484-8506, Japan; PRESTO, Japan Science and Technology Agency, Kawaguchi, Saitama 332-0012, Japan
| | - Roger Janz
- Department of Neurobiology and Anatomy, McGovern Medical School, University of Texas-Houston, Houston, TX 77030, USA
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Rundong Jiang
- School of Life Sciences, Peking University, Beijing 100871, China
| | - Niansheng Ju
- School of Life Sciences, Peking University, Beijing 100871, China
| | - Kohitij Kar
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Carsten Klein
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany
| | - Adam Kohn
- Dominik P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA; Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY 10461, USA; Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Misako Komatsu
- Laboratory for Adaptive Intelligence, RIKEN Brain Science Institute, Wako, Saitama 351-0198, Japan
| | - Kazutaka Maeda
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Julio C Martinez-Trujillo
- Robarts Research Institute, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada; Brain and Mind Institute, University of Western Ontario, London, ON, Canada
| | - Masayuki Matsumoto
- Division of Biomedical Science, Faculty of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan; Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - John H R Maunsell
- Department of Neurobiology and Grossman Institute for Neuroscience, Quantitative Biology and Human Behavior, University of Chicago, Chicago, IL 60637, USA
| | - Diego Mendoza-Halliday
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Ilya E Monosov
- Department of Neuroscience, Biomedical Engineering, Electrical Engineering, Neurosurgery and Pain Center, Washington University, St. Louis, MO 63110, USA
| | - Ross S Muers
- Biosciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle NE2 4HH, UK
| | - Lauri Nurminen
- Department of Ophthalmology, Moran Eye Institute, University of Utah, Salt Lake City, UT 84132, USA
| | - Michael Ortiz-Rios
- German Primate Center - Leibniz Institute for Primate Research, 37077 Göttingen, Germany; Leibniz Science Campus Primate Cognition, Göttingen, Germany; Biosciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle NE2 4HH, UK
| | - Daniel J O'Shea
- Department of Electrical Engineering, Wu Tsai Neurosciences Institute, and Bio-X Institute, and Neuroscience Graduate Program, Stanford University, Stanford, CA 94305, USA
| | - Stéphane Palfi
- Department of Neurosurgery, Assistance Publique-Hopitaux de Paris (APHP), U955 INSERM IMRB eq.15, University of Paris 12 UPEC, Faculté de Médecine, Créteil 94010, France
| | - Christopher I Petkov
- Biosciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle NE2 4HH, UK
| | - Sorin Pojoga
- Department of Neurobiology and Anatomy, McGovern Medical School, University of Texas-Houston, Houston, TX 77030, USA
| | - Rishi Rajalingham
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Charu Ramakrishnan
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA
| | - Evan D Remington
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Cambria Revsine
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104, USA; Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20814, USA
| | - Anna W Roe
- Division of Neuroscience, Oregon National Primate Resource Center, Oregon Health and Science University, Beaverton, OR 97006, USA; Interdisciplinary Institute of Neuroscience and Technology, Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou 310029, China; Key Laboratory of Biomedical Engineering of the Ministry of Education, Zhejiang University, Hangzhou 310029, China
| | - Philip N Sabes
- Center for Integrative Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Richard C Saunders
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Department of Health and Human Services, Bethesda, MD 20892, USA
| | - Hansjörg Scherberger
- German Primate Center - Leibniz Institute for Primate Research, 37077 Göttingen, Germany; Bernstein Center for Computational Neuroscience, Göttingen, Germany; Faculty for Biology and Psychology, University of Göttingen, Göttingen, Germany; Leibniz Science Campus Primate Cognition, Göttingen, Germany
| | - Michael C Schmid
- Biosciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle NE2 4HH, UK; Department of Neurosciences and Movement Sciences, Faculty of Science and Medicine, University of Fribourg, 1700 Fribourg, Switzerland
| | - Wolfram Schultz
- Department of Physiology, Development of Neuroscience, University of Cambridge, Cambridge CB3 0LT, UK
| | - Eyal Seidemann
- Departments of Psychology and Neuroscience, Center for Perceptual Systems, University of Texas, Austin, TX 78712, USA
| | - Yann-Suhan Senova
- Department of Neurosurgery, Assistance Publique-Hopitaux de Paris (APHP), U955 INSERM IMRB eq.15, University of Paris 12 UPEC, Faculté de Médecine, Créteil 94010, France
| | - Michael N Shadlen
- Department of Neuroscience, Mortimer B. Zuckerman Mind Brain Behavior Institute, The Kavli Institute for Brain Science & Howard Hughes Medical Institute, Columbia University, NY 10027, USA
| | - David L Sheinberg
- Department of Neuroscience and Carney Institute for Brain Science, Brown University, Providence, RI 02912, USA
| | - Caitlin Siu
- Department of Ophthalmology, Moran Eye Institute, University of Utah, Salt Lake City, UT 84132, USA
| | - Yoland Smith
- Yerkes National Primate Research Center, Morris K. Udall Center of Excellence for Parkinson's Disease, Department of Neurology, Emory University, GA 30329, USA
| | - Selina S Solomon
- Dominik P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Marc A Sommer
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - John L Spudich
- Department of Biochemistry and Molecular Biology, McGovern Medical School, The University of Texas-Houston, Houston, TX 77030, USA
| | - William R Stauffer
- Systems Neuroscience Institute, Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA 15260, USA
| | - Masahiko Takada
- Systems Neuroscience Section, Primate Research Institute, Kyoto University, Inuyama, Aichi 484-8506, Japan
| | - Shiming Tang
- School of Life Sciences, Peking University, Beijing 100871, China
| | - Alexander Thiele
- Biosciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle NE2 4HH, UK
| | - Stefan Treue
- German Primate Center - Leibniz Institute for Primate Research, 37077 Göttingen, Germany; Bernstein Center for Computational Neuroscience, Göttingen, Germany; Faculty for Biology and Psychology, University of Göttingen, Göttingen, Germany; Leibniz Science Campus Primate Cognition, Göttingen, Germany
| | - Wim Vanduffel
- Laboratory of Neuro- and Psychophysiology, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium; MGH Martinos Center, Charlestown, MA 02129, USA; Harvard Medical School, Boston, MA 02144, USA
| | - Rufin Vogels
- Laboratory of Neuro- and Psychophysiology, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium
| | - Matthew P Whitmire
- Departments of Psychology and Neuroscience, Center for Perceptual Systems, University of Texas, Austin, TX 78712, USA
| | - Thomas Wichmann
- Yerkes National Primate Research Center, Morris K. Udall Center of Excellence for Parkinson's Disease, Department of Neurology, Emory University, GA 30329, USA
| | - Robert H Wurtz
- Laboratory of Sensorimotor Research, National Eye Institute, NIH, Bethesda, MD 20982, USA
| | - Haoran Xu
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Azadeh Yazdan-Shahmorad
- Center for Integrative Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA; Departments of Bioengineering and Electrical and Computer Engineering, Washington National Primate Research Center, University of Washington, Seattle, WA 98105, USA
| | - Krishna V Shenoy
- Departments of Electrical Engineering, Bioengineering, and Neurobiology, Wu Tsai Neurosciences Institute and Bio-X Institute, Neuroscience Graduate Program, and Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - James J DiCarlo
- McGovern Institute for Brain Research, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Michael L Platt
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Marketing, Wharton School, University of Pennsylvania, Philadelphia, PA 19104, USA
| |
Collapse
|
25
|
Al Borno M, Vyas S, Shenoy KV, Delp SL. High-fidelity musculoskeletal modeling reveals that motor planning variability contributes to the speed-accuracy tradeoff. eLife 2020; 9:57021. [PMID: 33325369 PMCID: PMC7787661 DOI: 10.7554/elife.57021] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Accepted: 12/15/2020] [Indexed: 11/16/2022] Open
Abstract
A long-standing challenge in motor neuroscience is to understand the relationship between movement speed and accuracy, known as the speed-accuracy tradeoff. Here, we introduce a biomechanically realistic computational model of three-dimensional upper extremity movements that reproduces well-known features of reaching movements. This model revealed that the speed-accuracy tradeoff, as described by Fitts’ law, emerges even without the presence of motor noise, which is commonly believed to underlie the speed-accuracy tradeoff. Next, we analyzed motor cortical neural activity from monkeys reaching to targets of different sizes. We found that the contribution of preparatory neural activity to movement duration (MD) variability is greater for smaller targets than larger targets, and that movements to smaller targets exhibit less variability in population-level preparatory activity, but greater MD variability. These results propose a new theory underlying the speed-accuracy tradeoff: Fitts’ law emerges from greater task demands constraining the optimization landscape in a fashion that reduces the number of ‘good’ control solutions (i.e., faster reaches). Thus, contrary to current beliefs, the speed-accuracy tradeoff could be a consequence of motor planning variability and not exclusively signal-dependent noise.
Collapse
Affiliation(s)
- Mazen Al Borno
- Department of Bioengineering, Stanford University, Stanford, United States.,Department of Computer Science and Engineering, University of Colorado Denver, Denver, United States
| | - Saurabh Vyas
- Department of Bioengineering, Stanford University, Stanford, United States
| | - Krishna V Shenoy
- Department of Bioengineering, Stanford University, Stanford, United States.,Neurosciences Program, Stanford University, Stanford, United States.,Department of Electrical Engineering, Stanford University, Stanford, United States.,Wu Tsai Neuroscience Institute, Stanford University, Stanford, United States.,Department of Neurobiology, Stanford University, Stanford, United States.,Howard Hughes Medical Institute, Stanford University, Stanford, United States
| | - Scott L Delp
- Department of Bioengineering, Stanford University, Stanford, United States.,Department of Mechanical Engineering, Stanford University, Stanford, United States
| |
Collapse
|
26
|
Wilson GH, Stavisky SD, Willett FR, Avansino DT, Kelemen JN, Hochberg LR, Henderson JM, Druckmann S, Shenoy KV. Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus. J Neural Eng 2020; 17:066007. [PMID: 33236720 PMCID: PMC8293867 DOI: 10.1088/1741-2552/abbfef] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
OBJECTIVE To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the 'hand knob' area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. APPROACH Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the 'Brain-to-Speech' pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times. MAIN RESULTS A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio. SIGNIFICANCE The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.
Collapse
Affiliation(s)
- Guy H Wilson
- Neurosciences Graduate Program, Stanford University, Stanford, CA, United States of America
| | - Sergey D Stavisky
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, United States of America
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
| | - Francis R Willett
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, United States of America
| | - Donald T Avansino
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
| | - Jessica N Kelemen
- Department of Neurology, Harvard Medical School, Boston, MA, United States of America
| | - Leigh R Hochberg
- Department of Neurology, Harvard Medical School, Boston, MA, United States of America
- Center for Neurotechnology and Neurorecovery, Dept. of Neurology, Massachusetts General Hospital, Boston, MA, United States of America
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, United States of America
- Carney Institute for Brain Science and School of Engineering, Brown University, Providence, RI, United States of America
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, United States of America
| | - Shaul Druckmann
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, United States of America
- Department of Neurobiology, Stanford University, Stanford, CA, United States of America
| | - Krishna V Shenoy
- Wu Tsai Neurosciences Institute and Bio-X Institute, Stanford University, Stanford, CA, United States of America
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, United States of America
- Department of Neurobiology, Stanford University, Stanford, CA, United States of America
- Department of Bioengineering, Stanford University, Stanford, CA, United States of America
| |
Collapse
|
27
|
Even-Chen N, Muratore DG, Stavisky SD, Hochberg LR, Henderson JM, Murmann B, Shenoy KV. Power-saving design opportunities for wireless intracortical brain-computer interfaces. Nat Biomed Eng 2020; 4:984-996. [PMID: 32747834 PMCID: PMC8286886 DOI: 10.1038/s41551-020-0595-9] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2018] [Accepted: 06/30/2020] [Indexed: 12/17/2022]
Abstract
The efficacy of wireless intracortical brain-computer interfaces (iBCIs) is limited in part by the number of recording channels, which is constrained by the power budget of the implantable system. Designing wireless iBCIs that provide the high-quality recordings of today's wired neural interfaces may lead to inadvertent over-design at the expense of power consumption and scalability. Here, we report analyses of neural signals collected from experimental iBCI measurements in rhesus macaques and from a clinical-trial participant with implanted 96-channel Utah multielectrode arrays to understand the trade-offs between signal quality and decoder performance. Moreover, we propose an efficient hardware design for clinically viable iBCIs, and suggest that the circuit design parameters of current recording iBCIs can be relaxed considerably without loss of performance. The proposed design may allow for an order-of-magnitude power savings and lead to clinically viable iBCIs with a higher channel count.
Collapse
Affiliation(s)
- Nir Even-Chen
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Dante G Muratore
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Sergey D Stavisky
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- Department of Veterans Affairs Medical Center, Center for Neurorestoration and Neurotechnology, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jaimie M Henderson
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Boris Murmann
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- The Bio-X Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|
28
|
Jiang X, Saggar H, Ryu SI, Shenoy KV, Kao JC. Structure in Neural Activity during Observed and Executed Movements Is Shared at the Neural Population Level, Not in Single Neurons. Cell Rep 2020; 32:108148. [PMID: 32905762 DOI: 10.1016/j.celrep.2020.108148] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022] Open
|
29
|
Jiang X, Saggar H, Ryu SI, Shenoy KV, Kao JC. Structure in Neural Activity during Observed and Executed Movements Is Shared at the Neural Population Level, Not in Single Neurons. Cell Rep 2020; 32:108006. [DOI: 10.1016/j.celrep.2020.108006] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2020] [Revised: 05/24/2020] [Accepted: 07/16/2020] [Indexed: 12/30/2022] Open
|
30
|
Nason SR, Vaskov AK, Willsey MS, Welle EJ, An H, Vu PP, Bullard AJ, Nu CS, Kao JC, Shenoy KV, Jang T, Kim HS, Blaauw D, Patil PG, Chestek CA. A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces. Nat Biomed Eng 2020; 4:973-983. [PMID: 32719512 DOI: 10.1038/s41551-020-0591-0] [Citation(s) in RCA: 48] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Accepted: 06/24/2020] [Indexed: 12/18/2022]
Abstract
The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.
Collapse
Affiliation(s)
- Samuel R Nason
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Alex K Vaskov
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI, USA
| | - Matthew S Willsey
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.,Department of Neurosurgery, University of Michigan Medical School, Ann Arbor, MI, USA
| | - Elissa J Welle
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Hyochan An
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - Philip P Vu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Autumn J Bullard
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Chrono S Nu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Jonathan C Kao
- Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA.,Neurosciences Program, University of California, Los Angeles, Los Angeles, CA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Department of Bioengineering, Stanford University, Stanford, CA, USA.,Department of Neurobiology, Stanford University, Stanford, CA, USA.,The Bio-X Program, Stanford University, Stanford, CA, USA.,Wu Tsai Neuroscience Institute, Stanford University, Stanford, CA, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| | - Taekwang Jang
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA.,Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
| | - Hun-Seok Kim
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - David Blaauw
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - Parag G Patil
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.,Department of Neurosurgery, University of Michigan Medical School, Ann Arbor, MI, USA.,Department of Neurology, University of Michigan Medical School, Ann Arbor, MI, USA.,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA
| | - Cynthia A Chestek
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA. .,Robotics Graduate Program, University of Michigan, Ann Arbor, MI, USA. .,Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA. .,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
31
|
Vyas S, O'Shea DJ, Ryu SI, Shenoy KV. Causal Role of Motor Preparation during Error-Driven Learning. Neuron 2020; 106:329-339.e4. [PMID: 32053768 PMCID: PMC7185427 DOI: 10.1016/j.neuron.2020.01.019] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2019] [Revised: 11/12/2019] [Accepted: 01/16/2020] [Indexed: 11/28/2022]
Abstract
Current theories suggest that an error-driven learning process updates trial-by-trial to facilitate motor adaptation. How this process interacts with motor cortical preparatory activity-which current models suggest plays a critical role in movement initiation-remains unknown. Here, we evaluated the role of motor preparation during visuomotor adaptation. We found that preparation time was inversely correlated to variance of errors on current trials and mean error on subsequent trials. We also found causal evidence that intracortical microstimulation during motor preparation was sufficient to disrupt learning. Surprisingly, stimulation did not affect current trials, but instead disrupted the update computation of a learning process, thereby affecting subsequent trials. This is consistent with a Bayesian estimation framework where the motor system reduces its learning rate by virtue of lowering error sensitivity when faced with uncertainty. This interaction between motor preparation and the error-driven learning system may facilitate new probes into mechanisms underlying trial-by-trial adaptation.
Collapse
Affiliation(s)
- Saurabh Vyas
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA.
| | - Daniel J O'Shea
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA; Neurosciences Graduate Program, Stanford University, Stanford, CA 94305, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA; Palo Alto Medical Foundation, Palo Alto, CA 94301, USA
| | - Krishna V Shenoy
- Department of Bioengineering, Stanford University, Stanford, CA 94305, USA; Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA; Department of Neurobiology, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA 94305, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| |
Collapse
|
32
|
Stavisky SD, Willett FR, Avansino DT, Hochberg LR, Shenoy KV, Henderson JM. Speech-related dorsal motor cortex activity does not interfere with iBCI cursor control. J Neural Eng 2020; 17:016049. [PMID: 32023225 PMCID: PMC8288044 DOI: 10.1088/1741-2552/ab5b72] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
OBJECTIVE Speech-related neural modulation was recently reported in 'arm/hand' area of human dorsal motor cortex that is used as a signal source for intracortical brain-computer interfaces (iBCIs). This raises the concern that speech-related modulation might deleteriously affect the decoding of arm movement intentions, for instance by affecting velocity command outputs. This study sought to clarify whether or not speaking would interfere with ongoing iBCI use. APPROACH A participant in the BrainGate2 iBCI clinical trial used an iBCI to control a computer cursor; spoke short words in a stand-alone speech task; and spoke short words during ongoing iBCI use. We examined neural activity in all three behaviors and compared iBCI performance with and without concurrent speech. MAIN RESULTS Dorsal motor cortex firing rates modulated strongly during stand-alone speech, but this activity was largely attenuated when speaking occurred during iBCI cursor control using attempted arm movements. 'Decoder-potent' projections of the attenuated speech-related neural activity were small, explaining why cursor task performance was similar between iBCI use with and without concurrent speaking. SIGNIFICANCE These findings indicate that speaking does not directly interfere with iBCIs that decode attempted arm movements. This suggests that patients who are able to speak will be able to use motor cortical-driven computer interfaces or prostheses without needing to forgo speaking while using these devices.
Collapse
Affiliation(s)
- Sergey D. Stavisky
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Francis R. Willett
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | | | - Leigh R Hochberg
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
- Center for Neurotechnology and Neurorecovery, Dept. of Neurology, Massachusetts General Hospital, Boston, MA, USA
| | - Krishna V. Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Jaimie M. Henderson
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|
33
|
Stavisky SD, Willett FR, Wilson GH, Murphy BA, Rezaii P, Avansino DT, Memberg WD, Miller JP, Kirsch RF, Hochberg LR, Ajiboye AB, Druckmann S, Shenoy KV, Henderson JM. Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis. eLife 2019; 8:e46015. [PMID: 31820736 PMCID: PMC6954053 DOI: 10.7554/elife.46015] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2019] [Accepted: 11/14/2019] [Indexed: 01/20/2023] Open
Abstract
Speaking is a sensorimotor behavior whose neural basis is difficult to study with single neuron resolution due to the scarcity of human intracortical measurements. We used electrode arrays to record from the motor cortex 'hand knob' in two people with tetraplegia, an area not previously implicated in speech. Neurons modulated during speaking and during non-speaking movements of the tongue, lips, and jaw. This challenges whether the conventional model of a 'motor homunculus' division by major body regions extends to the single-neuron scale. Spoken words and syllables could be decoded from single trials, demonstrating the potential of intracortical recordings for brain-computer interfaces to restore speech. Two neural population dynamics features previously reported for arm movements were also present during speaking: a component that was mostly invariant across initiating different words, followed by rotatory dynamics during speaking. This suggests that common neural dynamical motifs may underlie movement of arm and speech articulators.
Collapse
Affiliation(s)
- Sergey D Stavisky
- Department of NeurosurgeryStanford UniversityStanfordUnited States
- Department of Electrical EngineeringStanford UniversityStanfordUnited States
| | - Francis R Willett
- Department of NeurosurgeryStanford UniversityStanfordUnited States
- Department of Electrical EngineeringStanford UniversityStanfordUnited States
| | - Guy H Wilson
- Neurosciences ProgramStanford UniversityStanfordUnited States
| | - Brian A Murphy
- Department of Biomedical EngineeringCase Western Reserve UniversityClevelandUnited States
- FES Center, Rehab R&D ServiceLouis Stokes Cleveland Department of Veterans Affairs Medical CenterClevelandUnited States
| | - Paymon Rezaii
- Department of NeurosurgeryStanford UniversityStanfordUnited States
| | | | - William D Memberg
- Department of Biomedical EngineeringCase Western Reserve UniversityClevelandUnited States
- FES Center, Rehab R&D ServiceLouis Stokes Cleveland Department of Veterans Affairs Medical CenterClevelandUnited States
| | - Jonathan P Miller
- FES Center, Rehab R&D ServiceLouis Stokes Cleveland Department of Veterans Affairs Medical CenterClevelandUnited States
- Department of NeurosurgeryUniversity Hospitals Cleveland Medical CenterClevelandUnited States
| | - Robert F Kirsch
- Department of Biomedical EngineeringCase Western Reserve UniversityClevelandUnited States
- FES Center, Rehab R&D ServiceLouis Stokes Cleveland Department of Veterans Affairs Medical CenterClevelandUnited States
| | - Leigh R Hochberg
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D ServiceProvidence VA Medical CenterProvidenceUnited States
- Center for Neurotechnology and Neurorecovery, Department of NeurologyMassachusetts General Hospital, Harvard Medical SchoolBostonUnited States
- School of Engineering and Robert J. & Nandy D. Carney Institute for Brain ScienceBrown UniversityProvidenceUnited States
| | - A Bolu Ajiboye
- Department of Biomedical EngineeringCase Western Reserve UniversityClevelandUnited States
- FES Center, Rehab R&D ServiceLouis Stokes Cleveland Department of Veterans Affairs Medical CenterClevelandUnited States
| | - Shaul Druckmann
- Department of NeurobiologyStanford UniversityStanfordUnited States
| | - Krishna V Shenoy
- Department of Electrical EngineeringStanford UniversityStanfordUnited States
- Department of NeurobiologyStanford UniversityStanfordUnited States
- Department of BioengineeringStanford UniversityStanfordUnited States
- Howard Hughes Medical Institute, Stanford UniversityStanfordUnited States
- Wu Tsai Neurosciences InstituteStanford UniversityStanfordUnited States
- Bio-X ProgramStanford UniversityStanfordUnited States
| | - Jaimie M Henderson
- Department of NeurosurgeryStanford UniversityStanfordUnited States
- Wu Tsai Neurosciences InstituteStanford UniversityStanfordUnited States
- Bio-X ProgramStanford UniversityStanfordUnited States
| |
Collapse
|
34
|
Williams AH, Poole B, Maheswaranathan N, Dhawale AK, Fisher T, Wilson CD, Brann DH, Trautmann EM, Ryu S, Shusterman R, Rinberg D, Ölveczky BP, Shenoy KV, Ganguli S. Discovering Precise Temporal Patterns in Large-Scale Neural Recordings through Robust and Interpretable Time Warping. Neuron 2019; 105:246-259.e8. [PMID: 31786013 DOI: 10.1016/j.neuron.2019.10.020] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2019] [Revised: 09/17/2019] [Accepted: 10/10/2019] [Indexed: 12/22/2022]
Abstract
Though the temporal precision of neural computation has been studied intensively, a data-driven determination of this precision remains a fundamental challenge. Reproducible spike patterns may be obscured on single trials by uncontrolled temporal variability in behavior and cognition and may not be time locked to measurable signatures in behavior or local field potentials (LFP). To overcome these challenges, we describe a general-purpose time warping framework that reveals precise spike-time patterns in an unsupervised manner, even when these patterns are decoupled from behavior or are temporally stretched across single trials. We demonstrate this method across diverse systems: cued reaching in nonhuman primates, motor sequence production in rats, and olfaction in mice. This approach flexibly uncovers diverse dynamical firing patterns, including pulsatile responses to behavioral events, LFP-aligned oscillatory spiking, and even unanticipated patterns, such as 7 Hz oscillations in rat motor cortex that are not time locked to measured behaviors or LFP.
Collapse
Affiliation(s)
- Alex H Williams
- Neuroscience Program, Stanford University, Stanford, CA 94305, USA.
| | - Ben Poole
- Google Brain, Google Inc., Mountain View, CA 94043, USA
| | | | - Ashesh K Dhawale
- Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Tucker Fisher
- Neuroscience Program, Stanford University, Stanford, CA 94305, USA
| | - Christopher D Wilson
- Neuroscience Institute, New York University School of Medicine, New York, NY 10016, USA
| | - David H Brann
- Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Eric M Trautmann
- Neuroscience Program, Stanford University, Stanford, CA 94305, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Stephen Ryu
- Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA 94301, USA
| | - Roman Shusterman
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403, USA
| | - Dmitry Rinberg
- Neuroscience Institute, New York University School of Medicine, New York, NY 10016, USA; Center for Neural Science, New York University, New York, NY 10016, USA
| | - Bence P Ölveczky
- Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Krishna V Shenoy
- Neurobiology Department, Stanford University, Stanford, CA 94305, USA; Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Bioengineering Department, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Wu Tsai Stanford Neurosciences Institute, Stanford University, Stanford, CA 94305, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Surya Ganguli
- Applied Physics Department, Stanford University, Stanford, CA 94305, USA; Neurobiology Department, Stanford University, Stanford, CA 94305, USA; Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Wu Tsai Stanford Neurosciences Institute, Stanford University, Stanford, CA 94305, USA; Google Brain, Google Inc., Mountain View, CA 94043, USA.
| |
Collapse
|
35
|
Brandman DM, Hosman T, Saab J, Burkhart MC, Shanahan BE, Ciancibello JG, Sarma AA, Milstein DJ, Vargas-Irwin CE, Franco B, Kelemen J, Blabe C, Murphy BA, Young DR, Willett FR, Pandarinath C, Stavisky SD, Kirsch RF, Walter BL, Bolu Ajiboye A, Cash SS, Eskandar EN, Miller JP, Sweet JA, Shenoy KV, Henderson JM, Jarosiewicz B, Harrison MT, Simeral JD, Hochberg LR. Rapid calibration of an intracortical brain-computer interface for people with tetraplegia. J Neural Eng 2019; 15:026007. [PMID: 29363625 DOI: 10.1088/1741-2552/aa9ee7] [Citation(s) in RCA: 63] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) can enable individuals with tetraplegia to communicate and control external devices. Though much progress has been made in improving the speed and robustness of neural control provided by intracortical BCIs, little research has been devoted to minimizing the amount of time spent on decoder calibration. APPROACH We investigated the amount of time users needed to calibrate decoders and achieve performance saturation using two markedly different decoding algorithms: the steady-state Kalman filter, and a novel technique using Gaussian process regression (GP-DKF). MAIN RESULTS Three people with tetraplegia gained rapid closed-loop neural cursor control and peak, plateaued decoder performance within 3 min of initializing calibration. We also show that a BCI-naïve user (T5) was able to rapidly attain closed-loop neural cursor control with the GP-DKF using self-selected movement imagery on his first-ever day of closed-loop BCI use, acquiring a target 37 s after initiating calibration. SIGNIFICANCE These results demonstrate the potential for an intracortical BCI to be used immediately after deployment by people with paralysis, without the need for user learning or extensive system calibration.
Collapse
Affiliation(s)
- David M Brandman
- Neuroscience Graduate Program, Brown University, Providence, RI, United States of America. Department of Neuroscience, Brown University, Providence, RI, United States of America. Brown Institute for Brain Science, Brown University, Providence, RI, United States of America. Department of Surgery (Neurosurgery), Dalhousie University, Halifax, NS, Canada
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
36
|
Stavisky SD, Rezaii P, Willett FR, Hochberg LR, Shenoy KV, Henderson JM. Decoding Speech from Intracortical Multielectrode Arrays in Dorsal "Arm/Hand Areas" of Human Motor Cortex. Annu Int Conf IEEE Eng Med Biol Soc 2019; 2018:93-97. [PMID: 30440349 DOI: 10.1109/embc.2018.8512199] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Neural prostheses are being developed to restore speech to people with neurological injury or disease. A key design consideration is where and how to access neural correlates of intended speech. Most prior work has examined cortical field potentials at a coarse resolution using electroencephalography (EEG) or medium resolution using electrocorticography (ECoG). The few studies of speech with single-neuron resolution recorded from ventral areas known to be part of the speech network. Here, we recorded from two 96- electrode arrays chronically implanted into the 'hand knob' area of motor cortex while a person with tetraplegia spoke. Despite being located in an area previously demonstrated to modulate during attempted arm movements, many electrodes' neuronal firing rates responded to speech production. In offline analyses, we could classify which of 9 phonemes (plus silence) was spoken with 81% single-trial accuracy using a combination of spike rate and local field potential (LFP) power. This suggests that high-fidelity speech prostheses may be possible using large-scale intracortical recordings in motor cortical areas involved in controlling speech articulators.
Collapse
|
37
|
Ames KC, Ryu SI, Shenoy KV. Simultaneous motor preparation and execution in a last-moment reach correction task. Nat Commun 2019; 10:2718. [PMID: 31221968 PMCID: PMC6586876 DOI: 10.1038/s41467-019-10772-2] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2017] [Accepted: 05/31/2019] [Indexed: 11/28/2022] Open
Abstract
Motor preparation typically precedes movement and is thought to determine properties of upcoming movements. However, preparation has mostly been studied in point-to-point delayed reaching tasks. Here, we ask whether preparation is engaged during mid-reach modifications. Monkeys reach to targets that occasionally jump locations prior to movement onset, requiring a mid-reach correction. In motor cortex and dorsal premotor cortex, we find that the neural activity that signals when to reach predicts monkeys’ jump responses on a trial-by-trial basis. We further identify neural patterns that signal where to reach, either during motor preparation or during motor execution. After a target jump, neural activity responds in both preparatory and movement-related dimensions, even though error in preparatory dimensions can be small at that time. This suggests that the same preparatory process used in delayed reaching is also involved in reach correction. Furthermore, it indicates that motor preparation and execution can be performed simultaneously. Motor preparation processes guide movement. Here, by recording neural activity in monkeys reaching toward targets that can change location, the authors provide evidence that changing a prepared movement midway through completion reengages motor preparation.
Collapse
Affiliation(s)
- K Cora Ames
- Neurosciences Program, School of Medicine, Stanford University, Stanford, CA, 94305, USA. .,Department of Neuroscience, Columbia University Medical Center, New York, NY, 10032, USA.
| | - Stephen I Ryu
- Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA, 94301, USA.,Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA.,Wu Tsai Neuroscience Institute, Stanford University, Stanford, CA, 94305, USA
| | - Krishna V Shenoy
- Neurosciences Program, School of Medicine, Stanford University, Stanford, CA, 94305, USA.,Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA.,Wu Tsai Neuroscience Institute, Stanford University, Stanford, CA, 94305, USA.,Department of Bioengineering, Stanford University, Stanford, CA, 94305, USA.,Department of Neurobiology, School of Medicine, Stanford University, Stanford, CA, 94305, USA.,Howard Hughes Medical Institute at Stanford University, Stanford University, Stanford, CA, 94305, USA
| |
Collapse
|
38
|
Willett FR, Young DR, Murphy BA, Memberg WD, Blabe CH, Pandarinath C, Stavisky SD, Rezaii P, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral JD, Jarosiewicz B, Hochberg LR, Kirsch RF, Bolu Ajiboye A. Principled BCI Decoder Design and Parameter Selection Using a Feedback Control Model. Sci Rep 2019; 9:8881. [PMID: 31222030 PMCID: PMC6586941 DOI: 10.1038/s41598-019-44166-7] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2018] [Accepted: 03/04/2019] [Indexed: 02/01/2023] Open
Abstract
Decoders optimized offline to reconstruct intended movements from neural recordings sometimes fail to achieve optimal performance online when they are used in closed-loop as part of an intracortical brain-computer interface (iBCI). This is because typical decoder calibration routines do not model the emergent interactions between the decoder, the user, and the task parameters (e.g. target size). Here, we investigated the feasibility of simulating online performance to better guide decoder parameter selection and design. Three participants in the BrainGate2 pilot clinical trial controlled a computer cursor using a linear velocity decoder under different gain (speed scaling) and temporal smoothing parameters and acquired targets with different radii and distances. We show that a user-specific iBCI feedback control model can predict how performance changes under these different decoder and task parameters in held-out data. We also used the model to optimize a nonlinear speed scaling function for the decoder. When used online with two participants, it increased the dynamic range of decoded speeds and decreased the time taken to acquire targets (compared to an optimized standard decoder). These results suggest that it is feasible to simulate iBCI performance accurately enough to be useful for quantitative decoder optimization and design.
Collapse
Affiliation(s)
- Francis R Willett
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA. .,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA. .,Department of Neurosurgery, Stanford University, Stanford, California, USA. .,Department of Electrical Engineering, Stanford University, Stanford, California, USA.
| | - Daniel R Young
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - Brian A Murphy
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - William D Memberg
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - Christine H Blabe
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Chethan Pandarinath
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Sergey D Stavisky
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Paymon Rezaii
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Jad Saab
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA
| | - Benjamin L Walter
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurology, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jennifer A Sweet
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurosurgery, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jonathan P Miller
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurosurgery, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, 94305, California, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, California, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, 94305, California, USA.,Department of Bioengineering, Stanford University, Stanford, California, 94305, USA.,Department of Neurobiology, Stanford University, Stanford, California, 94305, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, California, 94305, USA.,Neurosciences Program, Stanford University, Stanford, California, 94305, USA.,Bio-X Program, Stanford University, Stanford, California, 94305, USA
| | - John D Simeral
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA.,Carney Institute for Brain Science, Brown University, Providence, Rhode Island, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Beata Jarosiewicz
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Leigh R Hochberg
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA.,Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
| | - Robert F Kirsch
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - A Bolu Ajiboye
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| |
Collapse
|
39
|
Trautmann EM, Stavisky SD, Lahiri S, Ames KC, Kaufman MT, O'Shea DJ, Vyas S, Sun X, Ryu SI, Ganguli S, Shenoy KV. Accurate Estimation of Neural Population Dynamics without Spike Sorting. Neuron 2019; 103:292-308.e4. [PMID: 31171448 DOI: 10.1016/j.neuron.2019.05.003] [Citation(s) in RCA: 121] [Impact Index Per Article: 24.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2017] [Revised: 02/06/2019] [Accepted: 04/30/2019] [Indexed: 11/25/2022]
Abstract
A central goal of systems neuroscience is to relate an organism's neural activity to behavior. Neural population analyses often reduce the data dimensionality to focus on relevant activity patterns. A major hurdle to data analysis is spike sorting, and this problem is growing as the number of recorded neurons increases. Here, we investigate whether spike sorting is necessary to estimate neural population dynamics. The theory of random projections suggests that we can accurately estimate the geometry of low-dimensional manifolds from a small number of linear projections of the data. We recorded data using Neuropixels probes in motor cortex of nonhuman primates and reanalyzed data from three previous studies and found that neural dynamics and scientific conclusions are quite similar using multiunit threshold crossings rather than sorted neurons. This finding unlocks existing data for new analyses and informs the design and use of new electrode arrays for laboratory and clinical use.
Collapse
Affiliation(s)
- Eric M Trautmann
- Neurosciences Program, Stanford University, Stanford, CA, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.
| | - Sergey D Stavisky
- Neurosciences Program, Stanford University, Stanford, CA, USA; Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Subhaneil Lahiri
- Department of Applied Physics, Stanford University, Stanford, CA, USA
| | - Katherine C Ames
- Neurosciences Program, Stanford University, Stanford, CA, USA; Department of Neuroscience, Columbia University, New York, NY, USA
| | - Matthew T Kaufman
- Neurosciences Program, Stanford University, Stanford, CA, USA; Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL, USA
| | - Daniel J O'Shea
- Neurosciences Program, Stanford University, Stanford, CA, USA; Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Saurabh Vyas
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Xulu Sun
- Department of Biology, Stanford University, Stanford, CA, USA
| | - Stephen I Ryu
- Palo Alto Medical Foundation, Palo Alto, CA, USA; Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Surya Ganguli
- Department of Applied Physics, Stanford University, Stanford, CA, USA; Department of Electrical Engineering, Stanford University, Stanford, CA, USA; Department of Neurobiology, Stanford University, Stanford, CA, USA; Stanford Neurosciences Institute, Stanford, CA, USA; Bio-X Program, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Neurosciences Program, Stanford University, Stanford, CA, USA; Department of Electrical Engineering, Stanford University, Stanford, CA, USA; Department of Neurobiology, Stanford University, Stanford, CA, USA; Stanford Neurosciences Institute, Stanford, CA, USA; Bio-X Program, Stanford University, Stanford, CA, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|
40
|
Wang M, Montanède C, Chandrasekaran C, Peixoto D, Shenoy KV, Kalaska JF. Macaque dorsal premotor cortex exhibits decision-related activity only when specific stimulus-response associations are known. Nat Commun 2019; 10:1793. [PMID: 30996222 PMCID: PMC6470163 DOI: 10.1038/s41467-019-09460-y] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2018] [Accepted: 03/12/2019] [Indexed: 01/16/2023] Open
Abstract
How deliberation on sensory cues and action selection interact in decision-related brain areas is still not well understood. Here, monkeys reached to one of two targets, whose colors alternated randomly between trials, by discriminating the dominant color of a checkerboard cue composed of different numbers of squares of the two target colors in different trials. In a Targets First task the colored targets appeared first, followed by the checkerboard; in a Checkerboard First task, this order was reversed. After both cues appeared in both tasks, responses of dorsal premotor cortex (PMd) units covaried with action choices, strength of evidence for action choices, and RTs- hallmarks of decision-related activity. However, very few units were modulated by checkerboard color composition or the color of the chosen target, even during the checkerboard deliberation epoch of the Checkerboard First task. These findings implicate PMd in the action-selection but not the perceptual components of the decision-making process in these tasks.
Collapse
Affiliation(s)
- Megan Wang
- Neurosciences Graduate Program, Stanford University, Stanford, CA, 94305, USA
| | - Christéva Montanède
- Département de Neurosciences, Pavillon Paul-G.-Desmarais, Faculté de Médecine, Université de Montréal, succursale Centre-ville, Montréal, QC, H3C 3J7, Canada
| | - Chandramouli Chandrasekaran
- Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, 94305, USA
- Department of Anatomy and Neurobiology, Boston University, Boston, MA, 02118, USA
- Department of Psychological and Brain Sciences, Boston University, Boston, MA, 02215, USA
| | - Diogo Peixoto
- Department of Neurobiology, Stanford University, Stanford, CA, 94305, USA
- Champalimaud Neuroscience Programme, 1400-038, Lisbon, Portugal
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, 94305, USA
- Department of Neurobiology, Stanford University, Stanford, CA, 94305, USA
- Department of Bioengineering, Stanford University, Stanford, CA, 94305, USA
- Bio-X Program, Stanford University, Stanford, CA, 94305, USA
- Stanford Neurosciences Institute, Stanford University, Stanford, CA, 94305, USA
| | - John F Kalaska
- Département de Neurosciences, Pavillon Paul-G.-Desmarais, Faculté de Médecine, Université de Montréal, succursale Centre-ville, Montréal, QC, H3C 3J7, Canada.
| |
Collapse
|
41
|
Even-Chen N, Sheffer B, Vyas S, Ryu SI, Shenoy KV. Structure and variability of delay activity in premotor cortex. PLoS Comput Biol 2019; 15:e1006808. [PMID: 30794541 PMCID: PMC6402694 DOI: 10.1371/journal.pcbi.1006808] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2018] [Revised: 03/06/2019] [Accepted: 01/21/2019] [Indexed: 11/18/2022] Open
Abstract
Voluntary movements are widely considered to be planned before they are executed. Recent studies have hypothesized that neural activity in motor cortex during preparation acts as an ‘initial condition’ which seeds the proceeding neural dynamics. Here, we studied these initial conditions in detail by investigating 1) the organization of neural states for different reaches and 2) the variance of these neural states from trial to trial. We examined population-level responses in macaque premotor cortex (PMd) during the preparatory stage of an instructed-delay center-out reaching task with dense target configurations. We found that after target onset the neural activity on single trials converges to neural states that have a clear low-dimensional structure which is organized by both the reach endpoint and maximum speed of the following reach. Further, we found that variability of the neural states during preparation resembles the spatial variability of reaches made in the absence of visual feedback: there is less variability in direction than distance in neural state space. We also used offline decoding to understand the implications of this neural population structure for brain-machine interfaces (BMIs). We found that decoding of angle between reaches is dependent on reach distance, while decoding of arc-length is independent. Thus, it might be more appropriate to quantify decoding performance for discrete BMIs by using arc-length between reach end-points rather than the angle between them. Lastly, we show that in contrast to the common notion that direction can better be decoded than distance, their decoding capabilities are comparable. These results provide new insights into the dynamical neural processes that underline motor control and can inform the design of BMIs. Early studies of premotor cortex explored how individual neurons directly encode aspects of an upcoming movement during preparation. Recent developments have proposed that the dynamics of populations of neurons underlie motor control, and that neural activity during preparation serves to set up these dynamics. While the dynamics of motor control have been studied extensively, several aspects of preparatory activity remain unresolved. Here, we ask how the patterns of neural activity during preparation for different reaches are related to one another. We found that the neural activity during preparation for reaches to different targets has a clear ‘structure’. Additionally, we found that the activity on a given trial was predictive of the initial trajectory of the reach. Lastly, we assessed the implications of our findings for predicting upcoming movements from neural activity, as in brain-machine interfaces.
Collapse
Affiliation(s)
- Nir Even-Chen
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- * E-mail:
| | - Blue Sheffer
- Department of Computer Science, Stanford University, Stanford, CA, United States of America
| | - Saurabh Vyas
- Department of Bioengineering, Stanford University, Stanford, CA, United States of America
| | - Stephen I. Ryu
- Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA, United States of America
| | - Krishna V. Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Department of Bioengineering, Stanford University, Stanford, CA, United States of America
- Department of Neurobiology, Stanford University, Stanford, CA, United States of America
- The Bio-X Program, Stanford University, Stanford, CA, United States of America
- The Stanford Neurosciences Institute, Stanford University, Stanford, CA, United States of America
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, United States of America
| |
Collapse
|
42
|
Milekovic T, Bacher D, Sarma AA, Simeral JD, Saab J, Pandarinath C, Yvert B, Sorice BL, Blabe C, Oakley EM, Tringale KR, Eskandar E, Cash SS, Shenoy KV, Henderson JM, Hochberg LR, Donoghue JP. Volitional control of single-electrode high gamma local field potentials by people with paralysis. J Neurophysiol 2019; 121:1428-1450. [PMID: 30785814 DOI: 10.1152/jn.00131.2018] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
Intracortical brain-computer interfaces (BCIs) can enable individuals to control effectors, such as a computer cursor, by directly decoding the user's movement intentions from action potentials and local field potentials (LFPs) recorded within the motor cortex. However, the accuracy and complexity of effector control achieved with such "biomimetic" BCIs will depend on the degree to which the intended movements used to elicit control modulate the neural activity. In particular, channels that do not record distinguishable action potentials and only record LFP modulations may be of limited use for BCI control. In contrast, a biofeedback approach may surpass these limitations by letting the participants generate new control signals and learn strategies that improve the volitional control of signals used for effector control. Here, we show that, by using a biofeedback paradigm, three individuals with tetraplegia achieved volitional control of gamma LFPs (40-400 Hz) recorded by a single microelectrode implanted in the precentral gyrus. Control was improved over a pair of consecutive sessions up to 3 days apart. In all but one session, the channel used to achieve control lacked distinguishable action potentials. Our results indicate that biofeedback LFP-based BCIs may potentially contribute to the neural modulation necessary to obtain reliable and useful control of effectors. NEW & NOTEWORTHY Our study demonstrates that people with tetraplegia can volitionally control individual high-gamma local-field potential (LFP) channels recorded from the motor cortex, and that this control can be improved using biofeedback. Motor cortical LFP signals are thought to be both informative and stable intracortical signals and, thus, of importance for future brain-computer interfaces.
Collapse
Affiliation(s)
- Tomislav Milekovic
- Department of Neuroscience, Brown University , Providence, Rhode Island.,Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,Department of Fundamental Neuroscience, Faculty of Medicine, University of Geneva , Geneva , Switzerland
| | - Daniel Bacher
- Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,School of Engineering, Brown University , Providence, Rhode Island
| | - Anish A Sarma
- Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,School of Engineering, Brown University , Providence, Rhode Island.,Center for Neurorestoration and Neurotechnology, Rehabilitation Research & Development Service, Department of Veterans Affairs , Providence, Rhode Island
| | - John D Simeral
- Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,School of Engineering, Brown University , Providence, Rhode Island.,Center for Neurorestoration and Neurotechnology, Rehabilitation Research & Development Service, Department of Veterans Affairs , Providence, Rhode Island
| | - Jad Saab
- Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,School of Engineering, Brown University , Providence, Rhode Island
| | - Chethan Pandarinath
- Department of Neurosurgery, Stanford University , Stanford, California.,Department of Electrical Engineering, Stanford University , Stanford, California.,Stanford Neurosciences Institute, Stanford University , Stanford, California
| | - Blaise Yvert
- Department of Neuroscience, Brown University , Providence, Rhode Island.,Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,Inserm, University of Grenoble, Clinatec-Lab U1205, Grenoble , France
| | - Brittany L Sorice
- Department of Neurology, Massachusetts General Hospital , Boston, Massachusetts
| | - Christine Blabe
- Department of Neurosurgery, Stanford University , Stanford, California
| | - Erin M Oakley
- Department of Neurology, Massachusetts General Hospital , Boston, Massachusetts
| | - Kathryn R Tringale
- Department of Neurology, Massachusetts General Hospital , Boston, Massachusetts
| | - Emad Eskandar
- Department of Neurosurgery, Massachusetts General Hospital , Boston, Massachusetts.,Harvard Medical School , Boston, Massachusetts
| | - Sydney S Cash
- Department of Neurology, Massachusetts General Hospital , Boston, Massachusetts.,Harvard Medical School , Boston, Massachusetts
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University , Stanford, California.,Stanford Neurosciences Institute, Stanford University , Stanford, California.,Neurosciences Program, Stanford University , Stanford, California.,Department of Neurobiology, Stanford University , Stanford, California.,Department of Bioengineering, Stanford University , Stanford, California
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University , Stanford, California.,Stanford Neurosciences Institute, Stanford University , Stanford, California.,Department of Neurology and Neurological Sciences, Stanford University , Stanford, California
| | - Leigh R Hochberg
- Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,School of Engineering, Brown University , Providence, Rhode Island.,Center for Neurorestoration and Neurotechnology, Rehabilitation Research & Development Service, Department of Veterans Affairs , Providence, Rhode Island.,Department of Neurology, Massachusetts General Hospital , Boston, Massachusetts.,Harvard Medical School , Boston, Massachusetts
| | - John P Donoghue
- Department of Neuroscience, Brown University , Providence, Rhode Island.,Carney Institute for Brain Science, Brown University , Providence, Rhode Island.,Center for Neurorestoration and Neurotechnology, Rehabilitation Research & Development Service, Department of Veterans Affairs , Providence, Rhode Island
| |
Collapse
|
43
|
Young D, Willett F, Memberg WD, Murphy B, Rezaii P, Walter B, Sweet J, Miller J, Shenoy KV, Hochberg LR, Kirsch RF, Ajiboye AB. Closed-loop cortical control of virtual reach and posture using Cartesian and joint velocity commands. J Neural Eng 2018; 16:026011. [PMID: 30523839 DOI: 10.1088/1741-2552/aaf606] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) are a promising technology for the restoration of function to people with paralysis, especially for controlling coordinated reaching. Typical BCI studies decode Cartesian endpoint velocities as commands, but human arm movements might be better controlled in a joint-based coordinate frame, which may match underlying movement encoding in the motor cortex. A better understanding of BCI controlled reaching by people with paralysis may lead to performance improvements in brain-controlled assistive devices. APPROACH Two intracortical BCI participants in the BrainGate2 pilot clinical trial performed a visual 3D endpoint virtual reality reaching task using two decoders: Cartesian and joint velocity. Task performance metrics (i.e. success rate and path efficiency) and single feature and population tuning were compared across the two decoder conditions. The participants also demonstrated the first BCI control of a fourth dimension of reaching, the arm's swivel angle, in a 4D posture matching task. MAIN RESULTS Both users achieved significantly higher success rates using Cartesian velocity control, and joint controlled trajectories were more variable and significantly more curved. Neural tuning analyses showed that most single feature activity was best described by a Cartesian kinematic encoding model, and population analyses revealed only slight differences in aggregate activity between the decoder conditions. Simulations of a BCI user reproduced trajectory features seen during closed-loop joint control when assuming only Cartesian-tuned features passed through a joint decoder. With minimal training, both participants controlled the virtual arm's swivel angle to complete a 4D posture matching task, and achieved significantly higher success using a Cartesian + swivel velocity decoder compared to a joint velocity decoder. SIGNIFICANCE These results suggest that Cartesian velocity command interfaces may provide better BCI control of arm movements than other kinematic variables, even in 4D posture tasks with swivel angle targets.
Collapse
Affiliation(s)
- D Young
- Case Western Reserve University, Cleveland, OH, United States of America. Department of VA Medical Center, FES Center of Excellence, Rehabilitation R&D Service, Louis Stokes Cleveland, Cleveland, OH, United States of America
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
44
|
Nuyujukian P, Albites Sanabria J, Saab J, Pandarinath C, Jarosiewicz B, Blabe CH, Franco B, Mernoff ST, Eskandar EN, Simeral JD, Hochberg LR, Shenoy KV, Henderson JM. Cortical control of a tablet computer by people with paralysis. PLoS One 2018; 13:e0204566. [PMID: 30462658 PMCID: PMC6248919 DOI: 10.1371/journal.pone.0204566] [Citation(s) in RCA: 64] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2017] [Accepted: 09/11/2018] [Indexed: 12/16/2022] Open
Abstract
General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.
Collapse
Affiliation(s)
- Paul Nuyujukian
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Department of Bioengineering, Stanford University, Stanford, CA, United States of America
- Neurosciences Institute, Stanford University, Stanford, CA, United States of America
- Bio-X Institute, Stanford University, Stanford, CA, United States of America
- Neurosciences Program, Stanford University, Stanford, CA, United States of America
| | - Jose Albites Sanabria
- School of Engineering, Brown University, Providence, RI, United States of America
- Carney Institute for Brain Science, Brown University, Providence, RI, United States of America
| | - Jad Saab
- School of Engineering, Brown University, Providence, RI, United States of America
- Carney Institute for Brain Science, Brown University, Providence, RI, United States of America
- Center for Neurorestoration and Neurotechnology, Rehabilitation Research and Development Service, VA Medical Center, Providence, RI, United States of America
| | - Chethan Pandarinath
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Department of Biomedical Engineering, Emory University and the Georgia Institute of Technology, Atlanta, GA, United States of America
- Department of Neurosurgery, Emory University, Atlanta, GA, United States of America
| | - Beata Jarosiewicz
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Carney Institute for Brain Science, Brown University, Providence, RI, United States of America
- Department of Neuroscience, Brown University, Providence, RI, United States of America
| | - Christine H. Blabe
- Department of Neurosurgery, Stanford University, Stanford, CA, United States of America
| | - Brian Franco
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, United States of America
| | - Stephen T. Mernoff
- Center for Neurorestoration and Neurotechnology, Rehabilitation Research and Development Service, VA Medical Center, Providence, RI, United States of America
- Department of Neurology, Warren Alpert Medical School of Brown University, Providence, RI, United States of America
| | - Emad N. Eskandar
- Department of Neurosurgery, Harvard Medical School, Boston, MA, United States of America
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, United States of America
| | - John D. Simeral
- School of Engineering, Brown University, Providence, RI, United States of America
- Carney Institute for Brain Science, Brown University, Providence, RI, United States of America
- Center for Neurorestoration and Neurotechnology, Rehabilitation Research and Development Service, VA Medical Center, Providence, RI, United States of America
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, United States of America
| | - Leigh R. Hochberg
- School of Engineering, Brown University, Providence, RI, United States of America
- Carney Institute for Brain Science, Brown University, Providence, RI, United States of America
- Center for Neurorestoration and Neurotechnology, Rehabilitation Research and Development Service, VA Medical Center, Providence, RI, United States of America
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, United States of America
- Department of Neurology, Harvard Medical School, Boston, MA, United States of America
| | - Krishna V. Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Department of Bioengineering, Stanford University, Stanford, CA, United States of America
- Neurosciences Institute, Stanford University, Stanford, CA, United States of America
- Bio-X Institute, Stanford University, Stanford, CA, United States of America
- Neurosciences Program, Stanford University, Stanford, CA, United States of America
- Department of Neurobiology, Stanford University, Stanford, CA, United States of America
- Howard Hughes Medical Institute at Stanford University, Chevy Chase, MD, United States of America
| | - Jaimie M. Henderson
- Department of Electrical Engineering, Stanford University, Stanford, CA, United States of America
- Neurosciences Institute, Stanford University, Stanford, CA, United States of America
- Bio-X Institute, Stanford University, Stanford, CA, United States of America
| |
Collapse
|
45
|
Jiang X, Ryu SI, Shenoy KV, Kao JC. Single Neuron Firing Rate Statistics in Motor Cortex During Execution and Observation of Movement. Annu Int Conf IEEE Eng Med Biol Soc 2018; 2018:981-986. [PMID: 30440555 DOI: 10.1109/embc.2018.8512445] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Mirror neurons, which fire during both the execution and observation of movement, are believed to play an important role in motor processing and learning. However, much work still remains to understand the similarities and differences in how these neurons compute in the motor cortex during movement execution and observation. Here, we performed experiments where a monkey both executes and observes a center-out-and-back task within the same experimental session. By recording from putatively the same neural population, we were able to analyze and compare single neuron statistics between movement execution and observation. We found that a majority of neurons in the primary motor cortex (M1) and dorsal premotor cortex (PMd) have statistically different firing rate statistics between movement execution and observation. As a result of this difference, we then wondered if neurons during movement observation exhibited a similar characteristic to those during movement execution: changing of preferred directions as a function of movement speed. Interestingly, we found that while observed movement speed is encoded in the neural population, it only alters a small proportion of the neuron's firing rate statistics. These results suggest that neural populations in Ml and PMd process information related to movement differently between execution and observation.
Collapse
|
46
|
Stavisky SD, Kao JC, Nuyujukian P, Pandarinath C, Blabe C, Ryu SI, Hochberg LR, Henderson JM, Shenoy KV. Brain-machine interface cursor position only weakly affects monkey and human motor cortical activity in the absence of arm movements. Sci Rep 2018; 8:16357. [PMID: 30397281 PMCID: PMC6218537 DOI: 10.1038/s41598-018-34711-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2018] [Accepted: 10/24/2018] [Indexed: 12/26/2022] Open
Abstract
Brain-machine interfaces (BMIs) that decode movement intentions should ignore neural modulation sources distinct from the intended command. However, neurophysiology and control theory suggest that motor cortex reflects the motor effector's position, which could be a nuisance variable. We investigated motor cortical correlates of BMI cursor position with or without concurrent arm movement. We show in two monkeys that subtracting away estimated neural correlates of position improves online BMI performance only if the animals were allowed to move their arm. To understand why, we compared the neural variance attributable to cursor position when the same task was performed using arm reaching, versus arms-restrained BMI use. Firing rates correlated with both BMI cursor and hand positions, but hand positional effects were greater. To examine whether BMI position influences decoding in people with paralysis, we analyzed data from two intracortical BMI clinical trial participants and performed an online decoder comparison in one participant. We found only small motor cortical correlates, which did not affect performance. These results suggest that arm movement and proprioception are the major contributors to position-related motor cortical correlates. Cursor position visual feedback is therefore unlikely to affect the performance of BMI-driven prosthetic systems being developed for people with paralysis.
Collapse
Affiliation(s)
- Sergey D Stavisky
- Neurosurgery Department, Stanford University, Stanford, CA, USA.
- Electrical Engineering Department, Stanford University, Stanford, CA, USA.
| | - Jonathan C Kao
- Electrical Engineering Department, Stanford University, Stanford, CA, USA
- Electrical and Computer Engineering Department, University of California at Los Angeles, Los Angeles, CA, USA
| | - Paul Nuyujukian
- Neurosurgery Department, Stanford University, Stanford, CA, USA
- Electrical Engineering Department, Stanford University, Stanford, CA, USA
- Bioengineering Department, Stanford University, Stanford, CA, USA
- Stanford Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Program, Stanford University, Stanford, CA, USA
| | - Chethan Pandarinath
- Neurosurgery Department, Stanford University, Stanford, CA, USA
- Electrical Engineering Department, Stanford University, Stanford, CA, USA
| | - Christine Blabe
- Neurosurgery Department, Stanford University, Stanford, CA, USA
| | - Stephen I Ryu
- Electrical Engineering Department, Stanford University, Stanford, CA, USA
- Neurosurgery Department, Palo Alto Medical Foundation, Palo Alto, CA, USA
| | - Leigh R Hochberg
- Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, VA Medical Center, Providence, RI, USA
- School of Engineering and Carney Institute for Brain Science Brown University, Providence, RI, USA
- Department of Neurology, Harvard Medical School, Boston, MA, USA
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
| | - Jaimie M Henderson
- Neurosurgery Department, Stanford University, Stanford, CA, USA
- Stanford Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Program, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Electrical Engineering Department, Stanford University, Stanford, CA, USA
- Bioengineering Department, Stanford University, Stanford, CA, USA
- Stanford Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Program, Stanford University, Stanford, CA, USA
- Neurobiology Department, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| |
Collapse
|
47
|
Pandarinath C, O'Shea DJ, Collins J, Jozefowicz R, Stavisky SD, Kao JC, Trautmann EM, Kaufman MT, Ryu SI, Hochberg LR, Henderson JM, Shenoy KV, Abbott LF, Sussillo D. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat Methods 2018; 15:805-815. [PMID: 30224673 PMCID: PMC6380887 DOI: 10.1038/s41592-018-0109-9] [Citation(s) in RCA: 242] [Impact Index Per Article: 40.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2018] [Accepted: 06/28/2018] [Indexed: 01/28/2023]
Abstract
Neuroscience is experiencing a revolution in which simultaneous recording
of many thousands of neurons is revealing population dynamics that are not
apparent from single-neuron responses. This structure is typically extracted
from trial-averaged data, but deeper understanding requires studying
single-trial phenomena, which is challenging due to incomplete sampling of the
neural population, trial-to-trial variability, and fluctuations in action
potential timing. We introduce Latent Factor Analysis via Dynamical Systems
(LFADS), a deep learning method to infer latent dynamics from single-trial
neural spiking data. LFADS uses a nonlinear dynamical system to infer the
dynamics underlying observed spiking activity and to extract
‘de-noised’ single-trial firing rates. When applied to a variety
of monkey and human motor cortical datasets, LFADS predicts observed behavioral
variables with unprecedented accuracy, extracts precise estimates of neural
dynamics on single trials, infers perturbations to those dynamics that correlate
with behavioral choices, and combines data from non-overlapping recording
sessions spanning months to improve inference of underlying dynamics.
Collapse
Affiliation(s)
- Chethan Pandarinath
- Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA. .,Department of Neurosurgery, Emory University, Atlanta, GA, USA. .,Department of Neurosurgery, Stanford University, Stanford, CA, USA. .,Department of Electrical Engineering, Stanford University, Stanford, CA, USA. .,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA.
| | - Daniel J O'Shea
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Jasmine Collins
- Google AI, Google Inc., Mountain View, CA, USA.,University of California, Berkeley, Berkeley, CA, USA
| | - Rafal Jozefowicz
- Google AI, Google Inc., Mountain View, CA, USA.,OpenAI, San Francisco, CA, USA
| | - Sergey D Stavisky
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.,Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA.,Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Jonathan C Kao
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Department of Electrical Engineering, University of California, Los Angeles, Los Angeles, CA, USA
| | - Eric M Trautmann
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Matthew T Kaufman
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA.,Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA, USA
| | - Leigh R Hochberg
- VA RR&D Center for Neurorestoration and Neurotechnology, Veterans Affairs Medical Center, Providence, RI, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.,School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA.,Department of Neurobiology, Stanford University, Stanford, CA, USA.,Department of Bioengineering, Stanford University, Stanford, CA, USA.,Bio-X Program, Stanford University, Stanford, CA, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| | - L F Abbott
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA.,Department of Neuroscience, Columbia University, New York, NY, USA.,Department of Physiology and Cellular Biophysics, Columbia University, New York, NY, USA
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA. .,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA. .,Google AI, Google Inc., Mountain View, CA, USA.
| |
Collapse
|
48
|
Willett FR, Murphy BA, Young DR, Memberg WD, Blabe CH, Pandarinath C, Franco B, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral JD, Jarosiewicz B, Hochberg LR, Kirsch RF, Ajiboye AB. A Comparison of Intention Estimation Methods for Decoder Calibration in Intracortical Brain-Computer Interfaces. IEEE Trans Biomed Eng 2018; 65:2066-2078. [PMID: 29989927 PMCID: PMC6043406 DOI: 10.1109/tbme.2017.2783358] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
OBJECTIVE Recent reports indicate that making better assumptions about the user's intended movement can improve the accuracy of decoder calibration for intracortical brain-computer interfaces. Several methods now exist for estimating user intent, including an optimal feedback control model, a piecewise-linear feedback control model, ReFIT, and other heuristics. Which of these methods yields the best decoding performance? METHODS Using data from the BrainGate2 pilot clinical trial, we measured how a steady-state velocity Kalman filter decoder was affected by the choice of intention estimation method. We examined three separate components of the Kalman filter: dimensionality reduction, temporal smoothing, and output gain (speed scaling). RESULTS The decoder's dimensionality reduction properties were largely unaffected by the intention estimation method. Decoded velocity vectors differed by <5% in terms of angular error and speed vs. target distance curves across methods. In contrast, the smoothing and gain properties of the decoder were greatly affected (> 50% difference in average values). Since the optimal gain and smoothing properties are task-specific (e.g. lower gains are better for smaller targets but worse for larger targets), no one method was better for all tasks. CONCLUSION Our results show that, when gain and smoothing differences are accounted for, current intention estimation methods yield nearly equivalent decoders and that simple models of user intent, such as a position error vector (target position minus cursor position), perform comparably to more elaborate models. Our results also highlight that simple differences in gain and smoothing properties have a large effect on online performance and can confound decoder comparisons.
Collapse
|
49
|
Williams AH, Kim TH, Wang F, Vyas S, Ryu SI, Shenoy KV, Schnitzer M, Kolda TG, Ganguli S. Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis. Neuron 2018; 98:1099-1115.e8. [PMID: 29887338 DOI: 10.1016/j.neuron.2018.05.015] [Citation(s) in RCA: 112] [Impact Index Per Article: 18.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2017] [Revised: 03/18/2018] [Accepted: 05/08/2018] [Indexed: 01/19/2023]
Abstract
Perceptions, thoughts, and actions unfold over millisecond timescales, while learned behaviors can require many days to mature. While recent experimental advances enable large-scale and long-term neural recordings with high temporal fidelity, it remains a formidable challenge to extract unbiased and interpretable descriptions of how rapid single-trial circuit dynamics change slowly over many trials to mediate learning. We demonstrate a simple tensor component analysis (TCA) can meet this challenge by extracting three interconnected, low-dimensional descriptions of neural data: neuron factors, reflecting cell assemblies; temporal factors, reflecting rapid circuit dynamics mediating perceptions, thoughts, and actions within each trial; and trial factors, describing both long-term learning and trial-to-trial changes in cognitive state. We demonstrate the broad applicability of TCA by revealing insights into diverse datasets derived from artificial neural networks, large-scale calcium imaging of rodent prefrontal cortex during maze navigation, and multielectrode recordings of macaque motor cortex during brain machine interface learning.
Collapse
Affiliation(s)
- Alex H Williams
- Neurosciences Graduate Program, Stanford University, Stanford, CA 94305, USA.
| | - Tony Hyun Kim
- Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA
| | - Forea Wang
- Neurosciences Graduate Program, Stanford University, Stanford, CA 94305, USA
| | - Saurabh Vyas
- Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Bioengineering Department, Stanford University, Stanford, CA 94305, USA
| | - Stephen I Ryu
- Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA 94301, USA
| | - Krishna V Shenoy
- Electrical Engineering Department, Stanford University, Stanford, CA 94305, USA; Bioengineering Department, Stanford University, Stanford, CA 94305, USA; Neurobiology Department, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Stanford Neurosciences Institute, Stanford University, Stanford, CA 94305, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA
| | - Mark Schnitzer
- Applied Physics Department, Stanford University, Stanford, CA 94305, USA; Biology Department, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Howard Hughes Medical Institute, Stanford University, Stanford, CA 94305, USA; CNC Program, Stanford University, Stanford, CA 94305, USA
| | | | - Surya Ganguli
- Applied Physics Department, Stanford University, Stanford, CA 94305, USA; Neurobiology Department, Stanford University, Stanford, CA 94305, USA; Bio-X Program, Stanford University, Stanford, CA 94305, USA; Stanford Neurosciences Institute, Stanford University, Stanford, CA 94305, USA.
| |
Collapse
|
50
|
O'Shea DJ, Kalanithi P, Ferenczi EA, Hsueh B, Chandrasekaran C, Goo W, Diester I, Ramakrishnan C, Kaufman MT, Ryu SI, Yeom KW, Deisseroth K, Shenoy KV. Development of an optogenetic toolkit for neural circuit dissection in squirrel monkeys. Sci Rep 2018; 8:6775. [PMID: 29712920 PMCID: PMC5928036 DOI: 10.1038/s41598-018-24362-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2017] [Accepted: 04/03/2018] [Indexed: 01/01/2023] Open
Abstract
Optogenetic tools have opened a rich experimental landscape for understanding neural function and disease. Here, we present the first validation of eight optogenetic constructs driven by recombinant adeno-associated virus (AAV) vectors and a WGA-Cre based dual injection strategy for projection targeting in a widely-used New World primate model, the common squirrel monkey Saimiri sciureus. We observed opsin expression around the local injection site and in axonal projections to downstream regions, as well as transduction to thalamic neurons, resembling expression patterns observed in macaques. Optical stimulation drove strong, reliable excitatory responses in local neural populations for two depolarizing opsins in anesthetized monkeys. Finally, we observed continued, healthy opsin expression for at least one year. These data suggest that optogenetic tools can be readily applied in squirrel monkeys, an important first step in enabling precise, targeted manipulation of neural circuits in these highly trainable, cognitively sophisticated animals. In conjunction with similar approaches in macaques and marmosets, optogenetic manipulation of neural circuits in squirrel monkeys will provide functional, comparative insights into neural circuits which subserve dextrous motor control as well as other adaptive behaviors across the primate lineage. Additionally, development of these tools in squirrel monkeys, a well-established model system for several human neurological diseases, can aid in identifying novel treatment strategies.
Collapse
Affiliation(s)
- Daniel J O'Shea
- Neurosciences Program, Stanford University, Stanford, CA, USA.
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Paul Kalanithi
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | | | - Brian Hsueh
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | | | - Werapong Goo
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Ilka Diester
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Otophysiologie, Albert Ludwig University of Freiburg, Freiburg im Breisgau, Germany
- BrainLinks-BrainTools, Albert Ludwig University of Freiburg, Freiburg im Breisgau, Germany
| | | | - Matthew T Kaufman
- Neurosciences Program, Stanford University, Stanford, CA, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Palo Alto Medical Foundation, Palo Alto, CA, USA
| | - Kristen W Yeom
- Department of Radiology, Stanford University, Stanford, CA, USA
| | - Karl Deisseroth
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Psychiatry and Behavioral Science, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Neurosciences Program, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|