1
|
Deo DR, Willett FR, Avansino DT, Hochberg LR, Henderson JM, Shenoy KV. Brain control of bimanual movement enabled by recurrent neural networks. Sci Rep 2024; 14:1598. [PMID: 38238386 PMCID: PMC10796685 DOI: 10.1038/s41598-024-51617-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 01/07/2024] [Indexed: 01/22/2024] Open
Abstract
Brain-computer interfaces have so far focused largely on enabling the control of a single effector, for example a single computer cursor or robotic arm. Restoring multi-effector motion could unlock greater functionality for people with paralysis (e.g., bimanual movement). However, it may prove challenging to decode the simultaneous motion of multiple effectors, as we recently found that a compositional neural code links movements across all limbs and that neural tuning changes nonlinearly during dual-effector motion. Here, we demonstrate the feasibility of high-quality bimanual control of two cursors via neural network (NN) decoders. Through simulations, we show that NNs leverage a neural 'laterality' dimension to distinguish between left and right-hand movements as neural tuning to both hands become increasingly correlated. In training recurrent neural networks (RNNs) for two-cursor control, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously. Our results suggest that neural network decoders may be advantageous for multi-effector decoding, provided they are designed to transfer to the online setting.
Collapse
Affiliation(s)
- Darrel R Deo
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
| | - Francis R Willett
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | - Donald T Avansino
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | - Leigh R Hochberg
- School of Engineering, Brown University, Providence, RI, USA
- Carney Institute for Brain Science, Brown University, Providence, RI, USA
- VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA
- Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| |
Collapse
|
2
|
Deo DR, Willett FR, Avansino DT, Hochberg LR, Henderson JM, Shenoy KV. Translating deep learning to neuroprosthetic control. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.21.537581. [PMID: 37131830 PMCID: PMC10153231 DOI: 10.1101/2023.04.21.537581] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Advances in deep learning have given rise to neural network models of the relationship between movement and brain activity that appear to far outperform prior approaches. Brain-computer interfaces (BCIs) that enable people with paralysis to control external devices, such as robotic arms or computer cursors, might stand to benefit greatly from these advances. We tested recurrent neural networks (RNNs) on a challenging nonlinear BCI problem: decoding continuous bimanual movement of two computer cursors. Surprisingly, we found that although RNNs appeared to perform well in offline settings, they did so by overfitting to the temporal structure of the training data and failed to generalize to real-time neuroprosthetic control. In response, we developed a method that alters the temporal structure of the training data by dilating/compressing it in time and re-ordering it, which we show helps RNNs successfully generalize to the online setting. With this method, we demonstrate that a person with paralysis can control two computer cursors simultaneously, far outperforming standard linear methods. Our results provide evidence that preventing models from overfitting to temporal structure in training data may, in principle, aid in translating deep learning advances to the BCI setting, unlocking improved performance for challenging applications.
Collapse
|
3
|
Awasthi P, Lin TH, Bae J, Miller LE, Danziger ZC. Validation of a non-invasive, real-time, human-in-the-loop model of intracortical brain-computer interfaces. J Neural Eng 2022; 19:056038. [PMID: 36198278 PMCID: PMC9855658 DOI: 10.1088/1741-2552/ac97c3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Accepted: 10/05/2022] [Indexed: 01/26/2023]
Abstract
Objective. Despite the tremendous promise of invasive brain-computer interfaces (iBCIs), the associated study costs, risks, and ethical considerations limit the opportunity to develop and test the algorithms that decode neural activity into a user's intentions. Our goal was to address this challenge by designing an iBCI model capable of testing many human subjects in closed-loop.Approach. We developed an iBCI model that uses artificial neural networks (ANNs) to translate human finger movements into realistic motor cortex firing patterns, which can then be decoded in real time. We call the model the joint angle BCI, or jaBCI. jaBCI allows readily recruited, healthy subjects to perform closed-loop iBCI tasks using any neural decoder, preserving subjects' control-relevant short-latency error correction and learning dynamics.Main results. We validated jaBCI offline through emulated neuron firing statistics, confirming that emulated neural signals have firing rates, low-dimensional PCA geometry, and rotational jPCA dynamics that are quite similar to the actual neurons (recorded in monkey M1) on which we trained the ANN. We also tested jaBCI in closed-loop experiments, our single study examining roughly as many subjects as have been tested world-wide with iBCIs (n= 25). Performance was consistent with that of the paralyzed, human iBCI users with implanted intracortical electrodes. jaBCI allowed us to imitate the experimental protocols (e.g. the same velocity Kalman filter decoder and center-out task) and compute the same seven behavioral measures used in three critical studies.Significance. These encouraging results suggest the jaBCI's real-time firing rate emulation is a useful means to provide statistically robust sample sizes for rapid prototyping and optimization of decoding algorithms, the study of bi-directional learning in iBCIs, and improving iBCI control.
Collapse
Affiliation(s)
- Peeyush Awasthi
- Department of Biomedical Engineering, Florida International University, Miami, FL, United States of Amercia
| | - Tzu-Hsiang Lin
- Department of Biomedical Engineering, Florida International University, Miami, FL, United States of Amercia
| | - Jihye Bae
- Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, United States
| | - Lee E Miller
- Department of Neuroscience, Physical Medicine, and Rehabilitation, Northwestern University, Chicago, IL, United States
| | - Zachary C Danziger
- Department of Biomedical Engineering, Florida International University, Miami, FL, United States of Amercia,Author to whom any correspondence should be addressed
| |
Collapse
|
4
|
Shin H, Suma D, He B. Closed-loop motor imagery EEG simulation for brain-computer interfaces. Front Hum Neurosci 2022; 16:951591. [PMID: 36061506 PMCID: PMC9428352 DOI: 10.3389/fnhum.2022.951591] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 07/20/2022] [Indexed: 11/13/2022] Open
Abstract
In a brain-computer interface (BCI) system, the testing of decoding algorithms, tasks, and their parameters is critical for optimizing performance. However, conducting human experiments can be costly and time-consuming, especially when investigating broad sets of parameters. Attempts to utilize previously collected data in offline analysis lack a co-adaptive feedback loop between the system and the user present online, limiting the applicability of the conclusions obtained to real-world uses of BCI. As such, a number of studies have attempted to address this cost-wise middle ground between offline and live experimentation with real-time neural activity simulators. We present one such system which generates motor imagery electroencephalography (EEG) via forward modeling and novel motor intention encoding models for conducting sensorimotor rhythm (SMR)-based continuous cursor control experiments in a closed-loop setting. We use the proposed simulator with 10 healthy human subjects to test the effect of three decoder and task parameters across 10 different values. Our simulated approach produces similar statistical conclusions to those produced during parallel, paired, online experimentation, but in 55% of the time. Notably, both online and simulated experimentation expressed a positive effect of cursor velocity limit on performance regardless of subject average performance, supporting the idea of relaxing constraints on cursor gain in online continuous cursor control. We demonstrate the merits of our closed-loop motor imagery EEG simulation, and provide an open-source framework to the community for closed-loop SMR-based BCI studies in the future. All code including the simulator have been made available on GitHub.
Collapse
|
5
|
Costello JT, Nason SR, An H, Lee J, Mender MJ, Temmar H, Wallace DM, Lim J, Willsey MS, Patil PG, Jang T, Phillips JD, Kim HS, Blaauw D, Chestek CA. A low-power communication scheme for wireless, 1000 channel brain-machine interfaces. J Neural Eng 2022; 19. [PMID: 35613546 DOI: 10.1088/1741-2552/ac7352] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Accepted: 05/24/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Brain-machine interfaces (BMIs) have the potential to restore motor function but are currently limited by electrode count and long-term recording stability. These challenges may be solved through the use of free-floating "motes" which wirelessly transmit recorded neural signals, if power consumption can be kept within safe levels when scaling to thousands of motes. Here, we evaluated a pulse-interval modulation (PIM) communication scheme for infrared (IR)-based motes that aims to reduce the wireless data rate and system power consumption. APPROACH To test PIM's ability to efficiently communicate neural information, we simulated the communication scheme in a real-time closed-loop BMI with non-human primates. Additionally, we performed circuit simulations of an IR-based 1000-mote system to calculate communication accuracy and total power consumption. MAIN RESULTS We found that PIM at 1kb/s per channel maintained strong correlations with true firing rate and matched online BMI performance of a traditional wired system. Closed-loop BMI tests suggest that lags as small as 30 ms can have significant performance effects. Finally, unlike other IR communication schemes, PIM is feasible in terms of power, and neural data can accurately be recovered on a receiver using 3mW for 1000 channels. SIGNIFICANCE These results suggest that PIM-based communication could significantly reduce power usage of wireless motes to enable higher channel-counts for high-performance BMIs.
Collapse
Affiliation(s)
- Joseph T Costello
- Electrical and Computer Engineering, University of Michigan, 2800 Plymouth Rd, B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Samuel R Nason
- Biomedical Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Hyochan An
- Electrical and Computer Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Jungho Lee
- Electrical and Computer Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Matthew J Mender
- Biomedical Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Hisham Temmar
- Biomedical Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Dylan M Wallace
- Robotics Institute, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, 48109-1382, UNITED STATES
| | - Jongyup Lim
- Electrical and Computer Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Matthew S Willsey
- Department of Neurosurgery, University of Michigan Medical School, 1500 E Medical Center Drive, Ann Arbor, 48109-0624, UNITED STATES
| | - Parag G Patil
- Neurosurgery, University of Michigan, 1500 E Medical Center Drive, Ann Arbor, Michigan, 48109, UNITED STATES
| | - Taekwang Jang
- Department of Information Technology and Electrical Engineering, ETH Zurich, ETH Zurich, Rämistrasse 101, Zurich, 8092, SWITZERLAND
| | - Jamie Dean Phillips
- Department of Electrical and Computer Engineering, University of Delaware, Evans Hall, 139 The Green,, Newark, Delaware, 19716-5600, UNITED STATES
| | - Hun-Seok Kim
- Electrical and Computer Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - David Blaauw
- University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| | - Cynthia A Chestek
- Biomedical Engineering, University of Michigan, 2800 Plymouth Rd B10, Ann Arbor, Michigan, 48109-1382, UNITED STATES
| |
Collapse
|
6
|
Pandarinath C, Bensmaia SJ. The science and engineering behind sensitized brain-controlled bionic hands. Physiol Rev 2022; 102:551-604. [PMID: 34541898 PMCID: PMC8742729 DOI: 10.1152/physrev.00034.2020] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 09/07/2021] [Accepted: 09/13/2021] [Indexed: 12/13/2022] Open
Abstract
Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.
Collapse
Affiliation(s)
- Chethan Pandarinath
- Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, Georgia
- Department of Neurosurgery, Emory University, Atlanta, Georgia
| | - Sliman J Bensmaia
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois
- Committee on Computational Neuroscience, University of Chicago, Chicago, Illinois
- Grossman Institute for Neuroscience, Quantitative Biology, and Human Behavior, University of Chicago, Chicago, Illinois
| |
Collapse
|
7
|
Srisrisawang N, Müller-Putz GR. Applying Dimensionality Reduction Techniques in Source-Space Electroencephalography via Template and Magnetic Resonance Imaging-Derived Head Models to Continuously Decode Hand Trajectories. Front Hum Neurosci 2022; 16:830221. [PMID: 35399364 PMCID: PMC8988304 DOI: 10.3389/fnhum.2022.830221] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Accepted: 02/28/2022] [Indexed: 11/13/2022] Open
Abstract
Several studies showed evidence supporting the possibility of hand trajectory decoding from low-frequency electroencephalography (EEG). However, the decoding in the source space via source localization is scarcely investigated. In this study, we tried to tackle the problem of collinearity due to the higher number of signals in the source space by two folds: first, we selected signals in predefined regions of interest (ROIs); second, we applied dimensionality reduction techniques to each ROI. The dimensionality reduction techniques were computing the mean (Mean), principal component analysis (PCA), and locality preserving projections (LPP). We also investigated the effect of decoding between utilizing a template head model and a subject-specific head model during the source localization. The results indicated that applying source-space decoding with PCA yielded slightly higher correlations and signal-to-noise (SNR) ratios than the sensor-space approach. We also observed slightly higher correlations and SNRs when applying the subject-specific head model than the template head model. However, the statistical tests revealed no significant differences between the source-space and sensor-space approaches and no significant differences between subject-specific and template head models. The decoder with Mean and PCA utilizes information mainly from precuneus and cuneus to decode the velocity kinematics similarly in the subject-specific and template head models.
Collapse
Affiliation(s)
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
- BioTechMed Graz, Graz, Austria
- *Correspondence: Gernot R. Müller-Putz,
| |
Collapse
|
8
|
Deo DR, Rezaii P, Hochberg LR, M Okamura A, Shenoy KV, Henderson JM. Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex. IEEE TRANSACTIONS ON HAPTICS 2021; 14:762-775. [PMID: 33844633 PMCID: PMC8745032 DOI: 10.1109/toh.2021.3072615] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.
Collapse
|
9
|
Olsen S, Zhang J, Liang KF, Lam M, Riaz U, Kao JC. An artificial intelligence that increases simulated brain-computer interface performance. J Neural Eng 2021; 18. [PMID: 33978599 DOI: 10.1088/1741-2552/abfaaa] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Accepted: 04/22/2021] [Indexed: 12/14/2022]
Abstract
Objective.Brain-computer interfaces (BCIs) translate neural activity into control signals for assistive devices in order to help people with motor disabilities communicate effectively. In this work, we introduce a new BCI architecture that improves control of a BCI computer cursor to type on a virtual keyboard.Approach.Our BCI architecture incorporates an external artificial intelligence (AI) that beneficially augments the movement trajectories of the BCI. This AI-BCI leverages past user actions, at both long (100 s of seconds ago) and short (100 s of milliseconds ago) timescales, to modify the BCI's trajectories.Main results.We tested our AI-BCI in a closed-loop BCI simulator with nine human subjects performing a typing task. We demonstrate that our AI-BCI achieves: (1) categorically higher information communication rates, (2) quicker ballistic movements between targets, (3) improved precision control to 'dial in' on targets, and (4) more efficient movement trajectories. We further show that our AI-BCI increases performance across a wide control quality spectrum from poor to proficient control.Significance.This AI-BCI architecture, by increasing BCI performance across all key metrics evaluated, may increase the clinical viability of BCI systems.
Collapse
Affiliation(s)
- Sebastian Olsen
- Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90024, United States of America
| | - Jianwei Zhang
- Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90024, United States of America
| | - Ken-Fu Liang
- Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90024, United States of America
| | - Michelle Lam
- Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90024, United States of America
| | - Usama Riaz
- Department of Computer Science, University of California, Los Angeles, CA 90024, United States of America
| | - Jonathan C Kao
- Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90024, United States of America.,Neurosciences Program, University of California, Los Angeles, CA 90024, United States of America
| |
Collapse
|
10
|
|
11
|
Nason SR, Vaskov AK, Willsey MS, Welle EJ, An H, Vu PP, Bullard AJ, Nu CS, Kao JC, Shenoy KV, Jang T, Kim HS, Blaauw D, Patil PG, Chestek CA. A low-power band of neuronal spiking activity dominated by local single units improves the performance of brain-machine interfaces. Nat Biomed Eng 2020; 4:973-983. [PMID: 32719512 DOI: 10.1038/s41551-020-0591-0] [Citation(s) in RCA: 52] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Accepted: 06/24/2020] [Indexed: 12/18/2022]
Abstract
The large power requirement of current brain-machine interfaces is a major hindrance to their clinical translation. In basic behavioural tasks, the downsampled magnitude of the 300-1,000 Hz band of spiking activity can predict movement similarly to the threshold crossing rate (TCR) at 30 kilo-samples per second. However, the relationship between such a spiking-band power (SBP) and neural activity remains unclear, as does the capability of using the SBP to decode complicated behaviour. By using simulations of recordings of neural activity, here we show that the SBP is dominated by local single-unit spikes with spatial specificity comparable to or better than that of the TCR, and that the SBP correlates better with the firing rates of lower signal-to-noise-ratio units than the TCR. With non-human primates, in an online task involving the one-dimensional decoding of the movement of finger groups and in an offline two-dimensional cursor-control task, the SBP performed equally well or better than the TCR. The SBP may enhance the decoding performance of neural interfaces while enabling substantial cuts in power consumption.
Collapse
Affiliation(s)
- Samuel R Nason
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Alex K Vaskov
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI, USA
| | - Matthew S Willsey
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.,Department of Neurosurgery, University of Michigan Medical School, Ann Arbor, MI, USA
| | - Elissa J Welle
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Hyochan An
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - Philip P Vu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Autumn J Bullard
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Chrono S Nu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Jonathan C Kao
- Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA.,Neurosciences Program, University of California, Los Angeles, Los Angeles, CA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Department of Bioengineering, Stanford University, Stanford, CA, USA.,Department of Neurobiology, Stanford University, Stanford, CA, USA.,The Bio-X Program, Stanford University, Stanford, CA, USA.,Wu Tsai Neuroscience Institute, Stanford University, Stanford, CA, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| | - Taekwang Jang
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA.,Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
| | - Hun-Seok Kim
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - David Blaauw
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - Parag G Patil
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.,Department of Neurosurgery, University of Michigan Medical School, Ann Arbor, MI, USA.,Department of Neurology, University of Michigan Medical School, Ann Arbor, MI, USA.,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA
| | - Cynthia A Chestek
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA. .,Robotics Graduate Program, University of Michigan, Ann Arbor, MI, USA. .,Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA. .,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, USA.
| |
Collapse
|
12
|
Bullard AJ, Hutchison BC, Lee J, Chestek CA, Patil PG. Estimating Risk for Future Intracranial, Fully Implanted, Modular Neuroprosthetic Systems: A Systematic Review of Hardware Complications in Clinical Deep Brain Stimulation and Experimental Human Intracortical Arrays. Neuromodulation 2019; 23:411-426. [DOI: 10.1111/ner.13069] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2019] [Revised: 08/05/2019] [Accepted: 09/10/2019] [Indexed: 01/08/2023]
Affiliation(s)
- Autumn J. Bullard
- Department of Biomedical Engineering University of Michigan Ann Arbor MI USA
| | | | - Jiseon Lee
- Department of Biomedical Engineering University of Michigan Ann Arbor MI USA
| | - Cynthia A. Chestek
- Department of Biomedical Engineering University of Michigan Ann Arbor MI USA
- Department of Electrical Engineering and Computer Science University of Michigan Ann Arbor MI USA
| | - Parag G. Patil
- Department of Biomedical Engineering University of Michigan Ann Arbor MI USA
- Department of Neurosurgery University of Michigan Medical School Ann Arbor MI USA
| |
Collapse
|
13
|
Brandman DM, Hosman T, Saab J, Burkhart MC, Shanahan BE, Ciancibello JG, Sarma AA, Milstein DJ, Vargas-Irwin CE, Franco B, Kelemen J, Blabe C, Murphy BA, Young DR, Willett FR, Pandarinath C, Stavisky SD, Kirsch RF, Walter BL, Bolu Ajiboye A, Cash SS, Eskandar EN, Miller JP, Sweet JA, Shenoy KV, Henderson JM, Jarosiewicz B, Harrison MT, Simeral JD, Hochberg LR. Rapid calibration of an intracortical brain-computer interface for people with tetraplegia. J Neural Eng 2019; 15:026007. [PMID: 29363625 DOI: 10.1088/1741-2552/aa9ee7] [Citation(s) in RCA: 63] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
OBJECTIVE Brain-computer interfaces (BCIs) can enable individuals with tetraplegia to communicate and control external devices. Though much progress has been made in improving the speed and robustness of neural control provided by intracortical BCIs, little research has been devoted to minimizing the amount of time spent on decoder calibration. APPROACH We investigated the amount of time users needed to calibrate decoders and achieve performance saturation using two markedly different decoding algorithms: the steady-state Kalman filter, and a novel technique using Gaussian process regression (GP-DKF). MAIN RESULTS Three people with tetraplegia gained rapid closed-loop neural cursor control and peak, plateaued decoder performance within 3 min of initializing calibration. We also show that a BCI-naïve user (T5) was able to rapidly attain closed-loop neural cursor control with the GP-DKF using self-selected movement imagery on his first-ever day of closed-loop BCI use, acquiring a target 37 s after initiating calibration. SIGNIFICANCE These results demonstrate the potential for an intracortical BCI to be used immediately after deployment by people with paralysis, without the need for user learning or extensive system calibration.
Collapse
Affiliation(s)
- David M Brandman
- Neuroscience Graduate Program, Brown University, Providence, RI, United States of America. Department of Neuroscience, Brown University, Providence, RI, United States of America. Brown Institute for Brain Science, Brown University, Providence, RI, United States of America. Department of Surgery (Neurosurgery), Dalhousie University, Halifax, NS, Canada
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
14
|
Willett FR, Young DR, Murphy BA, Memberg WD, Blabe CH, Pandarinath C, Stavisky SD, Rezaii P, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral JD, Jarosiewicz B, Hochberg LR, Kirsch RF, Bolu Ajiboye A. Principled BCI Decoder Design and Parameter Selection Using a Feedback Control Model. Sci Rep 2019; 9:8881. [PMID: 31222030 PMCID: PMC6586941 DOI: 10.1038/s41598-019-44166-7] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2018] [Accepted: 03/04/2019] [Indexed: 02/01/2023] Open
Abstract
Decoders optimized offline to reconstruct intended movements from neural recordings sometimes fail to achieve optimal performance online when they are used in closed-loop as part of an intracortical brain-computer interface (iBCI). This is because typical decoder calibration routines do not model the emergent interactions between the decoder, the user, and the task parameters (e.g. target size). Here, we investigated the feasibility of simulating online performance to better guide decoder parameter selection and design. Three participants in the BrainGate2 pilot clinical trial controlled a computer cursor using a linear velocity decoder under different gain (speed scaling) and temporal smoothing parameters and acquired targets with different radii and distances. We show that a user-specific iBCI feedback control model can predict how performance changes under these different decoder and task parameters in held-out data. We also used the model to optimize a nonlinear speed scaling function for the decoder. When used online with two participants, it increased the dynamic range of decoded speeds and decreased the time taken to acquire targets (compared to an optimized standard decoder). These results suggest that it is feasible to simulate iBCI performance accurately enough to be useful for quantitative decoder optimization and design.
Collapse
Affiliation(s)
- Francis R Willett
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA. .,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA. .,Department of Neurosurgery, Stanford University, Stanford, California, USA. .,Department of Electrical Engineering, Stanford University, Stanford, California, USA.
| | - Daniel R Young
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - Brian A Murphy
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - William D Memberg
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - Christine H Blabe
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Chethan Pandarinath
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Sergey D Stavisky
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Department of Electrical Engineering, Stanford University, Stanford, California, USA
| | - Paymon Rezaii
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Jad Saab
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA
| | - Benjamin L Walter
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurology, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jennifer A Sweet
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurosurgery, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jonathan P Miller
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA.,Department of Neurosurgery, University Hospitals Case Medical Center, Cleveland, Ohio, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, California, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, 94305, California, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, California, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, 94305, California, USA.,Department of Bioengineering, Stanford University, Stanford, California, 94305, USA.,Department of Neurobiology, Stanford University, Stanford, California, 94305, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, California, 94305, USA.,Neurosciences Program, Stanford University, Stanford, California, 94305, USA.,Bio-X Program, Stanford University, Stanford, California, 94305, USA
| | - John D Simeral
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA.,Carney Institute for Brain Science, Brown University, Providence, Rhode Island, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Beata Jarosiewicz
- Department of Neurosurgery, Stanford University, Stanford, California, USA
| | - Leigh R Hochberg
- School of Engineering, Brown University, Providence, RI, USA.,Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA.,Department of Neurology, Harvard Medical School, Boston, Massachusetts, USA
| | - Robert F Kirsch
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| | - A Bolu Ajiboye
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, USA.,Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, Ohio, USA
| |
Collapse
|
15
|
Progress towards restoring upper limb movement and sensation through intracortical brain-computer interfaces. CURRENT OPINION IN BIOMEDICAL ENGINEERING 2018. [DOI: 10.1016/j.cobme.2018.11.005] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
16
|
Vaskov AK, Irwin ZT, Nason SR, Vu PP, Nu CS, Bullard AJ, Hill M, North N, Patil PG, Chestek CA. Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter. Front Neurosci 2018; 12:751. [PMID: 30455621 PMCID: PMC6231049 DOI: 10.3389/fnins.2018.00751] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2018] [Accepted: 09/28/2018] [Indexed: 01/01/2023] Open
Abstract
Objective: To date, many brain-machine interface (BMI) studies have developed decoding algorithms for neuroprostheses that provide users with precise control of upper arm reaches with some limited grasping capabilities. However, comparatively few have focused on quantifying the performance of precise finger control. Here we expand upon this work by investigating online control of individual finger groups. Approach: We have developed a novel training manipulandum for non-human primate (NHP) studies to isolate the movements of two specific finger groups: index and middle-ring-pinkie (MRP) fingers. We use this device in combination with the ReFIT (Recalibrated Feedback Intention-Trained) Kalman filter to decode the position of each finger group during a single degree of freedom task in two rhesus macaques with Utah arrays in motor cortex. The ReFIT Kalman filter uses a two-stage training approach that improves online control of upper arm tasks with substantial reductions in orbiting time, thus making it a logical first choice for precise finger control. Results: Both animals were able to reliably acquire fingertip targets with both index and MRP fingers, which they did in blocks of finger group specific trials. Decoding from motor signals online, the ReFIT Kalman filter reliably outperformed the standard Kalman filter, measured by bit rate, across all tested finger groups and movements by 31.0 and 35.2%. These decoders were robust when the manipulandum was removed during online control. While index finger movements and middle-ring-pinkie finger movements could be differentiated from each other with 81.7% accuracy across both subjects, the linear Kalman filter was not sufficient for decoding both finger groups together due to significant unwanted movement in the stationary finger, potentially due to co-contraction. Significance: To our knowledge, this is the first systematic and biomimetic separation of digits for continuous online decoding in a NHP as well as the first demonstration of the ReFIT Kalman filter improving the performance of precise finger decoding. These results suggest that novel nonlinear approaches, apparently not necessary for center out reaches or gross hand motions, may be necessary to achieve independent and precise control of individual fingers.
Collapse
Affiliation(s)
- Alex K Vaskov
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI, United States
| | - Zachary T Irwin
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Department of Neurology, University of Alabama, Birmingham, AL, United States
| | - Samuel R Nason
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Philip P Vu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Chrono S Nu
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Autumn J Bullard
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States
| | - Mackenna Hill
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Department of Biomedical Engineering, Duke University, Durham, NC, United States
| | - Naia North
- Mechanical Engineering Department, University of Michigan, Ann Arbor, MI, United States
| | - Parag G Patil
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Department of Neurosurgery, University of Michigan, Ann Arbor, MI, United States.,Department of Neurology, University of Michigan, Ann Arbor, MI, United States.,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, United States
| | - Cynthia A Chestek
- Robotics Graduate Program, University of Michigan, Ann Arbor, MI, United States.,Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States.,Neuroscience Graduate Program, University of Michigan, Ann Arbor, MI, United States.,Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
17
|
Quick KM, Mischel JL, Loughlin PJ, Batista AP. The critical stability task: quantifying sensory-motor control during ongoing movement in nonhuman primates. J Neurophysiol 2018; 120:2164-2181. [PMID: 29947593 DOI: 10.1152/jn.00300.2017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Everyday behaviors require that we interact with the environment, using sensory information in an ongoing manner to guide our actions. Yet, by design, many of the tasks used in primate neurophysiology laboratories can be performed with limited sensory guidance. As a consequence, our knowledge about the neural mechanisms of motor control is largely limited to the feedforward aspects of the motor command. To study the feedback aspects of volitional motor control, we adapted the critical stability task (CST) from the human performance literature (Jex H, McDonnell J, Phatak A. IEEE Trans Hum Factors Electron 7: 138-145, 1966). In the CST, our monkey subjects interact with an inherently unstable (i.e., divergent) virtual system and must generate sensory-guided actions to stabilize it about an equilibrium point. The difficulty of the CST is determined by a single parameter, which allows us to quantitatively establish the limits of performance in the task for different sensory feedback conditions. Two monkeys learned to perform the CST with visual or vibrotactile feedback. Performance was better under visual feedback, as expected, but both monkeys were able to utilize vibrotactile feedback alone to successfully perform the CST. We also observed changes in behavioral strategy as the task became more challenging. The CST will have value for basic science investigations of the neural basis of sensory-motor integration during ongoing actions, and it may also provide value for the design and testing of bidirectional brain computer interface systems. NEW & NOTEWORTHY Currently, most behavioral tasks used in motor neurophysiology studies require primates to make short-duration, stereotyped movements that do not necessitate sensory feedback. To improve our understanding of sensorimotor integration, and to engineer meaningful artificial sensory feedback systems for brain-computer interfaces, it is crucial to have a task that requires sensory feedback for good control. The critical stability task demands that sensory information be used to guide long-duration movements.
Collapse
Affiliation(s)
- Kristin M Quick
- Department of Bioengineering, University of Pittsburgh , Pittsburgh, Pennsylvania.,Center for the Neural Basis of Cognition , Pittsburgh, Pennsylvania
| | - Jessica L Mischel
- Department of Bioengineering, University of Pittsburgh , Pittsburgh, Pennsylvania.,Center for the Neural Basis of Cognition , Pittsburgh, Pennsylvania
| | - Patrick J Loughlin
- Department of Bioengineering, University of Pittsburgh , Pittsburgh, Pennsylvania.,Center for the Neural Basis of Cognition , Pittsburgh, Pennsylvania
| | - Aaron P Batista
- Department of Bioengineering, University of Pittsburgh , Pittsburgh, Pennsylvania.,Center for the Neural Basis of Cognition , Pittsburgh, Pennsylvania
| |
Collapse
|
18
|
Skomrock ND, Schwemmer MA, Ting JE, Trivedi HR, Sharma G, Bockbrader MA, Friedenberg DA. A Characterization of Brain-Computer Interface Performance Trade-Offs Using Support Vector Machines and Deep Neural Networks to Decode Movement Intent. Front Neurosci 2018; 12:763. [PMID: 30459542 PMCID: PMC6232881 DOI: 10.3389/fnins.2018.00763] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2018] [Accepted: 10/03/2018] [Indexed: 12/18/2022] Open
Abstract
Laboratory demonstrations of brain-computer interface (BCI) systems show promise for reducing disability associated with paralysis by directly linking neural activity to the control of assistive devices. Surveys of potential users have revealed several key BCI performance criteria for clinical translation of such a system. Of these criteria, high accuracy, short response latencies, and multi-functionality are three key characteristics directly impacted by the neural decoding component of the BCI system, the algorithm that translates neural activity into control signals. Building a decoder that simultaneously addresses these three criteria is complicated because optimizing for one criterion may lead to undesirable changes in the other criteria. Unfortunately, there has been little work to date to quantify how decoder design simultaneously affects these performance characteristics. Here, we systematically explore the trade-off between accuracy, response latency, and multi-functionality for discrete movement classification using two different decoding strategies-a support vector machine (SVM) classifier which represents the current state-of-the-art for discrete movement classification in laboratory demonstrations and a proposed deep neural network (DNN) framework. We utilized historical intracortical recordings from a human tetraplegic study participant, who imagined performing several different hand and finger movements. For both decoders, we found that response time increases (i.e., slower reaction) and accuracy decreases as the number of functions increases. However, we also found that both the increase of response times and the decline in accuracy with additional functions is less for the DNN than the SVM. We also show that data preprocessing steps can affect the performance characteristics of the two decoders in drastically different ways. Finally, we evaluated the performance of our tetraplegic participant using the DNN decoder in real-time to control functional electrical stimulation (FES) of his paralyzed forearm. We compared his performance to that of able-bodied participants performing the same task, establishing a quantitative target for ideal BCI-FES performance on this task. Cumulatively, these results help quantify BCI decoder performance characteristics relevant to potential users and the complex interactions between them.
Collapse
Affiliation(s)
- Nicholas D. Skomrock
- Advanced Analytics and Health Research, Battelle Memorial Institute, Columbus, OH, United States
| | - Michael A. Schwemmer
- Advanced Analytics and Health Research, Battelle Memorial Institute, Columbus, OH, United States
| | - Jordyn E. Ting
- Medical Devices and Neuromodulation, Battelle Memorial Institute, Columbus, OH, United States
| | - Hemang R. Trivedi
- Medical Devices and Neuromodulation, Battelle Memorial Institute, Columbus, OH, United States
| | - Gaurav Sharma
- Medical Devices and Neuromodulation, Battelle Memorial Institute, Columbus, OH, United States
| | - Marcia A. Bockbrader
- Neurological Institute, The Ohio State University, Columbus, OH, United States
- Department of Physical Medicine and Rehabilitation, The Ohio State University, Columbus, OH, United States
| | - David A. Friedenberg
- Advanced Analytics and Health Research, Battelle Memorial Institute, Columbus, OH, United States
| |
Collapse
|
19
|
Pandarinath C, O'Shea DJ, Collins J, Jozefowicz R, Stavisky SD, Kao JC, Trautmann EM, Kaufman MT, Ryu SI, Hochberg LR, Henderson JM, Shenoy KV, Abbott LF, Sussillo D. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat Methods 2018; 15:805-815. [PMID: 30224673 PMCID: PMC6380887 DOI: 10.1038/s41592-018-0109-9] [Citation(s) in RCA: 259] [Impact Index Per Article: 43.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2018] [Accepted: 06/28/2018] [Indexed: 01/28/2023]
Abstract
Neuroscience is experiencing a revolution in which simultaneous recording
of many thousands of neurons is revealing population dynamics that are not
apparent from single-neuron responses. This structure is typically extracted
from trial-averaged data, but deeper understanding requires studying
single-trial phenomena, which is challenging due to incomplete sampling of the
neural population, trial-to-trial variability, and fluctuations in action
potential timing. We introduce Latent Factor Analysis via Dynamical Systems
(LFADS), a deep learning method to infer latent dynamics from single-trial
neural spiking data. LFADS uses a nonlinear dynamical system to infer the
dynamics underlying observed spiking activity and to extract
‘de-noised’ single-trial firing rates. When applied to a variety
of monkey and human motor cortical datasets, LFADS predicts observed behavioral
variables with unprecedented accuracy, extracts precise estimates of neural
dynamics on single trials, infers perturbations to those dynamics that correlate
with behavioral choices, and combines data from non-overlapping recording
sessions spanning months to improve inference of underlying dynamics.
Collapse
Affiliation(s)
- Chethan Pandarinath
- Wallace H. Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, USA. .,Department of Neurosurgery, Emory University, Atlanta, GA, USA. .,Department of Neurosurgery, Stanford University, Stanford, CA, USA. .,Department of Electrical Engineering, Stanford University, Stanford, CA, USA. .,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA.
| | - Daniel J O'Shea
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Jasmine Collins
- Google AI, Google Inc., Mountain View, CA, USA.,University of California, Berkeley, Berkeley, CA, USA
| | - Rafal Jozefowicz
- Google AI, Google Inc., Mountain View, CA, USA.,OpenAI, San Francisco, CA, USA
| | - Sergey D Stavisky
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.,Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA.,Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Jonathan C Kao
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Department of Electrical Engineering, University of California, Los Angeles, Los Angeles, CA, USA
| | - Eric M Trautmann
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA
| | - Matthew T Kaufman
- Neurosciences Graduate Program, Stanford University, Stanford, CA, USA.,Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA
| | - Stephen I Ryu
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Department of Neurosurgery, Palo Alto Medical Foundation, Palo Alto, CA, USA
| | - Leigh R Hochberg
- VA RR&D Center for Neurorestoration and Neurotechnology, Veterans Affairs Medical Center, Providence, RI, USA.,Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.,School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - Jaimie M Henderson
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Krishna V Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA.,Department of Neurobiology, Stanford University, Stanford, CA, USA.,Department of Bioengineering, Stanford University, Stanford, CA, USA.,Bio-X Program, Stanford University, Stanford, CA, USA.,Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA
| | - L F Abbott
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA.,Department of Neuroscience, Columbia University, New York, NY, USA.,Department of Physiology and Cellular Biophysics, Columbia University, New York, NY, USA
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA. .,Stanford Neurosciences Institute, Stanford University, Stanford, CA, USA. .,Google AI, Google Inc., Mountain View, CA, USA.
| |
Collapse
|
20
|
Brandman DM, Burkhart MC, Kelemen J, Franco B, Harrison MT, Hochberg LR. Robust Closed-Loop Control of a Cursor in a Person with Tetraplegia using Gaussian Process Regression. Neural Comput 2018; 30:2986-3008. [PMID: 30216140 DOI: 10.1162/neco_a_01129] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
Intracortical brain computer interfaces can enable individuals with paralysis to control external devices through voluntarily modulated brain activity. Decoding quality has been previously shown to degrade with signal nonstationarities-specifically, the changes in the statistics of the data between training and testing data sets. This includes changes to the neural tuning profiles and baseline shifts in firing rates of recorded neurons, as well as nonphysiological noise. While progress has been made toward providing long-term user control via decoder recalibration, relatively little work has been dedicated to making the decoding algorithm more resilient to signal nonstationarities. Here, we describe how principled kernel selection with gaussian process regression can be used within a Bayesian filtering framework to mitigate the effects of commonly encountered nonstationarities. Given a supervised training set of (neural features, intention to move in a direction)-pairs, we use gaussian process regression to predict the intention given the neural data. We apply kernel embedding for each neural feature with the standard radial basis function. The multiple kernels are then summed together across each neural dimension, which allows the kernel to effectively ignore large differences that occur only in a single feature. The summed kernel is used for real-time predictions of the posterior mean and variance under a gaussian process framework. The predictions are then filtered using the discriminative Kalman filter to produce an estimate of the neural intention given the history of neural data. We refer to the multiple kernel approach combined with the discriminative Kalman filter as the MK-DKF. We found that the MK-DKF decoder was more resilient to nonstationarities frequently encountered in-real world settings yet provided similar performance to the currently used Kalman decoder. These results demonstrate a method by which neural decoding can be made more resistant to nonstationarities.
Collapse
Affiliation(s)
- David M Brandman
- Neuroscience Graduate Program, Department of Neuroscience, Carney Institute for Brain Science, and School of Engineering, Brown University, Providence, RI 02912, U.S.A.; and Department of Surgery (Neurosurgery), Dalhousie University, Halifax, NS B3H 347 Canada
| | - Michael C Burkhart
- Division of Applied Mathematics, Brown University, Providence, RI 02912, U.S.A.
| | - Jessica Kelemen
- Center for Neurotechnology and Neurorecovery, Neurology, Massachusetts General Hospital, Boston, MA 02114, U.S.A.
| | - Brian Franco
- Center for Neurotechnology and Neurorecovery, Neurology, Massachusetts General Hospital, Boston, MA 02114, U.S.A.
| | - Matthew T Harrison
- Division of Applied Mathematics, Brown University, Providence, RI 02912, U.S.A.
| | - Leigh R Hochberg
- Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI 02908; Carney Institute for Brain Science and School of Engineering, Brown University, Providence, RI 02912; Center for Neurotechnology and Neurorecovery, Neurology, Massachusetts General Hospital, Boston, MA 02114; and Neurology, Harvard Medical School, Boston, MA 02115, U.S.A.
| |
Collapse
|
21
|
Willett FR, Murphy BA, Young DR, Memberg WD, Blabe CH, Pandarinath C, Franco B, Saab J, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Simeral JD, Jarosiewicz B, Hochberg LR, Kirsch RF, Ajiboye AB. A Comparison of Intention Estimation Methods for Decoder Calibration in Intracortical Brain-Computer Interfaces. IEEE Trans Biomed Eng 2018; 65:2066-2078. [PMID: 29989927 PMCID: PMC6043406 DOI: 10.1109/tbme.2017.2783358] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023]
Abstract
OBJECTIVE Recent reports indicate that making better assumptions about the user's intended movement can improve the accuracy of decoder calibration for intracortical brain-computer interfaces. Several methods now exist for estimating user intent, including an optimal feedback control model, a piecewise-linear feedback control model, ReFIT, and other heuristics. Which of these methods yields the best decoding performance? METHODS Using data from the BrainGate2 pilot clinical trial, we measured how a steady-state velocity Kalman filter decoder was affected by the choice of intention estimation method. We examined three separate components of the Kalman filter: dimensionality reduction, temporal smoothing, and output gain (speed scaling). RESULTS The decoder's dimensionality reduction properties were largely unaffected by the intention estimation method. Decoded velocity vectors differed by <5% in terms of angular error and speed vs. target distance curves across methods. In contrast, the smoothing and gain properties of the decoder were greatly affected (> 50% difference in average values). Since the optimal gain and smoothing properties are task-specific (e.g. lower gains are better for smaller targets but worse for larger targets), no one method was better for all tasks. CONCLUSION Our results show that, when gain and smoothing differences are accounted for, current intention estimation methods yield nearly equivalent decoders and that simple models of user intent, such as a position error vector (target position minus cursor position), perform comparably to more elaborate models. Our results also highlight that simple differences in gain and smoothing properties have a large effect on online performance and can confound decoder comparisons.
Collapse
|
22
|
Cyborgs, Robots and Society: Implications for the Future of Society from Human Enhancement with In-The-Body Technologies. TECHNOLOGIES 2018. [DOI: 10.3390/technologies6020050] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
23
|
YOUNG D, WILLETT F, MEMBERG WD, MURPHY B, WALTER B, SWEET J, MILLER J, HOCHBERG LR, KIRSCH RF, AJIBOYE AB. Signal processing methods for reducing artifacts in microelectrode brain recordings caused by functional electrical stimulation. J Neural Eng 2018; 15:026014. [PMID: 29199642 PMCID: PMC5818316 DOI: 10.1088/1741-2552/aa9ee8] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
OBJECTIVE Functional electrical stimulation (FES) is a promising technology for restoring movement to paralyzed limbs. Intracortical brain-computer interfaces (iBCIs) have enabled intuitive control over virtual and robotic movements, and more recently over upper extremity FES neuroprostheses. However, electrical stimulation of muscles creates artifacts in intracortical microelectrode recordings that could degrade iBCI performance. Here, we investigate methods for reducing the cortically recorded artifacts that result from peripheral electrical stimulation. APPROACH One participant in the BrainGate2 pilot clinical trial had two intracortical microelectrode arrays placed in the motor cortex, and thirty-six stimulating intramuscular electrodes placed in the muscles of the contralateral limb. We characterized intracortically recorded electrical artifacts during both intramuscular and surface stimulation. We compared the performance of three artifact reduction methods: blanking, common average reference (CAR) and linear regression reference (LRR), which creates channel-specific reference signals, composed of weighted sums of other channels. MAIN RESULTS Electrical artifacts resulting from surface stimulation were 175 × larger than baseline neural recordings (which were 110 µV peak-to-peak), while intramuscular stimulation artifacts were only 4 × larger. The artifact waveforms were highly consistent across electrodes within each array. Application of LRR reduced artifact magnitudes to less than 10 µV and largely preserved the original neural feature values used for decoding. Unmitigated stimulation artifacts decreased iBCI decoding performance, but performance was almost completely recovered using LRR, which outperformed CAR and blanking and extracted useful neural information during stimulation artifact periods. SIGNIFICANCE The LRR method was effective at reducing electrical artifacts resulting from both intramuscular and surface FES, and almost completely restored iBCI decoding performance (>90% recovery for surface stimulation and full recovery for intramuscular stimulation). The results demonstrate that FES-induced artifacts can be easily mitigated in FES + iBCI systems by using LRR for artifact reduction, and suggest that the LRR method may also be useful in other noise reduction applications.
Collapse
Affiliation(s)
- D. YOUNG
- Case Western Reserve Univ., Cleveland, OH
- FES Ctr. of Excellence, Rehab. R&D Service, Louis Stokes Cleveland Dept. of VA Med. Ctr., Cleveland, OH
| | - F. WILLETT
- Case Western Reserve Univ., Cleveland, OH
- FES Ctr. of Excellence, Rehab. R&D Service, Louis Stokes Cleveland Dept. of VA Med. Ctr., Cleveland, OH
- Dept of Neurosurgery, Stanford University, Stanford, CA
| | - W. D. MEMBERG
- Case Western Reserve Univ., Cleveland, OH
- FES Ctr. of Excellence, Rehab. R&D Service, Louis Stokes Cleveland Dept. of VA Med. Ctr., Cleveland, OH
| | - B. MURPHY
- Case Western Reserve Univ., Cleveland, OH
| | - B. WALTER
- Neurol., UH Cleveland Med. Ctr., Cleveland, OH
- Neurol., CWRU Sch. of Med., Cleveland, OH
| | - J. SWEET
- Neurosurg., UH Cleveland Med. Ctr., Cleveland, OH
- Neurolog. Surgery, CWRU Sch. of Med., Cleveland, OH
| | - J. MILLER
- Neurosurg., UH Cleveland Med. Ctr., Cleveland, OH
- Neurolog. Surgery, CWRU Sch. of Med., Cleveland, OH
| | - L. R. HOCHBERG
- Ctr. for Neurorestoration and Neurotechnology, Rehab. R&D Service, Dept. of VA Medical Center, Providence RI
- Sch. of Engin., Brown Univ., Providence, RI
- Neurol., Massachusetts Gen. Hosp., Boston, MA
- Neurol., Harvard Med. Sch., Boston, MA
- Inst. For Brain Sci., Brown Univ., Providence, RI
| | - R. F. KIRSCH
- Case Western Reserve Univ., Cleveland, OH
- FES Ctr. of Excellence, Rehab. R&D Service, Louis Stokes Cleveland Dept. of VA Med. Ctr., Cleveland, OH
| | - A. B. AJIBOYE
- Case Western Reserve Univ., Cleveland, OH
- FES Ctr. of Excellence, Rehab. R&D Service, Louis Stokes Cleveland Dept. of VA Med. Ctr., Cleveland, OH
| |
Collapse
|
24
|
|
25
|
Brandman DM, Cash SS, Hochberg LR. Review: Human Intracortical Recording and Neural Decoding for Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng 2017; 25:1687-1696. [PMID: 28278476 PMCID: PMC5815832 DOI: 10.1109/tnsre.2017.2677443] [Citation(s) in RCA: 50] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
Brain-computer interfaces (BCIs) use neural information recorded from the brain for the voluntary control of external devices. The development of BCI systems has largely focused on improving functional independence for individuals with severe motor impairments, including providing tools for communication and mobility. In this review, we describe recent advances in intracortical BCI technology and provide potential directions for further research.
Collapse
|
26
|
Prochazka A. Neurophysiology and neural engineering: a review. J Neurophysiol 2017; 118:1292-1309. [PMID: 28566462 PMCID: PMC5558026 DOI: 10.1152/jn.00149.2017] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2017] [Revised: 05/30/2017] [Accepted: 05/30/2017] [Indexed: 12/19/2022] Open
Abstract
Neurophysiology is the branch of physiology concerned with understanding the function of neural systems. Neural engineering (also known as neuroengineering) is a discipline within biomedical engineering that uses engineering techniques to understand, repair, replace, enhance, or otherwise exploit the properties and functions of neural systems. In most cases neural engineering involves the development of an interface between electronic devices and living neural tissue. This review describes the origins of neural engineering, the explosive development of methods and devices commencing in the late 1950s, and the present-day devices that have resulted. The barriers to interfacing electronic devices with living neural tissues are many and varied, and consequently there have been numerous stops and starts along the way. Representative examples are discussed. None of this could have happened without a basic understanding of the relevant neurophysiology. I also consider examples of how neural engineering is repaying the debt to basic neurophysiology with new knowledge and insight.
Collapse
Affiliation(s)
- Arthur Prochazka
- Department of Physiology, University of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|
27
|
Willett FR, Murphy BA, Memberg WD, Blabe CH, Pandarinath C, Walter BL, Sweet JA, Miller JP, Henderson JM, Shenoy KV, Hochberg LR, Kirsch RF, Ajiboye AB. Signal-independent noise in intracortical brain-computer interfaces causes movement time properties inconsistent with Fitts' law. J Neural Eng 2017; 14:026010. [PMID: 28177925 DOI: 10.1088/1741-2552/aa5990] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
OBJECTIVE Do movements made with an intracortical BCI (iBCI) have the same movement time properties as able-bodied movements? Able-bodied movement times typically obey Fitts' law: [Formula: see text] (where MT is movement time, D is target distance, R is target radius, and [Formula: see text] are parameters). Fitts' law expresses two properties of natural movement that would be ideal for iBCIs to restore: (1) that movement times are insensitive to the absolute scale of the task (since movement time depends only on the ratio [Formula: see text]) and (2) that movements have a large dynamic range of accuracy (since movement time is logarithmically proportional to [Formula: see text]). APPROACH Two participants in the BrainGate2 pilot clinical trial made cortically controlled cursor movements with a linear velocity decoder and acquired targets by dwelling on them. We investigated whether the movement times were well described by Fitts' law. MAIN RESULTS We found that movement times were better described by the equation [Formula: see text], which captures how movement time increases sharply as the target radius becomes smaller, independently of distance. In contrast to able-bodied movements, the iBCI movements we studied had a low dynamic range of accuracy (absence of logarithmic proportionality) and were sensitive to the absolute scale of the task (small targets had long movement times regardless of the [Formula: see text] ratio). We argue that this relationship emerges due to noise in the decoder output whose magnitude is largely independent of the user's motor command (signal-independent noise). Signal-independent noise creates a baseline level of variability that cannot be decreased by trying to move slowly or hold still, making targets below a certain size very hard to acquire with a standard decoder. SIGNIFICANCE The results give new insight into how iBCI movements currently differ from able-bodied movements and suggest that restoring a Fitts' law-like relationship to iBCI movements may require non-linear decoding strategies.
Collapse
Affiliation(s)
- Francis R Willett
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States of America. Louis Stokes Cleveland Department of Veterans Affairs Medical Center, FES Center of Excellence, Rehab. R&D Service, Cleveland, OH, United States of America
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
28
|
Huggins JE, Guger C, Ziat M, Zander TO, Taylor D, Tangermann M, Soria-Frisch A, Simeral J, Scherer R, Rupp R, Ruffini G, Robinson DKR, Ramsey NF, Nijholt A, Müller-Putz G, McFarland DJ, Mattia D, Lance BJ, Kindermans PJ, Iturrate I, Herff C, Gupta D, Do AH, Collinger JL, Chavarriaga R, Chase SM, Bleichner MG, Batista A, Anderson CW, Aarnoutse EJ. Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future. BRAIN-COMPUTER INTERFACES 2017; 4:3-36. [PMID: 29152523 PMCID: PMC5693371 DOI: 10.1080/2326263x.2016.1275488] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.
Collapse
Affiliation(s)
- Jane E. Huggins
- Department of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan, USA
| | - Christoph Guger
- G.Tec Medical Engineering GmbH, Guger Technologies OG, Schiedlberg, Austria
| | - Mounia Ziat
- Psychology Department, Northern Michigan University, Marquette, MI, USA
| | - Thorsten O. Zander
- Team PhyPA, Biological Psychology and Neuroergonomics, Technical University of Berlin, Berlin, Germany
| | | | - Michael Tangermann
- Cluster of Excellence BrainLinks-BrainTools, University of Freiburg, Germany
| | | | - John Simeral
- Ctr. For Neurorestoration and Neurotechnology, Rehab. R&D Service, Dept. of VA Medical Center, School of Engineering, Brown University, Providence, RI, USA
| | - Reinhold Scherer
- Institute of Neural Engineering, BCI- Lab, Graz University of Technology, Graz, Austria
| | - Rüdiger Rupp
- Section Experimental Neurorehabilitation, Spinal Cord Injury Center, University Hospital in Heidelberg, Heidelberg, Germany
| | - Giulio Ruffini
- Neuroscience Business Unit, Starlab Barcelona SLU, Barcelona, Spain
- Neuroelectrics Inc., Boston, USA
| | - Douglas K. R. Robinson
- Institute: Laboratoire Interdisciplinaire Sciences Innovations Sociétés (LISIS), Université Paris-Est Marne-la-Vallée, MARNE-LA-VALLÉE, France
| | - Nick F. Ramsey
- Dept Neurology & Neurosurgery, Brain Center Rudolf Magnus, University Medical Center Utrecht, University of Utrecht, Utrecht, Netherlands
| | - Anton Nijholt
- Faculty EEMCS, Enschede, University of Twente, The Netherlands & Imagineering Institute, Iskandar, Malaysia
| | - Gernot Müller-Putz
- Institute of Neural Engineering, BCI- Lab, Graz University of Technology, Graz, Austria
| | - Dennis J. McFarland
- New York State Department of Health, National Center for Adaptive Neurotechnologies, Wadsworth Center, Albany, New York USA
| | - Donatella Mattia
- Clinical Neurophysiology, Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, IRCCS, Rome, Italy
| | - Brent J. Lance
- Human Research and Engineering Directorate, U.S. Army Research Laboratory, Aberdeen Proving Ground, Aberdeen, MD USA
| | | | - Iñaki Iturrate
- Defitech Chair in Brain–machine Interface (CNBI), Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, EPFL-STI-CNBI, Campus Biotech H4, Geneva, Switzerland
| | - Christian Herff
- Cognitive Systems Lab, University of Bremen, Bremen, Germany
| | - Disha Gupta
- Brain Mind Research Inst, Weill Cornell Medical College, Early Brain Injury and Recovery Lab, Burke Medical Research Inst, White Plains, New York, USA
| | - An H. Do
- Department of Neurology, UC Irvine Brain Computer Interface Lab, University of California, Irvine, CA, USA
| | - Jennifer L. Collinger
- Department of Physical Medicine and Rehabilitation, Department of Veterans Affairs, VA Pittsburgh Healthcare System, University of Pittsburgh, Pittsburgh, PA, USA
| | - Ricardo Chavarriaga
- Defitech Chair in Brain–machine Interface (CNBI), Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, EPFL-STI-CNBI, Campus Biotech H4, Geneva, Switzerland
| | - Steven M. Chase
- Center for the Neural Basis of Cognition and Department Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Martin G. Bleichner
- Neuropsychology Lab, Department of Psychology, European Medical School, Cluster of Excellence Hearing4all, University of Oldenburg, Oldenburg, Germany
| | - Aaron Batista
- Department of Bioengineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA USA
| | - Charles W. Anderson
- Department of Computer Science, Colorado State University, Fort Collins, CO USA
| | - Erik J. Aarnoutse
- Brain Center Rudolf Magnus, Dept Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|