1
|
Canny E, Vansteensel MJ, van der Salm SMA, Müller-Putz GR, Berezutskaya J. Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome. J Neuroeng Rehabil 2023; 20:157. [PMID: 37980536 PMCID: PMC10656959 DOI: 10.1186/s12984-023-01272-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 10/23/2023] [Indexed: 11/20/2023] Open
Abstract
Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.
Collapse
Affiliation(s)
- Evan Canny
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Mariska J Vansteensel
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Sandra M A van der Salm
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Gernot R Müller-Putz
- Institute of Neural Engineering, Laboratory of Brain-Computer Interfaces, Graz University of Technology, Graz, Austria
| | - Julia Berezutskaya
- Department of Neurology and Neurosurgery, Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands.
| |
Collapse
|
2
|
Yan T, Suzuki K, Kameda S, Maeda M, Mihara T, Hirata M. Chronic subdural electrocorticography in nonhuman primates by an implantable wireless device for brain-machine interfaces. Front Neurosci 2023; 17:1260675. [PMID: 37841689 PMCID: PMC10568031 DOI: 10.3389/fnins.2023.1260675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 09/12/2023] [Indexed: 10/17/2023] Open
Abstract
Background Subdural electrocorticography (ECoG) signals have been proposed as a stable, good-quality source for brain-machine interfaces (BMIs), with a higher spatial and temporal resolution than electroencephalography (EEG). However, long-term implantation may lead to chronic inflammatory reactions and connective tissue encapsulation, resulting in a decline in signal recording quality. However, no study has reported the effects of the surrounding tissue on signal recording and device functionality thus far. Methods In this study, we implanted a wireless recording device with a customized 32-electrode-ECoG array subdurally in two nonhuman primates for 15 months. We evaluated the neural activities recorded from and wirelessly transmitted to the devices and the chronic tissue reactions around the electrodes. In addition, we measured the gain factor of the newly formed ventral fibrous tissue in vivo. Results Time-frequency analyses of the acute and chronic phases showed similar signal features. The average root mean square voltage and power spectral density showed relatively stable signal quality after chronic implantation. Histological examination revealed thickening of the reactive tissue around the electrode array; however, no evident inflammation in the cortex. From gain factor analysis, we found that tissue proliferation under electrodes reduced the amplitude power of signals. Conclusion This study suggests that subdural ECoG may provide chronic signal recordings for future clinical applications and neuroscience research. This study also highlights the need to reduce proliferation of reactive tissue ventral to the electrodes to enhance long-term stability.
Collapse
Affiliation(s)
- Tianfang Yan
- Department of Neurological Diagnosis and Restoration, Osaka University Graduate School of Medicine, Suita, Japan
| | | | - Seiji Kameda
- Department of Neurological Diagnosis and Restoration, Osaka University Graduate School of Medicine, Suita, Japan
| | - Masashi Maeda
- Candidate Discovery Science Labs, Astellas Pharma Inc., Tokyo, Japan
| | - Takuma Mihara
- Candidate Discovery Science Labs, Astellas Pharma Inc., Tokyo, Japan
| | - Masayuki Hirata
- Department of Neurological Diagnosis and Restoration, Osaka University Graduate School of Medicine, Suita, Japan
- Department of Neurosurgery, Osaka University Graduate School of Medicine, Suita, Japan
- Global Center for Medical Engineering and Informatics, Osaka University, Suita, Japan
| |
Collapse
|
3
|
Maiseli B, Abdalla AT, Massawe LV, Mbise M, Mkocha K, Nassor NA, Ismail M, Michael J, Kimambo S. Brain-computer interface: trend, challenges, and threats. Brain Inform 2023; 10:20. [PMID: 37540385 PMCID: PMC10403483 DOI: 10.1186/s40708-023-00199-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2023] [Accepted: 07/01/2023] [Indexed: 08/05/2023] Open
Abstract
Brain-computer interface (BCI), an emerging technology that facilitates communication between brain and computer, has attracted a great deal of research in recent years. Researchers provide experimental results demonstrating that BCI can restore the capabilities of physically challenged people, hence improving the quality of their lives. BCI has revolutionized and positively impacted several industries, including entertainment and gaming, automation and control, education, neuromarketing, and neuroergonomics. Notwithstanding its broad range of applications, the global trend of BCI remains lightly discussed in the literature. Understanding the trend may inform researchers and practitioners on the direction of the field, and on where they should invest their efforts more. Noting this significance, we have analyzed 25,336 metadata of BCI publications from Scopus to determine advancement of the field. The analysis shows an exponential growth of BCI publications in China from 2019 onwards, exceeding those from the United States that started to decline during the same period. Implications and reasons for this trend are discussed. Furthermore, we have extensively discussed challenges and threats limiting exploitation of BCI capabilities. A typical BCI architecture is hypothesized to address two prominent BCI threats, privacy and security, as an attempt to make the technology commercially viable to the society.
Collapse
Affiliation(s)
- Baraka Maiseli
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania.
| | - Abdi T Abdalla
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - Libe V Massawe
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - Mercy Mbise
- Department of Computer Science and Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - Khadija Mkocha
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - Nassor Ally Nassor
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - Moses Ismail
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - James Michael
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| | - Samwel Kimambo
- Department of Electronics and Telecommunications Engineering, College of Information and Communication Technologies, University of Dar es Salaam, 14113, Dar es Salaam, Tanzania
| |
Collapse
|
4
|
Lorach H, Galvez A, Spagnolo V, Martel F, Karakas S, Intering N, Vat M, Faivre O, Harte C, Komi S, Ravier J, Collin T, Coquoz L, Sakr I, Baaklini E, Hernandez-Charpak SD, Dumont G, Buschman R, Buse N, Denison T, van Nes I, Asboth L, Watrin A, Struber L, Sauter-Starace F, Langar L, Auboiroux V, Carda S, Chabardes S, Aksenova T, Demesmaeker R, Charvet G, Bloch J, Courtine G. Walking naturally after spinal cord injury using a brain-spine interface. Nature 2023; 618:126-133. [PMID: 37225984 PMCID: PMC10232367 DOI: 10.1038/s41586-023-06094-5] [Citation(s) in RCA: 88] [Impact Index Per Article: 88.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 04/17/2023] [Indexed: 05/26/2023]
Abstract
A spinal cord injury interrupts the communication between the brain and the region of the spinal cord that produces walking, leading to paralysis1,2. Here, we restored this communication with a digital bridge between the brain and spinal cord that enabled an individual with chronic tetraplegia to stand and walk naturally in community settings. This brain-spine interface (BSI) consists of fully implanted recording and stimulation systems that establish a direct link between cortical signals3 and the analogue modulation of epidural electrical stimulation targeting the spinal cord regions involved in the production of walking4-6. A highly reliable BSI is calibrated within a few minutes. This reliability has remained stable over one year, including during independent use at home. The participant reports that the BSI enables natural control over the movements of his legs to stand, walk, climb stairs and even traverse complex terrains. Moreover, neurorehabilitation supported by the BSI improved neurological recovery. The participant regained the ability to walk with crutches overground even when the BSI was switched off. This digital bridge establishes a framework to restore natural control of movement after paralysis.
Collapse
Affiliation(s)
- Henri Lorach
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Andrea Galvez
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Valeria Spagnolo
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Felix Martel
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, Grenoble, France
| | - Serpil Karakas
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, Grenoble, France
| | - Nadine Intering
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Molywan Vat
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Olivier Faivre
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, Grenoble, France
| | - Cathal Harte
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Salif Komi
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Jimmy Ravier
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Thibault Collin
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Laure Coquoz
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Icare Sakr
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Edeny Baaklini
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Sergio Daniel Hernandez-Charpak
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Gregory Dumont
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | | | | | - Tim Denison
- Medtronic, Minneapolis, MN, USA
- Department of Engineering Science, University of Oxford, Oxford, UK
| | - Ilse van Nes
- Department of Rehabilitation, Sint Maartenskliniek, Nijmegen, the Netherlands
| | - Leonie Asboth
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | | | - Lucas Struber
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, Grenoble, France
| | | | - Lilia Langar
- Univ. Grenoble Alpes, CHU Grenoble Alpes, Clinatec, Grenoble, France
| | | | - Stefano Carda
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
| | - Stephan Chabardes
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, Grenoble, France
- Univ. Grenoble Alpes, CHU Grenoble Alpes, Clinatec, Grenoble, France
| | | | - Robin Demesmaeker
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland
| | | | - Jocelyne Bloch
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland.
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland.
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland.
| | - Grégoire Courtine
- NeuroX Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland.
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland.
- NeuroRestore, Defitech Center for Interventional Neurotherapies, EPFL/CHUV/UNIL, Lausanne, Switzerland.
| |
Collapse
|
5
|
Kim HJ, Ho JS. Wireless interfaces for brain neurotechnologies. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2022; 380:20210020. [PMID: 35658679 DOI: 10.1098/rsta.2021.0020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 12/13/2021] [Indexed: 06/15/2023]
Abstract
Wireless interfaces enable brain-implanted devices to remotely interact with the external world. They are critical components in modern research and clinical neurotechnologies and play a central role in determining their overall size, lifetime and functionality. Wireless interfaces use a wide range of modalities-including radio-frequency fields, acoustic waves and light-to transfer energy and data to and from an implanted device. These forms of energy interact with living tissue through distinct mechanisms and therefore lead to systems with vastly different form factors, operating characteristics, and safety considerations. This paper reviews recent advances in the development of wireless interfaces for brain neurotechnologies. We summarize the requirements that state-of-the-art brain-implanted devices impose on the wireless interface, and discuss the working principles and applications of wireless interfaces based on each modality. We also investigate challenges associated with wireless brain neurotechnologies and discuss emerging solutions permitted by recent developments in electrical engineering and materials science. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.
Collapse
Affiliation(s)
- Han-Joon Kim
- Department of Electrical and Computer Engineering National University of Singapore, Queenstown, Singapore
| | - John S Ho
- Department of Electrical and Computer Engineering National University of Singapore, Queenstown, Singapore
- The N.1 Institute for Health National University of Singapore, Queenstown, Singapore
- Institute for Health Innovation and Technology, National University of Singapore, Queenstown, Singapore
| |
Collapse
|
6
|
Flint RD, Li Y, Wang P, Vaidya M, Barry A, Ghassemi M, Tomic G, Brkic N, Ripley D, Liu C, Kamper D, Do A, Slutzky MW. Noninvasively recorded high-gamma signals improve synchrony of force feedback in a novel neurorehabilitation brain-machine interface for brain injury. J Neural Eng 2022; 19. [PMID: 35576911 PMCID: PMC9728942 DOI: 10.1088/1741-2552/ac7004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2021] [Accepted: 05/16/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Brain injury is the leading cause of long-term disability worldwide, often resulting in impaired hand function. Brain-machine interfaces (BMIs) offer a potential way to improve hand function. BMIs often target replacing lost function, but may also be employed in neurorehabilitation (nrBMI) by facilitating neural plasticity and functional recovery. Here, we report a novel nrBMI capable of acquiring high-γ (70-115 Hz) information through a unique post-TBI hemicraniectomy window model, and delivering sensory feedback that is synchronized with, and proportional to, intended grasp force. APPROACH We developed the nrBMI to use electroencephalogram recorded over a hemicraniectomy (hEEG) in individuals with traumatic brain injury (TBI). The nrBMI empowered users to exert continuous, proportional control of applied force, and provided continuous force feedback. We report the results of an initial testing group of three human participants with TBI, along with a control group of three skull- and motor-intact volunteers. MAIN RESULTS All participants controlled the nrBMI successfully, with high initial success rates (2 of 6 participants) or performance that improved over time (4 of 6 participants). We observed high-γ modulation with force intent in hEEG but not skull-intact EEG. Most significantly, we found that high-γ control significantly improved the timing synchronization between neural modulation onset and nrBMI output/haptic feedback (compared to low-frequency nrBMI control). SIGNIFICANCE These proof-of-concept results show that high-γ nrBMIs can be used by individuals with impaired ability to control force (without immediately resorting to invasive signals like ECoG). Of note, the nrBMI includes a parameter to change the fraction of control shared between decoded intent and volitional force, to adjust for recovery progress. The improved synchrony between neural modulations and force control for high-γ signals is potentially important for maximizing the ability of nrBMIs to induce plasticity in neural circuits. Inducing plasticity is critical to functional recovery after brain injury.
Collapse
Affiliation(s)
- Robert D Flint
- Department of Physiology, Northwestern University, Northwestern University, The Feinberg School of Medicine, 303 E. Chicago Ave. , Chicago, IL 60611, USA, Chicago, Illinois, 60611, UNITED STATES
| | - Yongcheng Li
- University of California Irvine, 402 E Peltason Dr, Irvine, California, 92617, UNITED STATES
| | - Po Wang
- University of California Irvine, 402 E Peltason Dr, Irvine, California, 92617, UNITED STATES
| | - Mukta Vaidya
- Northwestern University Feinberg School of Medicine, 320 E Superior St, Chicago, Illinois, 60611-3008, UNITED STATES
| | - Alex Barry
- Shirley Ryan AbilityLab, 355 E Erie St, Chicago, Illinois, 60611-2654, UNITED STATES
| | - Mohammad Ghassemi
- North Carolina State University, Engineering Building III, 4130, Raleigh, North Carolina, 27695, UNITED STATES
| | - Goran Tomic
- Department of Physiology, Northwestern University, Northwestern University, The Feinberg School of Medicine, 303 E. Chicago Ave. , Chicago, IL 60611, USA, Chicago, Illinois, 60611, UNITED STATES
| | - Nenad Brkic
- Shirley Ryan AbilityLab, 355 E Erie St, Chicago, Illinois, 60611-2654, UNITED STATES
| | - David Ripley
- Shirley Ryan AbilityLab, 355 E Erie St, Chicago, Illinois, 60611-2654, UNITED STATES
| | - Charles Liu
- University of California Irvine, 402 E Peltason Dr, Irvine, California, 92617, UNITED STATES
| | - Derek Kamper
- North Carolina State University, Engineering Building III, 4130, Raleigh, North Carolina, 27695, UNITED STATES
| | - An Do
- University of California Irvine, 402 E Peltason Dr, Irvine, California, 92617, UNITED STATES
| | - Marc W Slutzky
- Department of Physiology, Northwestern University Medical School, 303 East Chicago Avenue, Chicago, Illinois, 60611, UNITED STATES
| |
Collapse
|
7
|
Śliwowski M, Martin M, Souloumiac A, Blanchart P, Aksenova T. Decoding ECoG signal into 3D hand translation using deep learning. J Neural Eng 2022; 19. [PMID: 35287119 DOI: 10.1088/1741-2552/ac5d69] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 03/14/2022] [Indexed: 12/29/2022]
Abstract
Objective.Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. BCIs would potentially compensate for arm and hand function loss, which is the top priority for individuals with tetraplegia. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal features and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.Approach.In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity.Main results.Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.Significance.This study shows that DL-based models could increase the accuracy of BCI systems in the case of 3D hand translation prediction in a tetraplegic subject.
Collapse
Affiliation(s)
- Maciej Śliwowski
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France.,Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France
| | - Matthieu Martin
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| | | | | | - Tetiana Aksenova
- Université Grenoble Alpes, CEA, LETI, Clinatec, F-38000 Grenoble, France
| |
Collapse
|
8
|
Moly A, Costecalde T, Martel F, Martin M, Larzabal C, Karakas S, Verney A, Charvet G, Chabardès S, Benabid AL, Aksenova T. An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic. J Neural Eng 2022; 19. [PMID: 35234665 DOI: 10.1088/1741-2552/ac59a0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 03/01/2022] [Indexed: 11/12/2022]
Abstract
OBJECTIVE The article aims at addressing 2 challenges to step motor BCI out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration. APPROACH Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based Recursive Exponentially Weighted Markov-Switching multi-Linear Model (REW-MSLM) decoder is proposed. REW-MSLM uses a Mixture of Expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a "gating" model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action. MAIN RESULTS Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of 6 months (without decoder recalibration) 8-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated. SIGNIFICANCE Based on the long-term (>36 months) chronic bilateral epidural ECoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behaviour (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.
Collapse
Affiliation(s)
- Alexandre Moly
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Thomas Costecalde
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Félix Martel
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Matthieu Martin
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des Martyrs, Grenoble, 38000, FRANCE
| | - Christelle Larzabal
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Serpil Karakas
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Alexandre Verney
- Université Paris-Saclay, Palaiseau, Palaiseau, Île-de-France, 91120, FRANCE
| | - Guillaume Charvet
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| | - Stephan Chabardès
- CHU Grenoble Alpes, Boulevard de la Chantourne, La Tronche, Auvergne-Rhône-Alpes, 38700, FRANCE
| | - Alim-Louis Benabid
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, 17, avenue des Martyrs, Grenoble, 38000, FRANCE
| | - Tatiana Aksenova
- Univ. Grenoble Alpes, CEA, LETI, Clinatec,, 17 av. des martyrs, Grenoble, Auvergne-Rhône-Alpes, 38000, FRANCE
| |
Collapse
|
9
|
Siegel A, Sauter-Starace F, Laporte S. Insights into the mechanical interaction between an active cranial implant and the skull subjected to moderate impact loadings. J Mech Behav Biomed Mater 2022; 126:105048. [PMID: 34999489 DOI: 10.1016/j.jmbbm.2021.105048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Revised: 12/06/2021] [Accepted: 12/12/2021] [Indexed: 11/18/2022]
Abstract
In the context of cochlear implants, which are now widely used, and innovative active devices, the cranial implantation of electronic devices raises new questions about the mechanical interactions between the implant and the skull. The aim of this study was to build a methodology using experimental data and numerical simulations to evaluate the mechanical interactions between the skull and the WIMAGINE® active cranial implant intended for use for tetraplegic patients. A finite element model of the implant housing and a simplified model of the three-layered skull were developed. 2.5 J-hammer impact tests were performed on implant housings and ovine cadaver heads for model calibration. The two models were then combined to analyze the interactions between the skull and the implant and compared against impact tests. The implant dissipates a certain amount of the impact energy which could be a parameter to include in implant design in addition to the implant integrity, tending to increase the implant stiffness. The non-implanted as well as the implanted lamb heads demonstrated an overall good resistance to the impact tests. The models correlated well with the experimental data, and improvements of the model through more realistic geometry (CT-scans) and more complex material behavior could now be implemented. Such a model could then be used with human head geometries and help for future implant design optimizations using numerical models of the implant-skull and even implant-head complex.
Collapse
Affiliation(s)
- Alice Siegel
- Univ. Grenoble Alpes, CEA, LETI, Clinatec, 38000, Grenoble, France; Arts et Métiers Institute of Technology, Université Sorbonne Paris Nord, IBHGC - Institut de Biomécanique Humaine Georges Charpak, HESAM Université, F-75013, Paris, France.
| | | | - Sébastien Laporte
- Arts et Métiers Institute of Technology, Université Sorbonne Paris Nord, IBHGC - Institut de Biomécanique Humaine Georges Charpak, HESAM Université, F-75013, Paris, France
| |
Collapse
|
10
|
Lorach H, Charvet G, Bloch J, Courtine G. Brain-spine interfaces to reverse paralysis. Natl Sci Rev 2022; 9:nwac009. [PMID: 36196116 PMCID: PMC9522392 DOI: 10.1093/nsr/nwac009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 11/23/2021] [Accepted: 01/14/2022] [Indexed: 11/24/2022] Open
Affiliation(s)
- Henri Lorach
- Center for Neuroprosthetics and Brain Mind Institute, School of Life Sciences, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- Defitech Center for Interventional Neurotherapies (NeuroRestore), EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Guillaume Charvet
- CEA, LETI, CLINATEC, Univ. Grenoble Alpes, MINATEC Campus, Grenoble, France
| | - Jocelyne Bloch
- Center for Neuroprosthetics and Brain Mind Institute, School of Life Sciences, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- Defitech Center for Interventional Neurotherapies (NeuroRestore), EPFL/CHUV/UNIL, Lausanne, Switzerland
| | - Grégoire Courtine
- Center for Neuroprosthetics and Brain Mind Institute, School of Life Sciences, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
- Department of Clinical Neuroscience, Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland
- Defitech Center for Interventional Neurotherapies (NeuroRestore), EPFL/CHUV/UNIL, Lausanne, Switzerland
| |
Collapse
|