1
|
Ivanov N, Chau T. Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training. Front Comput Neurosci 2023; 17:1108889. [PMID: 36860616 PMCID: PMC9968793 DOI: 10.3389/fncom.2023.1108889] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Accepted: 01/25/2023] [Indexed: 02/15/2023] Open
Abstract
Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.
Collapse
Affiliation(s)
- Nicolas Ivanov
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Tom Chau
- PRISM Lab, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
2
|
Tortora S, Beraldo G, Bettella F, Formaggio E, Rubega M, Del Felice A, Masiero S, Carli R, Petrone N, Menegatti E, Tonin L. Neural correlates of user learning during long-term BCI training for the Cybathlon competition. J Neuroeng Rehabil 2022; 19:69. [PMID: 35790978 PMCID: PMC9254548 DOI: 10.1186/s12984-022-01047-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 06/22/2022] [Indexed: 11/15/2022] Open
Abstract
Background Brain-computer interfaces (BCIs) are systems capable of translating human brain patterns, measured through electroencephalography (EEG), into commands for an external device. Despite the great advances in machine learning solutions to enhance the performance of BCI decoders, the translational impact of this technology remains elusive. The reliability of BCIs is often unsatisfactory for end-users, limiting their application outside a laboratory environment. Methods We present the analysis on the data acquired from an end-user during the preparation for two Cybathlon competitions, where our pilot won the gold medal twice in a row. These data are of particular interest given the mutual learning approach adopted during the longitudinal training phase (8 months), the long training break in between the two events (1 year) and the demanding evaluation scenario. A multifaceted perspective on long-term user learning is proposed: we enriched the information gathered through conventional metrics (e.g., accuracy, application performances) by investigating novel neural correlates of learning in different neural domains. Results First, we showed that by focusing the training on user learning, the pilot was capable of significantly improving his performance over time even with infrequent decoder re-calibrations. Second, we revealed that the analysis of the within-class modifications of the pilot’s neural patterns in the Riemannian domain is more effective in tracking the acquisition and the stabilization of BCI skills, especially after the 1-year break. These results further confirmed the key role of mutual learning in the acquisition of BCI skills, and particularly highlighted the importance of user learning as a key to enhance BCI reliability. Conclusion We firmly believe that our work may open new perspectives and fuel discussions in the BCI field to shift the focus of future research: not only to the machine learning of the decoder, but also in investigating novel training procedures to boost the user learning and the stability of the BCI skills in the long-term. To this end, the analyses and the metrics proposed could be used to monitor the user learning during training and provide a marker guiding the decoder re-calibration to maximize the mutual adaptation of the user to the BCI system. Supplementary Information The online version contains supplementary material available at 10.1186/s12984-022-01047-x.
Collapse
|
3
|
Gyoda T, Nojima I, Lin SC, Koganemaru S, Mima T, Tanabe S, Huang YZ. Strengthening the GABAergic system through neurofeedback training suppresses implicit motor learning. Neuroscience 2022; 488:112-121. [PMID: 35149145 DOI: 10.1016/j.neuroscience.2022.02.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Revised: 02/01/2022] [Accepted: 02/02/2022] [Indexed: 11/26/2022]
Abstract
Gamma-aminobutyric acid (GABA) activity within the primary motor cortex (M1) is essential for motor learning in cortical plasticity, and a recent study has suggested that real-time neurofeedback training (NFT) can self-regulate GABA activity. Therefore, this study aimed to investigate the effect of GABA activity strengthening via NFT on subsequent motor learning. Thirty-six healthy participants were randomly assigned to either an NFT group or control group, which received sham feedback. GABA activity was assessed for short intracortical inhibition (SICI) within the right M1 using paired-pulse transcranial magnetic stimulation. During the NFT intervention period, the participants tried to modulate the size of a circle, which was altered according to the degree of SICI in the NFT group. However, the size was altered independently of the degree of SICI in the control group. We measured the reaction time before, after (online learning), and 24 h after (offline learning) the finger-tapping task. Results showed the strengthening of GABA activity induced by the NFT intervention, and the suppression of the online but not the offline learning. These findings suggest that prior GABA activity modulation may affect online motor learning.
Collapse
Affiliation(s)
- Tomoya Gyoda
- Neuroscience Research Center and Department of Neurology, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Ippei Nojima
- Division of Physical Therapy, Shinshu University School of Health Sciences, Matsumoto, Nagano, Japan.
| | - Su-Chuan Lin
- Neuroscience Research Center and Department of Neurology, Chang Gung Memorial Hospital, Taoyuan, Taiwan; Medical School, College of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Satoko Koganemaru
- Human Brain Research Center, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Tatsuya Mima
- Graduate School of Core Ethics and Frontier Sciences, Ritsumeikan University, Kyoto, Japan
| | - Shigeo Tanabe
- Faculty of Rehabilitation, School of Health Sciences, Fujita Health University, Aichi, Japan
| | - Ying-Zu Huang
- Neuroscience Research Center and Department of Neurology, Chang Gung Memorial Hospital, Taoyuan, Taiwan; Medical School, College of Medicine, Chang Gung University, Taoyuan, Taiwan; Institute of Cognitive Neuroscience, National Central University, Taoyuan, Taiwan
| |
Collapse
|
4
|
Palumbo A, Gramigna V, Calabrese B, Ielpo N. Motor-Imagery EEG-Based BCIs in Wheelchair Movement and Control: A Systematic Literature Review. SENSORS (BASEL, SWITZERLAND) 2021; 21:6285. [PMID: 34577493 PMCID: PMC8473300 DOI: 10.3390/s21186285] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/07/2021] [Revised: 09/09/2021] [Accepted: 09/14/2021] [Indexed: 02/07/2023]
Abstract
The pandemic emergency of the coronavirus disease 2019 (COVID-19) shed light on the need for innovative aids, devices, and assistive technologies to enable people with severe disabilities to live their daily lives. EEG-based Brain-Computer Interfaces (BCIs) can lead individuals with significant health challenges to improve their independence, facilitate participation in activities, thus enhancing overall well-being and preventing impairments. This systematic review provides state-of-the-art applications of EEG-based BCIs, particularly those using motor-imagery (MI) data, to wheelchair control and movement. It presents a thorough examination of the different studies conducted since 2010, focusing on the algorithm analysis, features extraction, features selection, and classification techniques used as well as on wheelchair components and performance evaluation. The results provided in this paper could highlight the limitations of current biomedical instrumentations applied to people with severe disabilities and bring focus to innovative research topics.
Collapse
Affiliation(s)
- Arrigo Palumbo
- Department of Medical and Surgical Sciences, “Magna Græcia” University, 88100 Catanzaro, Italy; (A.P.); (B.C.); (N.I.)
| | - Vera Gramigna
- Neuroscience Research Center, Magna Græcia University, 88100 Catanzaro, Italy
| | - Barbara Calabrese
- Department of Medical and Surgical Sciences, “Magna Græcia” University, 88100 Catanzaro, Italy; (A.P.); (B.C.); (N.I.)
| | - Nicola Ielpo
- Department of Medical and Surgical Sciences, “Magna Græcia” University, 88100 Catanzaro, Italy; (A.P.); (B.C.); (N.I.)
| |
Collapse
|
5
|
Antonietti A, Balachandran P, Hossaini A, Hu Y, Valeriani D. The BCI Glossary: a first proposal for a community review. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.1969789] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Alberto Antonietti
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy
| | | | - Ali Hossaini
- Department of Engineering, King’s College London, London, UK
| | - Yaoping Hu
- Department of Electrical and Software Engineering,University of Calgary, Calgary, AB, Canada
| | - Davide Valeriani
- Department of Otolaryngology Head and Neck Surgery, Massachusetts Eye and Ear, Boston, MA, USA
- Department of Otolaryngology Head and Neck Surgery, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
6
|
Panachakel JT, Ramakrishnan AG. Decoding Covert Speech From EEG-A Comprehensive Review. Front Neurosci 2021; 15:642251. [PMID: 33994922 PMCID: PMC8116487 DOI: 10.3389/fnins.2021.642251] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Accepted: 03/18/2021] [Indexed: 11/13/2022] Open
Abstract
Over the past decade, many researchers have come up with different implementations of systems for decoding covert or imagined speech from EEG (electroencephalogram). They differ from each other in several aspects, from data acquisition to machine learning algorithms, due to which, a comparison between different implementations is often difficult. This review article puts together all the relevant works published in the last decade on decoding imagined speech from EEG into a single framework. Every important aspect of designing such a system, such as selection of words to be imagined, number of electrodes to be recorded, temporal and spatial filtering, feature extraction and classifier are reviewed. This helps a researcher to compare the relative merits and demerits of the different approaches and choose the one that is most optimal. Speech being the most natural form of communication which human beings acquire even without formal education, imagined speech is an ideal choice of prompt for evoking brain activity patterns for a BCI (brain-computer interface) system, although the research on developing real-time (online) speech imagery based BCI systems is still in its infancy. Covert speech based BCI can help people with disabilities to improve their quality of life. It can also be used for covert communication in environments that do not support vocal communication. This paper also discusses some future directions, which will aid the deployment of speech imagery based BCI for practical applications, rather than only for laboratory experiments.
Collapse
Affiliation(s)
- Jerrin Thomas Panachakel
- Medical Intelligence and Language Engineering Laboratory, Department of Electrical Engineering, Indian Institute of Science, Bangalore, India
| | | |
Collapse
|
7
|
Corsi MC, Chavez M, Schwartz D, George N, Hugueville L, Kahn AE, Dupont S, Bassett DS, De Vico Fallani F. BCI learning induces core-periphery reorganization in M/EEG multiplex brain networks. J Neural Eng 2021; 18. [PMID: 33725682 DOI: 10.1088/1741-2552/abef39] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Accepted: 03/16/2021] [Indexed: 11/11/2022]
Abstract
Brain-computer interfaces (BCIs) constitute a promising tool for communication and control. However, mastering non-invasive closed-loop systems remains a learned skill that is difficult to develop for a non-negligible proportion of users. The involved learning process induces neural changes associated with a brain network reorganization that remains poorly understood. To address this inter-subject variability, we adopted a multilayer approach to integrate brain network properties from electroencephalographic (EEG) and magnetoencephalographic (MEG) data resulting from a four-session BCI training program followed by a group of healthy subjects. Our method gives access to the contribution of each layer to multilayer network that tends to be equal with time. We show that regardless the chosen modality, a progressive increase in the integration of somatosensory areas in the α band was paralleled by a decrease of the integration of visual processing and working memory areas in the β band. Notably, only brain network properties in multilayer network correlated with future BCI scores in the α2 band: positively in somatosensory and decision-making related areas and negatively in associative areas. Our findings cast new light on neural processes underlying BCI training. Integrating multimodal brain network properties provides new information that correlates with behavioral performance and could be considered as a potential marker of BCI learning.
Collapse
Affiliation(s)
| | - Mario Chavez
- UMR-7225, CNRS, 47, boulevard de l'Hôpital, Paris, 75013, FRANCE
| | - Denis Schwartz
- INSERM, 47, boulevard de l'Hôpital, Paris, Île-de-France, 75013, FRANCE
| | - Nathalie George
- UMR-7225, CNRS, 47, boulevard de l'Hôpital, Paris, Île-de-France, 75013, FRANCE
| | - Laurent Hugueville
- Institut du Cerveau et de la Moelle Epiniere, 47, boulevard de l'Hôpital, Paris, Île-de-France, 75013, FRANCE
| | - Ari E Kahn
- Department of Neuroscience, University of Pennsylvania, 210 S. 33rd Street 240 Skirkanich Hall, Philadelphia, Pennsylvania, 19104-6321, UNITED STATES
| | - Sophie Dupont
- Institut du Cerveau et de la Moelle Epiniere, 47, boulevard de l'Hôpital, Paris, Île-de-France, 75013, FRANCE
| | - Danielle S Bassett
- Department of Bioengineering, University of Pennsylvania, 210 S. 33rd Street 240 Skirkanich Hall, USA, Philadelphia, Pennsylvania, 19104-6321, UNITED STATES
| | | |
Collapse
|
8
|
Alchalabi B, Faubert J, Labbé D. A multi-modal modified feedback self-paced BCI to control the gait of an avatar. J Neural Eng 2021; 18. [PMID: 33711832 DOI: 10.1088/1741-2552/abee51] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 03/12/2021] [Indexed: 11/12/2022]
Abstract
Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation. OBJECTIVE to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems. APPROACH This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants. MAIN RESULTS All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode. SIGNIFICANCE This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.
Collapse
Affiliation(s)
- Bilal Alchalabi
- biomedical engineering, University of Montreal, 2900 Boulevard Edouard mon Petit, Montreal, Quebec, H3C 3J7, CANADA
| | - Jocelyn Faubert
- Université de Montréal, 3744 Rue Jean Brillant, Montreal, Quebec, H3T 1P1, CANADA
| | - David Labbé
- École de technologie supérieure, 1100 Rue Notre-Dame ouest, Montreal, Quebec, H3C 1K3, CANADA
| |
Collapse
|
9
|
Kawala-Sterniuk A, Browarska N, Al-Bakri A, Pelc M, Zygarlicki J, Sidikova M, Martinek R, Gorzelanczyk EJ. Summary of over Fifty Years with Brain-Computer Interfaces-A Review. Brain Sci 2021; 11:43. [PMID: 33401571 PMCID: PMC7824107 DOI: 10.3390/brainsci11010043] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2020] [Revised: 12/25/2020] [Accepted: 12/27/2020] [Indexed: 11/16/2022] Open
Abstract
Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.
Collapse
Affiliation(s)
- Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Natalia Browarska
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Amir Al-Bakri
- Department of Biomedical Engineering, College of Engineering, University of Babylon, 51001 Babylon, Iraq;
| | - Mariusz Pelc
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
- Department of Computing and Information Systems, University of Greenwich, London SE10 9LS, UK
| | - Jaroslaw Zygarlicki
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Michaela Sidikova
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.S.); (R.M.)
| | - Radek Martinek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.S.); (R.M.)
| | - Edward Jacek Gorzelanczyk
- Department of Theoretical Basis of BioMedical Sciences and Medical Informatics, Nicolaus Copernicus University, Collegium Medicum, 85-067 Bydgoszcz, Poland;
- Institute of Philosophy, Kazimierz Wielki University, 85-092 Bydgoszcz, Poland
- Babinski Specialist Psychiatric Healthcare Center, Outpatient Addiction Treatment, 91-229 Lodz, Poland
- The Society for the Substitution Treatment of Addiction “Medically Assisted Recovery”, 85-791 Bydgoszcz, Poland
| |
Collapse
|
10
|
Yan T, Kameda S, Suzuki K, Kaiju T, Inoue M, Suzuki T, Hirata M. Minimal Tissue Reaction after Chronic Subdural Electrode Implantation for Fully Implantable Brain-Machine Interfaces. SENSORS 2020; 21:s21010178. [PMID: 33383864 PMCID: PMC7795822 DOI: 10.3390/s21010178] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/07/2020] [Revised: 12/12/2020] [Accepted: 12/25/2020] [Indexed: 11/16/2022]
Abstract
There is a growing interest in the use of electrocorticographic (ECoG) signals in brain–machine interfaces (BMIs). However, there is still a lack of studies involving the long-term evaluation of the tissue response related to electrode implantation. Here, we investigated biocompatibility, including chronic tissue response to subdural electrodes and a fully implantable wireless BMI device. We implanted a half-sized fully implantable device with subdural electrodes in six beagles for 6 months. Histological analysis of the surrounding tissues, including the dural membrane and cortices, was performed to evaluate the effects of chronic implantation. Our results showed no adverse events, including infectious signs, throughout the 6-month implantation period. Thick connective tissue proliferation was found in the surrounding tissues in the epidural space and subcutaneous space. Quantitative measures of subdural reactive tissues showed minimal encapsulation between the electrodes and the underlying cortex. Immunohistochemical evaluation showed no significant difference in the cell densities of neurons, astrocytes, and microglia between the implanted sites and contralateral sites. In conclusion, we established a beagle model to evaluate cortical implantable devices. We confirmed that a fully implantable wireless device and subdural electrodes could be stably maintained with sufficient biocompatibility in vivo.
Collapse
Affiliation(s)
- Tianfang Yan
- Department of Neurological Diagnosis and Restoration, Osaka University Graduate School of Medicine, Suita 565-0871, Japan; (T.Y.); (S.K.)
| | - Seiji Kameda
- Department of Neurological Diagnosis and Restoration, Osaka University Graduate School of Medicine, Suita 565-0871, Japan; (T.Y.); (S.K.)
- Global Center for Medical Engineering and Informatics, Osaka University, Suita 565-0871, Japan
- Department of Neurosurgery, Osaka University Graduate School of Medicine, Suita 565-0871, Japan
| | - Katsuyoshi Suzuki
- Ogino Memorial Laboratory, Nihon Kohden Corporation, Tokorozawa 359-0037, Japan;
| | - Taro Kaiju
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita 565-0871, Japan; (T.K.); (M.I.); (T.S.)
| | - Masato Inoue
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita 565-0871, Japan; (T.K.); (M.I.); (T.S.)
| | - Takafumi Suzuki
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita 565-0871, Japan; (T.K.); (M.I.); (T.S.)
| | - Masayuki Hirata
- Department of Neurological Diagnosis and Restoration, Osaka University Graduate School of Medicine, Suita 565-0871, Japan; (T.Y.); (S.K.)
- Global Center for Medical Engineering and Informatics, Osaka University, Suita 565-0871, Japan
- Department of Neurosurgery, Osaka University Graduate School of Medicine, Suita 565-0871, Japan
- Correspondence:
| |
Collapse
|
11
|
Gonzalez-Astudillo J, Cattai T, Bassignana G, Corsi MC, De Vico Fallani F. Network-based brain computer interfaces: principles and applications. J Neural Eng 2020; 18. [PMID: 33147577 DOI: 10.1088/1741-2552/abc760] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Accepted: 11/04/2020] [Indexed: 12/17/2022]
Abstract
Brain-computer interfaces (BCIs) make possible to interact with the external environment by decoding the mental intention of individuals. BCIs can therefore be used to address basic neuroscience questions but also to unlock a variety of applications from exoskeleton control to neurofeedback (NFB) rehabilitation. In general, BCI usability critically depends on the ability to comprehensively characterize brain functioning and correctly identify the user's mental state. To this end, much of the efforts have focused on improving the classification algorithms taking into account localized brain activities as input features. Despite considerable improvement BCI performance is still unstable and, as a matter of fact, current features represent oversimplified descriptors of brain functioning. In the last decade, growing evidence has shown that the brain works as a networked system composed of multiple specialized and spatially distributed areas that dynamically integrate information. While more complex, looking at how remote brain regions functionally interact represents a grounded alternative to better describe brain functioning. Thanks to recent advances in network science, i.e. a modern field that draws on graph theory, statistical mechanics, data mining and inferential modelling, scientists have now powerful means to characterize complex brain networks derived from neuroimaging data. Notably, summary features can be extracted from these networks to quantitatively measure specific organizational properties across a variety of topological scales. In this topical review, we aim to provide the state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability.
Collapse
|
12
|
Yang G, Pang Z, Jamal Deen M, Dong M, Zhang YT, Lovell N, Rahmani AM. Homecare Robotic Systems for Healthcare 4.0: Visions and Enabling Technologies. IEEE J Biomed Health Inform 2020; 24:2535-2549. [PMID: 32340971 DOI: 10.1109/jbhi.2020.2990529] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Powered by the technologies that have originated from manufacturing, the fourth revolution of healthcare technologies is happening (Healthcare 4.0). As an example of such revolution, new generation homecare robotic systems (HRS) based on the cyber-physical systems (CPS) with higher speed and more intelligent execution are emerging. In this article, the new visions and features of the CPS-based HRS are proposed. The latest progress in related enabling technologies is reviewed, including artificial intelligence, sensing fundamentals, materials and machines, cloud computing and communication, as well as motion capture and mapping. Finally, the future perspectives of the CPS-based HRS and the technical challenges faced in each technical area are discussed.
Collapse
|
13
|
Kim YJ, Nam HS, Lee WH, Seo HG, Leigh JH, Oh BM, Bang MS, Kim S. Vision-aided brain-machine interface training system for robotic arm control and clinical application on two patients with cervical spinal cord injury. Biomed Eng Online 2019; 18:14. [PMID: 30744661 PMCID: PMC6371594 DOI: 10.1186/s12938-019-0633-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2018] [Accepted: 02/05/2019] [Indexed: 12/05/2022] Open
Abstract
Background While spontaneous robotic arm control using motor imagery has been reported, most previous successful cases have used invasive approaches with advantages in spatial resolution. However, still many researchers continue to investigate methods for robotic arm control with noninvasive neural signal. Most of noninvasive control of robotic arm utilizes P300, steady state visually evoked potential, N2pc, and mental tasks differentiation. Even though these approaches demonstrated successful accuracy, they are limited in time efficiency and user intuition, and mostly require visual stimulation. Ultimately, velocity vector construction using electroencephalography activated by motion-related motor imagery can be considered as a substitution. In this study, a vision-aided brain–machine interface training system for robotic arm control is proposed and developed. Methods The proposed system uses a Microsoft Kinect to detect and estimates the 3D positions of the possible target objects. The predicted velocity vector for robot arm input is compensated using the artificial potential to follow an intended one among the possible targets. Two participants with cervical spinal cord injury trained with the system to explore its possible effects. Results In a situation with four possible targets, the proposed system significantly improved the distance error to the intended target compared to the unintended ones (p < 0.0001). Functional magnetic resonance imaging after five sessions of observation-based training with the developed system showed brain activation patterns with tendency of focusing to ipsilateral primary motor and sensory cortex, posterior parietal cortex, and contralateral cerebellum. However, shared control with blending parameter α less than 1 was not successful and success rate for touching an instructed target was less than the chance level (= 50%). Conclusions The pilot clinical study utilizing the training system suggested potential beneficial effects in characterizing the brain activation patterns.
Collapse
Affiliation(s)
- Yoon Jae Kim
- Interdisciplinary Program for Bioengineering, Graduate School, Seoul National University, Seoul, 08826, South Korea
| | - Hyung Seok Nam
- Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, 03080, South Korea
| | - Woo Hyung Lee
- Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, 03080, South Korea.,Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, 03080, South Korea
| | - Han Gil Seo
- Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, 03080, South Korea.,Department of Rehabilitation Medicine, Seoul National University Hospital, Seoul, 03080, South Korea
| | - Ja-Ho Leigh
- Department of Rehabilitation Medicine, Incheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Incheon, 21431, South Korea
| | - Byung-Mo Oh
- Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, 03080, South Korea.,Department of Rehabilitation Medicine, Seoul National University Hospital, Seoul, 03080, South Korea
| | - Moon Suk Bang
- Department of Rehabilitation Medicine, Seoul National University College of Medicine, Seoul, 03080, South Korea. .,Department of Rehabilitation Medicine, Seoul National University Hospital, Seoul, 03080, South Korea.
| | - Sungwan Kim
- Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, 03080, South Korea. .,Institute of Medical and Biological Engineering, Medical Research Center, Seoul National University, Seoul, 03080, South Korea.
| |
Collapse
|
14
|
Peng X, Hickman JL, Bowles SG, Donegan DC, Welle CG. Innovations in electrical stimulation harness neural plasticity to restore motor function. BIOELECTRONICS IN MEDICINE 2018; 1:251-263. [PMID: 33859830 PMCID: PMC8046169 DOI: 10.2217/bem-2019-0002] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2019] [Accepted: 02/21/2019] [Indexed: 12/28/2022]
Abstract
Novel technology and innovative stimulation paradigms allow for unprecedented spatiotemporal precision and closed-loop implementation of neurostimulation systems. In turn, precise, closed-loop neurostimulation appears to preferentially drive neural plasticity in motor networks, promoting neural repair. Recent clinical studies demonstrate that electrical stimulation can drive neural plasticity in damaged motor circuits, leading to meaningful improvement in users. Future advances in these areas hold promise for the treatment of a wide range of motor systems disorders.
Collapse
Affiliation(s)
- Xiaoyu Peng
- Dept. of Neurosurgery, University of Colorado, Anschutz Medical Campus, 12700 East 19th Avenue, Aurora, CO 80045
| | - Jordan L. Hickman
- Dept. of Neurosurgery, University of Colorado, Anschutz Medical Campus, 12700 East 19th Avenue, Aurora, CO 80045
| | - Spencer G. Bowles
- Dept. of Neurosurgery, University of Colorado, Anschutz Medical Campus, 12700 East 19th Avenue, Aurora, CO 80045
| | - Dane C. Donegan
- Dept. of Neurosurgery, University of Colorado, Anschutz Medical Campus, 12700 East 19th Avenue, Aurora, CO 80045
- ETH Zurich, Department Health Science and Technology, Institute for Neuroscience. Schorenstrasse 16, 8603 Schwerzenbach, Switzerland
| | - Cristin G. Welle
- Dept. of Neurosurgery, University of Colorado, Anschutz Medical Campus, 12700 East 19th Avenue, Aurora, CO 80045
| |
Collapse
|
15
|
Chu Y, Zhao X, Zou Y, Xu W, Han J, Zhao Y. A Decoding Scheme for Incomplete Motor Imagery EEG With Deep Belief Network. Front Neurosci 2018; 12:680. [PMID: 30323737 PMCID: PMC6172343 DOI: 10.3389/fnins.2018.00680] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2018] [Accepted: 09/10/2018] [Indexed: 01/03/2023] Open
Abstract
High accuracy decoding of electroencephalogram (EEG) signal is still a major challenge that can hardly be solved in the design of an effective motor imagery-based brain-computer interface (BCI), especially when the signal contains various extreme artifacts and outliers arose from data loss. The conventional process to avoid such cases is to directly reject the entire severely contaminated EEG segments, which leads to a drawback that the BCI has no decoding results during that certain period. In this study, a novel decoding scheme based on the combination of Lomb-Scargle periodogram (LSP) and deep belief network (DBN) was proposed to recognize the incomplete motor imagery EEG. Particularly, instead of discarding the entire segment, two forms of data removal were adopted to eliminate the EEG portions with extreme artifacts and data loss. The LSP was utilized to steadily extract the power spectral density (PSD) features from the incomplete EEG constructed by the remaining portions. A DBN structure based on the restricted Boltzmann machine (RBM) was exploited and optimized to perform the classification task. Various comparative experiments were conducted and evaluated on simulated signal and real incomplete motor imagery EEG, including the comparison of three PSD extraction methods (fast Fourier transform, Welch and LSP) and two classifiers (DBN and support vector machine, SVM). The results demonstrate that the LSP can estimate relative robust PSD features and the proposed scheme can significantly improve the decoding performance for the incomplete motor imagery EEG. This scheme can provide an alternative decoding solution for the motor imagery EEG contaminated by extreme artifacts and data loss. It can be beneficial to promote the stability, smoothness and maintain consecutive outputs without interruption for a BCI system that is suitable for the online and long-term application.
Collapse
Affiliation(s)
- Yaqi Chu
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Xingang Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Yijun Zou
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Weiliang Xu
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Department of Mechanical Engineering, University of Auckland, Auckland, New Zealand
| | - Jianda Han
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Yiwen Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
- Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| |
Collapse
|
16
|
Matsushita K, Hirata M, Suzuki T, Ando H, Yoshida T, Ota Y, Sato F, Morris S, Sugata H, Goto T, Yanagisawa T, Yoshimine T. A Fully Implantable Wireless ECoG 128-Channel Recording Device for Human Brain-Machine Interfaces: W-HERBS. Front Neurosci 2018; 12:511. [PMID: 30131666 PMCID: PMC6090147 DOI: 10.3389/fnins.2018.00511] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2017] [Accepted: 07/05/2018] [Indexed: 01/11/2023] Open
Abstract
Brain-machine interfaces (BMIs) are promising devices that can be used as neuroprostheses by severely disabled individuals. Brain surface electroencephalograms (electrocorticograms, ECoGs) can provide input signals that can then be decoded to enable communication with others and to control intelligent prostheses and home electronics. However, conventional systems use wired ECoG recordings. Therefore, the development of wireless systems for clinical ECoG BMIs is a major goal in the field. We developed a fully implantable ECoG signal recording device for human ECoG BMI, i.e., a wireless human ECoG-based real-time BMI system (W-HERBS). In this system, three-dimensional (3D) high-density subdural multiple electrodes are fitted to the brain surface and ECoG measurement units record 128-channel (ch) ECoG signals at a sampling rate of 1 kHz. The units transfer data to the data and power management unit implanted subcutaneously in the abdomen through a subcutaneous stretchable spiral cable. The data and power management unit then communicates with a workstation outside the body and wirelessly receives 400 mW of power from an external wireless transmitter. The workstation records and analyzes the received data in the frequency domain and controls external devices based on analyses. We investigated the performance of the proposed system. We were able to use W-HERBS to detect sine waves with a 4.8-μV amplitude and a 60-200-Hz bandwidth from the ECoG BMIs. W-HERBS is the first fully implantable ECoG-based BMI system with more than 100 ch. It is capable of recording 128-ch subdural ECoG signals with sufficient input-referred noise (3 μVrms) and with an acceptable time delay (250 ms). The system contributes to the clinical application of high-performance BMIs and to experimental brain research.
Collapse
Affiliation(s)
- Kojiro Matsushita
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
- Department of Mechanical Engineering, Gifu University, Gifu, Japan
| | - Masayuki Hirata
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
- Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Osaka, Japan
| | - Takafumi Suzuki
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, Osaka, Japan
| | - Hiroshi Ando
- Center for Information and Neural Networks, National Institute of Information and Communications Technology, Osaka, Japan
| | - Takeshi Yoshida
- Department of Semiconductor Electronics and Integration Science, Hiroshima University, Hiroshima, Japan
| | - Yuki Ota
- Department of Electrical Engineering, Graduate School of Engineering, Tohoku University, Miyagi, Japan
| | - Fumihiro Sato
- Department of Electrical Engineering, Graduate School of Engineering, Tohoku University, Miyagi, Japan
| | - Shayne Morris
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
| | - Hisato Sugata
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
- Faculty of Welfare and Health Science, Oita University, Oita, Japan
| | - Tetsu Goto
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
| | - Takufumi Yanagisawa
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
- Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Osaka, Japan
| | - Toshiki Yoshimine
- Department of Neurosurgery, Osaka University Medical School, Osaka, Japan
- Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Osaka, Japan
| |
Collapse
|
17
|
The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users. PLoS Biol 2018; 16:e2003787. [PMID: 29746465 PMCID: PMC5944920 DOI: 10.1371/journal.pbio.2003787] [Citation(s) in RCA: 74] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2017] [Accepted: 04/09/2018] [Indexed: 12/25/2022] Open
Abstract
This work aims at corroborating the importance and efficacy of mutual learning in motor imagery (MI) brain–computer interface (BCI) by leveraging the insights obtained through our participation in the BCI race of the Cybathlon event. We hypothesized that, contrary to the popular trend of focusing mostly on the machine learning aspects of MI BCI training, a comprehensive mutual learning methodology that reinstates the three learning pillars (at the machine, subject, and application level) as equally significant could lead to a BCI–user symbiotic system able to succeed in real-world scenarios such as the Cybathlon event. Two severely impaired participants with chronic spinal cord injury (SCI), were trained following our mutual learning approach to control their avatar in a virtual BCI race game. The competition outcomes substantiate the effectiveness of this type of training. Most importantly, the present study is one among very few to provide multifaceted evidence on the efficacy of subject learning during BCI training. Learning correlates could be derived at all levels of the interface—application, BCI output, and electroencephalography (EEG) neuroimaging—with two end-users, sufficiently longitudinal evaluation, and, importantly, under real-world and even adverse conditions. Noninvasive brain–computer interface (BCI) based on imagined movements can restore functions lost to disability by enabling spontaneous, direct brain control of external devices without risks associated with surgical implantation of neural interfaces. We hypothesized that, contrary to the popular trend of focusing on the machine learning aspects of BCI training, a comprehensive mutual learning methodology could strongly promote users’ acquisition of BCI skills and lead to a system able to succeed in real-world scenarios such as the Cybathlon event, the first international competition for disabled pilots in control of assistive technology. Two severely impaired participants with chronic spinal cord injury (SCI) were trained following our mutual learning approach to control their avatar in a virtual BCI race game. The evolution of the training process, including competition outcomes (gold medal, tournament record), substantiates the effectiveness of this type of training. Most importantly, the present study provides multifaceted evidence on the efficacy of subject learning during BCI training. Learning correlates could be derived at all levels of the interface—application, BCI output, and electroencephalography—with two end-users, longitudinal evaluation, and, importantly, under real-world and even adverse conditions.
Collapse
|
18
|
Serruya MD, Harris JP, Adewole DO, Struzyna LA, Burrell JC, Nemes A, Petrov D, Kraft RH, Chen HI, Wolf JA, Cullen DK. Engineered Axonal Tracts as "Living Electrodes" for Synaptic-Based Modulation of Neural Circuitry. ADVANCED FUNCTIONAL MATERIALS 2018; 28:1701183. [PMID: 34045935 PMCID: PMC8152180 DOI: 10.1002/adfm.201701183] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
Brain-computer interface and neuromodulation strategies relying on penetrating non-organic electrodes/optrodes are limited by an inflammatory foreign body response that ultimately diminishes performance. A novel "biohybrid" strategy is advanced, whereby living neurons, biomaterials, and microelectrode/optical technology are used together to provide a biologically-based vehicle to probe and modulate nervous-system activity. Microtissue engineering techniques are employed to create axon-based "living electrodes", which are columnar microstructures comprised of neuronal population(s) projecting long axonal tracts within the lumen of a hydrogel designed to chaperone delivery into the brain. Upon microinjection, the axonal segment penetrates to prescribed depth for synaptic integration with local host neurons, with the perikaryal segment remaining externalized below conforming electrical-optical arrays. In this paradigm, only the biological component ultimately remains in the brain, potentially attenuating a chronic foreign-body response. Axon-based living electrodes are constructed using multiple neuronal subtypes, each with differential capacity to stimulate, inhibit, and/or modulate neural circuitry based on specificity uniquely afforded by synaptic integration, yet ultimately computer controlled by optical/electrical components on the brain surface. Current efforts are assessing the efficacy of this biohybrid interface for targeted, synaptic-based neuromodulation, and the specificity, spatial density and long-term fidelity versus conventional microelectronic or optical substrates alone.
Collapse
Affiliation(s)
- Mijail D Serruya
- Department of Neurology, Thomas Jefferson University, Philadelphia, PA 19107, USA
| | - James P Harris
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| | - Dayo O Adewole
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA; Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Laura A Struzyna
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA; Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Justin C Burrell
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| | - Ashley Nemes
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| | - Dmitriy Petrov
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| | - Reuben H Kraft
- Computational Biomechanics Group, Department of Mechanical & Nuclear Engineering, Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16801, USA
| | - H Isaac Chen
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| | - John A Wolf
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| | - D Kacy Cullen
- Center for Brain Injury & Repair, Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Center for Neurotrauma, Neurodegeneration & Restoration, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, PA 19104, USA
| |
Collapse
|
19
|
Casimo K, Weaver KE, Wander J, Ojemann JG. BCI Use and Its Relation to Adaptation in Cortical Networks. IEEE Trans Neural Syst Rehabil Eng 2017; 25:1697-1704. [PMID: 28320670 PMCID: PMC5685806 DOI: 10.1109/tnsre.2017.2681963] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Brain-computer interfaces (BCIs) carry great potential in the treatment of motor impairments. As a new motor output, BCIs interface with the native motor system, but acquisition of BCI proficiency requires a degree of learning to integrate this new function. In this review, we discuss how BCI designs often take advantage of the brain's motor system infrastructure as sources of command signals. We highlight a growing body of literature examining how this approach leads to changes in activity across cortex, including beyond motor regions, as a result of learning the new skill of BCI control. We discuss the previous research identifying patterns of neural activity associated with BCI skill acquisition and use that closely resembles those associated with learning traditional native motor tasks. We then discuss recent work in animals probing changes in connectivity of the BCI control site, which were linked to BCI skill acquisition, and use this as a foundation for our original work in humans. We present our novel work showing changes in resting state connectivity across cortex following the BCI learning process. We find substantial, heterogeneous changes in connectivity across regions and frequencies, including interactions that do not involve the BCI control site. We conclude from our review and original work that BCI skill acquisition may potentially lead to significant changes in evoked and resting state connectivity across multiple cortical regions. We recommend that future studies of BCIs look beyond motor regions to fully describe the cortical networks involved and long-term adaptations resulting from BCI skill acquisition.
Collapse
|
20
|
Hernández-González S, Andreu-Sánchez C, Martín-Pascual MÁ, Gruart A, Delgado-García JM. A Cognition-Related Neural Oscillation Pattern, Generated in the Prelimbic Cortex, Can Control Operant Learning in Rats. J Neurosci 2017; 37:5923-5935. [PMID: 28536269 PMCID: PMC6596507 DOI: 10.1523/jneurosci.3651-16.2017] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2016] [Revised: 03/25/2017] [Accepted: 04/02/2017] [Indexed: 11/21/2022] Open
Abstract
The prelimbic (PrL) cortex constitutes one of the highest levels of cortical hierarchy dedicated to the execution of adaptive behaviors. We have identified a specific local field potential (LFP) pattern generated in the PrL cortex and associated with cognition-related behaviors. We used this pattern to trigger the activation of a visual display on a touch screen as part of an operant conditioning task. Rats learned to increase the presentation rate of the selected θ to β-γ (θ/β-γ) transition pattern across training sessions. The selected LFP pattern appeared to coincide with a significant decrease in the firing of PrL pyramidal neurons and did not seem to propagate to other cortical or subcortical areas. An indication of the PrL cortex's cognitive nature is that the experimental disruption of this θ/β-γ transition pattern prevented the proper performance of the acquired task without affecting the generation of other motor responses. The use of this LFP pattern to trigger an operant task evoked only minor changes in its electrophysiological properties. Thus, the PrL cortex has the capability of generating an oscillatory pattern for dealing with environmental constraints. In addition, the selected θ/β-γ transition pattern could be a useful tool to activate the presentation of external cues or to modify the current circumstances.SIGNIFICANCE STATEMENT Brain-machine interfaces represent a solution for physically impaired people to communicate with external devices. We have identified a specific local field potential pattern generated in the prelimbic cortex and associated with goal-directed behaviors. We used the pattern to trigger the activation of a visual display on a touch screen as part of an operant conditioning task. Rats learned to increase the presentation rate of the selected field potential pattern across training. The selected pattern was not modified when used to activate the touch screen. Electrical stimulation of the recording site prevented the proper performance of the task. Our findings show that the prelimbic cortex can generate oscillatory patterns that rats can use to control their environment for achieving specific goals.
Collapse
Affiliation(s)
| | - Celia Andreu-Sánchez
- Audiovisual Communication and Advertising Department, Universitat Autònoma de Barcelona, 08193 Bellaterra, Spain
| | - Miguel Ángel Martín-Pascual
- Audiovisual Communication and Advertising Department, Universitat Autònoma de Barcelona, 08193 Bellaterra, Spain
| | - Agnès Gruart
- Division of Neurosciences, Pablo de Olavide University, 41013 Seville, Spain, and
| | | |
Collapse
|
21
|
Darvishi S, Gharabaghi A, Boulay CB, Ridding MC, Abbott D, Baumert M. Proprioceptive Feedback Facilitates Motor Imagery-Related Operant Learning of Sensorimotor β-Band Modulation. Front Neurosci 2017; 11:60. [PMID: 28232788 PMCID: PMC5299002 DOI: 10.3389/fnins.2017.00060] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2016] [Accepted: 01/27/2017] [Indexed: 01/26/2023] Open
Abstract
Motor imagery (MI) activates the sensorimotor system independent of actual movements and might be facilitated by neurofeedback. Knowledge on the interaction between feedback modality and the involved frequency bands during MI-related brain self-regulation is still scarce. Previous studies compared the cortical activity during the MI task with concurrent feedback (MI with feedback condition) to cortical activity during the relaxation task where no feedback was provided (relaxation without feedback condition). The observed differences might, therefore, be related to either the task or the feedback. A proper comparison would necessitate studying a relaxation condition with feedback and a MI task condition without feedback as well. Right-handed healthy subjects performed two tasks, i.e., MI and relaxation, in alternating order. Each of the tasks (MI vs. relaxation) was studied with and without feedback. The respective event-driven oscillatory activity, i.e., sensorimotor desynchronization (during MI) or synchronization (during relaxation), was rewarded with contingent feedback. Importantly, feedback onset was delayed to study the task-related cortical activity in the absence of feedback provision during the delay period. The reward modality was alternated every 15 trials between proprioceptive and visual feedback. Proprioceptive input was superior to visual input to increase the range of task-related spectral perturbations in the α- and β-band, and was necessary to consistently achieve MI-related sensorimotor desynchronization (ERD) significantly below baseline. These effects occurred in task periods without feedback as well. The increased accuracy and duration of learned brain self-regulation achieved in the proprioceptive condition was specific to the β-band. MI-related operant learning of brain self-regulation is facilitated by proprioceptive feedback and mediated in the sensorimotor β-band.
Collapse
Affiliation(s)
- Sam Darvishi
- School of Electrical and Electronic Engineering, University of AdelaideAdelaide, SA, Australia; Division of Functional and Restorative Neurosurgery, and Centre for Integrative Neuroscience, Eberhard Karls University TuebingenTubingen, Germany
| | - Alireza Gharabaghi
- Division of Functional and Restorative Neurosurgery, and Centre for Integrative Neuroscience, Eberhard Karls University Tuebingen Tubingen, Germany
| | - Chadwick B Boulay
- The Ottawa Hospital Research Institute, University of Ottawa Ottawa, ON, Canada
| | - Michael C Ridding
- The Robinson Research Institute, University of Adelaide Adelaide, SA, Australia
| | - Derek Abbott
- School of Electrical and Electronic Engineering, University of Adelaide Adelaide, SA, Australia
| | - Mathias Baumert
- School of Electrical and Electronic Engineering, University of Adelaide Adelaide, SA, Australia
| |
Collapse
|
22
|
Pinotsis DA, Geerts JP, Pinto L, FitzGerald THB, Litvak V, Auksztulewicz R, Friston KJ. Linking canonical microcircuits and neuronal activity: Dynamic causal modelling of laminar recordings. Neuroimage 2017; 146:355-366. [PMID: 27871922 PMCID: PMC5312791 DOI: 10.1016/j.neuroimage.2016.11.041] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2016] [Revised: 11/10/2016] [Accepted: 11/16/2016] [Indexed: 12/20/2022] Open
Abstract
Neural models describe brain activity at different scales, ranging from single cells to whole brain networks. Here, we attempt to reconcile models operating at the microscopic (compartmental) and mesoscopic (neural mass) scales to analyse data from microelectrode recordings of intralaminar neural activity. Although these two classes of models operate at different scales, it is relatively straightforward to create neural mass models of ensemble activity that are equipped with priors obtained after fitting data generated by detailed microscopic models. This provides generative (forward) models of measured neuronal responses that retain construct validity in relation to compartmental models. We illustrate our approach using cross spectral responses obtained from V1 during a visual perception paradigm that involved optogenetic manipulation of the basal forebrain. We find that the resulting neural mass model can distinguish between activity in distinct cortical layers - both with and without optogenetic activation - and that cholinergic input appears to enhance (disinhibit) superficial layer activity relative to deep layers. This is particularly interesting from the perspective of predictive coding, where neuromodulators are thought to boost prediction errors that ascend the cortical hierarchy.
Collapse
Affiliation(s)
- D A Pinotsis
- The Picower Institute for Learning & Memory and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, United States; The Wellcome Trust Centre for Neuroimaging, University College London, Queen Square, London WC1N 3BG, UK.
| | - J P Geerts
- The Wellcome Trust Centre for Neuroimaging, University College London, Queen Square, London WC1N 3BG, UK
| | - L Pinto
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, United States
| | - T H B FitzGerald
- The Wellcome Trust Centre for Neuroimaging, University College London, Queen Square, London WC1N 3BG, UK; MPS - UCL Centre for Computational Psychiatry and Ageing Research, Russell Square House, London, WC1B 5EH, UK
| | - V Litvak
- The Wellcome Trust Centre for Neuroimaging, University College London, Queen Square, London WC1N 3BG, UK
| | - R Auksztulewicz
- The Wellcome Trust Centre for Neuroimaging, University College London, Queen Square, London WC1N 3BG, UK; Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, Oxford OX3 7JX, UK
| | - K J Friston
- The Wellcome Trust Centre for Neuroimaging, University College London, Queen Square, London WC1N 3BG, UK
| |
Collapse
|
23
|
Quitadamo LR, Cavrini F, Sbernini L, Riillo F, Bianchi L, Seri S, Saggio G. Support vector machines to detect physiological patterns for EEG and EMG-based human-computer interaction: a review. J Neural Eng 2017; 14:011001. [PMID: 28068295 DOI: 10.1088/1741-2552/14/1/011001] [Citation(s) in RCA: 47] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
Support vector machines (SVMs) are widely used classifiers for detecting physiological patterns in human-computer interaction (HCI). Their success is due to their versatility, robustness and large availability of free dedicated toolboxes. Frequently in the literature, insufficient details about the SVM implementation and/or parameters selection are reported, making it impossible to reproduce study analysis and results. In order to perform an optimized classification and report a proper description of the results, it is necessary to have a comprehensive critical overview of the applications of SVM. The aim of this paper is to provide a review of the usage of SVM in the determination of brain and muscle patterns for HCI, by focusing on electroencephalography (EEG) and electromyography (EMG) techniques. In particular, an overview of the basic principles of SVM theory is outlined, together with a description of several relevant literature implementations. Furthermore, details concerning reviewed papers are listed in tables and statistics of SVM use in the literature are presented. Suitability of SVM for HCI is discussed and critical comparisons with other classifiers are reported.
Collapse
Affiliation(s)
- L R Quitadamo
- Department of Electronic Engineering, University of Rome Tor Vergata, Rome, Italy. School of Life and Health Sciences, Aston Brain Center, Aston University, Birmingham, UK
| | | | | | | | | | | | | |
Collapse
|
24
|
Seifzadeh S, Rezaei M, Faez K, Amiri M. Fast and Efficient Four‑class Motor Imagery Electroencephalography Signal Analysis Using Common Spatial Pattern-Ridge Regression Algorithm for the Purpose of Brain-Computer Interface. JOURNAL OF MEDICAL SIGNALS AND SENSORS 2017; 7:80-85. [PMID: 28553580 PMCID: PMC5437766] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Abstract
Brain-computer interfaces enable users to control devices with electroencephalographic (EEG) activity from the scalp or with single-neuron activity from within the brain. One of the most challenging issues in this regard is the balance between the accuracy of brain signals from patients and the speed of interpreting them into machine language. The main objective of this paper is to analyze different approaches to achieve the balance more quickly and in a better way. To reduce the ocular artifacts, the symmetric prewhitening independent component analysis (ICA) algorithm has been evaluated, which has the lowest runtime and lowest signal-to-interference (SIR) index, without destroying the original signal. After quick elimination of all undesirable signals, two successful feature extractors - the log-band power algorithm and common spatial patterns (CSPs) - are used to extract features. The emphasis is on identifying discriminative properties of the feature sets representing EEG trials recorded during the imagination of the tongue, feet, and left-right-hand movement. Finally, three well-known classifiers are evaluated, where the ridge regression classifier and CSPs as feature extractor have the highest accuracy classification rate about 83.06% with a standard deviation of 1.22%, counterposing the recent studies.
Collapse
Affiliation(s)
- Sahar Seifzadeh
- Department of Young Researchers and Elite Club, Qazvin Branch, Islamic Azad University, Qazvin, Iran
| | - Mohammad Rezaei
- Sleep Disorders Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran,Address for correspondence: Mr. Mohammad Rezaei, Sleep Disorders Research Center, Kermanshah University of Medical Sciences, Farabi Hospital, Kermanshah, Iran. E-mail:
| | - Karim Faez
- Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran
| | - Mahmood Amiri
- Medical Biology Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran
| |
Collapse
|
25
|
Kinney-Lang E, Auyeung B, Escudero J. Expanding the (kaleido)scope: exploring current literature trends for translating electroencephalography (EEG) based brain–computer interfaces for motor rehabilitation in children. J Neural Eng 2016; 13:061002. [DOI: 10.1088/1741-2560/13/6/061002] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
|
26
|
Neuroprosthetics in amputee and brain injury rehabilitation. Exp Neurol 2016; 287:479-485. [PMID: 27519275 DOI: 10.1016/j.expneurol.2016.08.004] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2016] [Accepted: 08/08/2016] [Indexed: 01/07/2023]
Abstract
The goals of rehabilitation medicine programs are to promote health, restore functional impairments and improve quality of life. The field of neuroprosthetics has evolved over the last decade given an improved understanding of neuroscience and the incorporation of advanced biotechnology and neuroengineering in the rehabilitation setting to develop adaptable applications to help facilitate recovery for individuals with amputations and brain injury. These applications may include a simple cognitive prosthetics aid for impaired memory in brain-injured individuals to myoelectric prosthetics arms with artificial proprioceptive feedback for those with upper extremity amputations. The integration of neuroprosthetics into the existing framework of current rehabilitation approaches not only improves quality-of-care and outcomes but help broadens current rehabilitation treatment paradigms. Although, we are in the infancy of the understanding the true benefit of neuroprosthetics and its clinical applications in the rehabilitation setting there is tremendous amount of promise for future research and development of tools to help facilitate recovery and improve quality of life in individuals with disabilities.
Collapse
|
27
|
Cheng MY, Aswendt M, Steinberg GK. Optogenetic Approaches to Target Specific Neural Circuits in Post-stroke Recovery. Neurotherapeutics 2016; 13:325-40. [PMID: 26701667 PMCID: PMC4824024 DOI: 10.1007/s13311-015-0411-5] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023] Open
Abstract
Stroke is a leading cause of death and disability in the USA, yet treatment options are very limited. Functional recovery can occur after stroke and is attributed, in part, to rewiring of neural connections in areas adjacent to or remotely connected to the infarct. A better understanding of neural circuit rewiring is thus an important step toward developing future therapeutic strategies for stroke recovery. Because stroke disrupts functional connections in peri-infarct and remotely connected regions, it is important to investigate brain-wide network dynamics during post-stroke recovery. Optogenetics is a revolutionary neuroscience tool that uses bioengineered light-sensitive proteins to selectively activate or inhibit specific cell types and neural circuits within milliseconds, allowing greater specificity and temporal precision for dissecting neural circuit mechanisms in diseases. In this review, we discuss the current view of post-stroke remapping and recovery, including recent studies that use optogenetics to investigate neural circuit remapping after stroke, as well as optogenetic stimulation to enhance stroke recovery. Multimodal approaches employing optogenetics in conjunction with other readouts (e.g., in vivo neuroimaging techniques, behavior assays, and next-generation sequencing) will advance our understanding of neural circuit reorganization during post-stroke recovery, as well as provide important insights into which brain circuits to target when designing brain stimulation strategies for future clinical studies.
Collapse
Affiliation(s)
- Michelle Y Cheng
- Department of Neurosurgery, R281, Stanford University School of Medicine, 300 Pasteur Drive, Stanford, CA, 94305-5327, USA.
| | - Markus Aswendt
- Department of Neurosurgery, R281, Stanford University School of Medicine, 300 Pasteur Drive, Stanford, CA, 94305-5327, USA
| | - Gary K Steinberg
- Department of Neurosurgery, R281, Stanford University School of Medicine, 300 Pasteur Drive, Stanford, CA, 94305-5327, USA.
| |
Collapse
|