1
|
Jin W, Zhu X, Qian L, Wu C, Yang F, Zhan D, Kang Z, Luo K, Meng D, Xu G. Electroencephalogram-based adaptive closed-loop brain-computer interface in neurorehabilitation: a review. Front Comput Neurosci 2024; 18:1431815. [PMID: 39371523 PMCID: PMC11449715 DOI: 10.3389/fncom.2024.1431815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2024] [Accepted: 09/10/2024] [Indexed: 10/08/2024] Open
Abstract
Brain-computer interfaces (BCIs) represent a groundbreaking approach to enabling direct communication for individuals with severe motor impairments, circumventing traditional neural and muscular pathways. Among the diverse array of BCI technologies, electroencephalogram (EEG)-based systems are particularly favored due to their non-invasive nature, user-friendly operation, and cost-effectiveness. Recent advancements have facilitated the development of adaptive bidirectional closed-loop BCIs, which dynamically adjust to users' brain activity, thereby enhancing responsiveness and efficacy in neurorehabilitation. These systems support real-time modulation and continuous feedback, fostering personalized therapeutic interventions that align with users' neural and behavioral responses. By incorporating machine learning algorithms, these BCIs optimize user interaction and promote recovery outcomes through mechanisms of activity-dependent neuroplasticity. This paper reviews the current landscape of EEG-based adaptive bidirectional closed-loop BCIs, examining their applications in the recovery of motor and sensory functions, as well as the challenges encountered in practical implementation. The findings underscore the potential of these technologies to significantly enhance patients' quality of life and social interaction, while also identifying critical areas for future research aimed at improving system adaptability and performance. As advancements in artificial intelligence continue, the evolution of sophisticated BCI systems holds promise for transforming neurorehabilitation and expanding applications across various domains.
Collapse
Affiliation(s)
- Wenjie Jin
- Department of Rehabilitation Medicine, Nanjing Medical University, Nanjing, China
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - XinXin Zhu
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - Lifeng Qian
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - Cunshu Wu
- Department of Rehabilitation Medicine, Nanjing Medical University, Nanjing, China
| | - Fan Yang
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - Daowei Zhan
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - Zhaoyin Kang
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - Kaitao Luo
- Rehabilitation Medicine Center, Zhejiang Chinese Medical University Affiliated Jiaxing TCM Hospital, Jiaxing, China
| | - Dianhuai Meng
- Department of Rehabilitation Medicine, Nanjing Medical University, Nanjing, China
- Rehabilitation Medicine Center, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Guangxu Xu
- Department of Rehabilitation Medicine, Nanjing Medical University, Nanjing, China
- Rehabilitation Medicine Center, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China
| |
Collapse
|
2
|
Vukelić M, Bui M, Vorreuther A, Lingelbach K. Combining brain-computer interfaces with deep reinforcement learning for robot training: a feasibility study in a simulation environment. FRONTIERS IN NEUROERGONOMICS 2023; 4:1274730. [PMID: 38234482 PMCID: PMC10790930 DOI: 10.3389/fnrgo.2023.1274730] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 10/31/2023] [Indexed: 01/19/2024]
Abstract
Deep reinforcement learning (RL) is used as a strategy to teach robot agents how to autonomously learn complex tasks. While sparsity is a natural way to define a reward in realistic robot scenarios, it provides poor learning signals for the agent, thus making the design of good reward functions challenging. To overcome this challenge learning from human feedback through an implicit brain-computer interface (BCI) is used. We combined a BCI with deep RL for robot training in a 3-D physical realistic simulation environment. In a first study, we compared the feasibility of different electroencephalography (EEG) systems (wet- vs. dry-based electrodes) and its application for automatic classification of perceived errors during a robot task with different machine learning models. In a second study, we compared the performance of the BCI-based deep RL training to feedback explicitly given by participants. Our findings from the first study indicate the use of a high-quality dry-based EEG-system can provide a robust and fast method for automatically assessing robot behavior using a sophisticated convolutional neural network machine learning model. The results of our second study prove that the implicit BCI-based deep RL version in combination with the dry EEG-system can significantly accelerate the learning process in a realistic 3-D robot simulation environment. Performance of the BCI-based trained deep RL model was even comparable to that achieved by the approach with explicit human feedback. Our findings emphasize the usage of BCI-based deep RL methods as a valid alternative in those human-robot applications where no access to cognitive demanding explicit human feedback is available.
Collapse
Affiliation(s)
- Mathias Vukelić
- Applied Neurocognitive Systems, Fraunhofer Institute for Industrial Engineering (IAO), Stuttgart, Germany
| | - Michael Bui
- Applied Neurocognitive Systems, Fraunhofer Institute for Industrial Engineering (IAO), Stuttgart, Germany
| | - Anna Vorreuther
- Applied Neurocognitive Systems, Institute of Human Factors and Technology Management (IAT), University of Stuttgart, Stuttgart, Germany
| | - Katharina Lingelbach
- Applied Neurocognitive Systems, Fraunhofer Institute for Industrial Engineering (IAO), Stuttgart, Germany
| |
Collapse
|
3
|
Wirth C, Toth J, Arvaneh M. Bayesian learning from multi-way EEG feedback for robot navigation and target identification. Sci Rep 2023; 13:16925. [PMID: 37805540 PMCID: PMC10560278 DOI: 10.1038/s41598-023-44077-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 10/03/2023] [Indexed: 10/09/2023] Open
Abstract
Many brain-computer interfaces require a high mental workload. Recent research has shown that this could be greatly alleviated through machine learning, inferring user intentions via reactive brain responses. These signals are generated spontaneously while users merely observe assistive robots performing tasks. Using reactive brain signals, existing studies have addressed robot navigation tasks with a very limited number of potential target locations. Moreover, they use only binary, error-vs-correct classification of robot actions, leaving more detailed information unutilised. In this study a virtual robot had to navigate towards, and identify, target locations in both small and large grids, wherein any location could be the target. For the first time, we apply a system utilising detailed EEG information: 4-way classification of movements is performed, including specific information regarding when the target is reached. Additionally, we classify whether targets are correctly identified. Our proposed Bayesian strategy infers the most likely target location from the brain's responses. The experimental results show that our novel use of detailed information facilitates a more efficient and robust system than the state-of-the-art. Furthermore, unlike state-of-the-art approaches, we show scalability of our proposed approach: By tuning parameters appropriately, our strategy correctly identifies 98% of targets, even in large search spaces.
Collapse
Affiliation(s)
- Christopher Wirth
- Automatic Control and Systems Engineering Department, University of Sheffield, Sheffield, S1 4DT, UK.
- School of Medical Sciences, University of Manchester, Manchester, M13 9NT, UK.
| | - Jake Toth
- Automatic Control and Systems Engineering Department, University of Sheffield, Sheffield, S1 4DT, UK
| | - Mahnaz Arvaneh
- Automatic Control and Systems Engineering Department, University of Sheffield, Sheffield, S1 4DT, UK
| |
Collapse
|
4
|
Yasemin M, Cruz A, Nunes UJ, Pires G. Single trial detection of error-related potentials in brain-machine interfaces: a survey and comparison of methods. J Neural Eng 2023; 20. [PMID: 36595316 DOI: 10.1088/1741-2552/acabe9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 12/15/2022] [Indexed: 12/23/2022]
Abstract
Objective.Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain-computer interfaces (BCIs), increase the naturalness of human-machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms).Approach.With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification.Main results.From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination).Significance.This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.
Collapse
Affiliation(s)
- Mine Yasemin
- Institute of Systems and Robotics (ISR-UC), University of Coimbra, Coimbra, Portugal.,Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, Portugal
| | - Aniana Cruz
- Institute of Systems and Robotics (ISR-UC), University of Coimbra, Coimbra, Portugal
| | - Urbano J Nunes
- Institute of Systems and Robotics (ISR-UC), University of Coimbra, Coimbra, Portugal.,Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, Portugal
| | - Gabriel Pires
- Institute of Systems and Robotics (ISR-UC), University of Coimbra, Coimbra, Portugal.,Engineering Department, Polytechnic Institute of Tomar, Tomar, Portugal
| |
Collapse
|
5
|
Värbu K, Muhammad N, Muhammad Y. Past, Present, and Future of EEG-Based BCI Applications. SENSORS (BASEL, SWITZERLAND) 2022; 22:3331. [PMID: 35591021 PMCID: PMC9101004 DOI: 10.3390/s22093331] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Revised: 04/05/2022] [Accepted: 04/25/2022] [Indexed: 06/15/2023]
Abstract
An electroencephalography (EEG)-based brain-computer interface (BCI) is a system that provides a pathway between the brain and external devices by interpreting EEG. EEG-based BCI applications have initially been developed for medical purposes, with the aim of facilitating the return of patients to normal life. In addition to the initial aim, EEG-based BCI applications have also gained increasing significance in the non-medical domain, improving the life of healthy people, for instance, by making it more efficient, collaborative and helping develop themselves. The objective of this review is to give a systematic overview of the literature on EEG-based BCI applications from the period of 2009 until 2019. The systematic literature review has been prepared based on three databases PubMed, Web of Science and Scopus. This review was conducted following the PRISMA model. In this review, 202 publications were selected based on specific eligibility criteria. The distribution of the research between the medical and non-medical domain has been analyzed and further categorized into fields of research within the reviewed domains. In this review, the equipment used for gathering EEG data and signal processing methods have also been reviewed. Additionally, current challenges in the field and possibilities for the future have been analyzed.
Collapse
Affiliation(s)
- Kaido Värbu
- Institute of Computer Science, University of Tartu, 51009 Tartu, Estonia;
| | - Naveed Muhammad
- Institute of Computer Science, University of Tartu, 51009 Tartu, Estonia;
| | - Yar Muhammad
- Department of Computing & Games, School of Computing, Engineering & Digital Technologies, Teesside University, Middlesbrough TS1 3BX, UK;
| |
Collapse
|
6
|
Portillo-Lara R, Tahirbegi B, Chapman CAR, Goding JA, Green RA. Mind the gap: State-of-the-art technologies and applications for EEG-based brain-computer interfaces. APL Bioeng 2021; 5:031507. [PMID: 34327294 PMCID: PMC8294859 DOI: 10.1063/5.0047237] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2021] [Accepted: 05/19/2021] [Indexed: 11/14/2022] Open
Abstract
Brain-computer interfaces (BCIs) provide bidirectional communication between the brain and output devices that translate user intent into function. Among the different brain imaging techniques used to operate BCIs, electroencephalography (EEG) constitutes the preferred method of choice, owing to its relative low cost, ease of use, high temporal resolution, and noninvasiveness. In recent years, significant progress in wearable technologies and computational intelligence has greatly enhanced the performance and capabilities of EEG-based BCIs (eBCIs) and propelled their migration out of the laboratory and into real-world environments. This rapid translation constitutes a paradigm shift in human-machine interaction that will deeply transform different industries in the near future, including healthcare and wellbeing, entertainment, security, education, and marketing. In this contribution, the state-of-the-art in wearable biosensing is reviewed, focusing on the development of novel electrode interfaces for long term and noninvasive EEG monitoring. Commercially available EEG platforms are surveyed, and a comparative analysis is presented based on the benefits and limitations they provide for eBCI development. Emerging applications in neuroscientific research and future trends related to the widespread implementation of eBCIs for medical and nonmedical uses are discussed. Finally, a commentary on the ethical, social, and legal concerns associated with this increasingly ubiquitous technology is provided, as well as general recommendations to address key issues related to mainstream consumer adoption.
Collapse
Affiliation(s)
- Roberto Portillo-Lara
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Bogachan Tahirbegi
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Christopher A. R. Chapman
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Josef A. Goding
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Rylie A. Green
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| |
Collapse
|
7
|
Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making. Sci Rep 2021; 11:17008. [PMID: 34417494 PMCID: PMC8379268 DOI: 10.1038/s41598-021-96434-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2020] [Accepted: 07/20/2021] [Indexed: 11/15/2022] Open
Abstract
In this paper we present, and test in two realistic environments, collaborative Brain-Computer Interfaces (cBCIs) that can significantly increase both the speed and the accuracy of perceptual group decision-making. The key distinguishing features of this work are: (1) our cBCIs combine behavioural, physiological and neural data in such a way as to be able to provide a group decision at any time after the quickest team member casts their vote, but the quality of a cBCI-assisted decision improves monotonically the longer the group decision can wait; (2) we apply our cBCIs to two realistic scenarios of military relevance (patrolling a dark corridor and manning an outpost at night where users need to identify any unidentified characters that appear) in which decisions are based on information conveyed through video feeds; and (3) our cBCIs exploit Event-Related Potentials (ERPs) elicited in brain activity by the appearance of potential threats but, uniquely, the appearance time is estimated automatically by the system (rather than being unrealistically provided to it). As a result of these elements, in the two test environments, groups assisted by our cBCIs make both more accurate and faster decisions than when individual decisions are integrated in more traditional manners.
Collapse
|
8
|
Fiederer LDJ, Völker M, Schirrmeister RT, Burgard W, Boedecker J, Ball T. Hybrid Brain-Computer-Interfacing for Human-Compliant Robots: Inferring Continuous Subjective Ratings With Deep Regression. Front Neurorobot 2019; 13:76. [PMID: 31649523 PMCID: PMC6795684 DOI: 10.3389/fnbot.2019.00076] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2018] [Accepted: 08/27/2019] [Indexed: 12/02/2022] Open
Abstract
Appropriate robot behavior during human-robot interaction is a key part in the development of human-compliant assistive robotic systems. This study poses the question of how to continuously evaluate the quality of robotic behavior in a hybrid brain-computer interfacing (BCI) task, combining brain and non-brain signals, and how to use the collected information to adapt the robot's behavior accordingly. To this aim, we developed a rating system compatible with EEG recordings, requiring the users to execute only small movements with their thumb on a wireless controller to rate the robot's behavior on a continuous scale. The ratings were recorded together with dry EEG, respiration, ECG, and robotic joint angles in ROS. Pilot experiments were conducted with three users that had different levels of previous experience with robots. The results demonstrate the feasibility to obtain continuous rating data that give insight into the subjective user perception during direct human-robot interaction. The rating data suggests differences in subjective perception for users with no, moderate, or substantial previous robot experience. Furthermore, a variety of regression techniques, including deep CNNs, allowed us to predict the subjective ratings. Performance was better when using the position of the robotic hand than when using EEG, ECG, or respiration. A consistent advantage of features expected to be related to a motor bias could not be found. Across-user predictions showed that the models most likely learned a combination of general and individual features across-users. A transfer of pre-trained regressor to a new user was especially accurate in users with more experience. For future research, studies with more participants will be needed to evaluate the methodology for its use in practice. Data and code to reproduce this study are available at https://github.com/TNTLFreiburg/NiceBot.
Collapse
Affiliation(s)
- Lukas D J Fiederer
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.,BrainLinks BrainTools, Cluster of Excellence, University of Freiburg, Freiburg, Germany
| | - Martin Völker
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.,BrainLinks BrainTools, Cluster of Excellence, University of Freiburg, Freiburg, Germany.,Graduate School of Robotics, University of Freiburg, Freiburg, Germany
| | - Robin T Schirrmeister
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.,BrainLinks BrainTools, Cluster of Excellence, University of Freiburg, Freiburg, Germany
| | - Wolfram Burgard
- BrainLinks BrainTools, Cluster of Excellence, University of Freiburg, Freiburg, Germany.,Department of Computer Science, University of Freiburg, Freiburg, Germany
| | - Joschka Boedecker
- BrainLinks BrainTools, Cluster of Excellence, University of Freiburg, Freiburg, Germany.,Department of Computer Science, University of Freiburg, Freiburg, Germany
| | - Tonio Ball
- Neuromedical AI Lab, Department of Neurosurgery, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany.,BrainLinks BrainTools, Cluster of Excellence, University of Freiburg, Freiburg, Germany
| |
Collapse
|
9
|
|
10
|
Cinel C, Valeriani D, Poli R. Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects. Front Hum Neurosci 2019; 13:13. [PMID: 30766483 PMCID: PMC6365771 DOI: 10.3389/fnhum.2019.00013] [Citation(s) in RCA: 51] [Impact Index Per Article: 10.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2018] [Accepted: 01/10/2019] [Indexed: 01/10/2023] Open
Abstract
Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.
Collapse
Affiliation(s)
- Caterina Cinel
- Brain Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| | - Davide Valeriani
- Brain Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
- Department of Otolaryngology, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, United States
| | - Riccardo Poli
- Brain Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| |
Collapse
|
11
|
Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X. A comprehensive review of EEG-based brain–computer interface paradigms. J Neural Eng 2019; 16:011001. [DOI: 10.1088/1741-2552/aaf12e] [Citation(s) in RCA: 270] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
12
|
Intrinsic interactive reinforcement learning - Using error-related potentials for real world human-robot interaction. Sci Rep 2017; 7:17562. [PMID: 29242555 PMCID: PMC5730605 DOI: 10.1038/s41598-017-17682-7] [Citation(s) in RCA: 55] [Impact Index Per Article: 7.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2017] [Accepted: 11/24/2017] [Indexed: 11/24/2022] Open
Abstract
Reinforcement learning (RL) enables robots to learn its optimal behavioral strategy in dynamic environments based on feedback. Explicit human feedback during robot RL is advantageous, since an explicit reward function can be easily adapted. However, it is very demanding and tiresome for a human to continuously and explicitly generate feedback. Therefore, the development of implicit approaches is of high relevance. In this paper, we used an error-related potential (ErrP), an event-related activity in the human electroencephalogram (EEG), as an intrinsically generated implicit feedback (rewards) for RL. Initially we validated our approach with seven subjects in a simulated robot learning scenario. ErrPs were detected online in single trial with a balanced accuracy (bACC) of 91%, which was sufficient to learn to recognize gestures and the correct mapping between human gestures and robot actions in parallel. Finally, we validated our approach in a real robot scenario, in which seven subjects freely chose gestures and the real robot correctly learned the mapping between gestures and actions (ErrP detection (90% bACC)). In this paper, we demonstrated that intrinsically generated EEG-based human feedback in RL can successfully be used to implicitly improve gesture-based robot control during human-robot interaction. We call our approach intrinsic interactive RL.
Collapse
|
13
|
Huggins JE, Guger C, Ziat M, Zander TO, Taylor D, Tangermann M, Soria-Frisch A, Simeral J, Scherer R, Rupp R, Ruffini G, Robinson DKR, Ramsey NF, Nijholt A, Müller-Putz G, McFarland DJ, Mattia D, Lance BJ, Kindermans PJ, Iturrate I, Herff C, Gupta D, Do AH, Collinger JL, Chavarriaga R, Chase SM, Bleichner MG, Batista A, Anderson CW, Aarnoutse EJ. Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future. BRAIN-COMPUTER INTERFACES 2017; 4:3-36. [PMID: 29152523 PMCID: PMC5693371 DOI: 10.1080/2326263x.2016.1275488] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.
Collapse
Affiliation(s)
- Jane E. Huggins
- Department of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan, USA
| | - Christoph Guger
- G.Tec Medical Engineering GmbH, Guger Technologies OG, Schiedlberg, Austria
| | - Mounia Ziat
- Psychology Department, Northern Michigan University, Marquette, MI, USA
| | - Thorsten O. Zander
- Team PhyPA, Biological Psychology and Neuroergonomics, Technical University of Berlin, Berlin, Germany
| | | | - Michael Tangermann
- Cluster of Excellence BrainLinks-BrainTools, University of Freiburg, Germany
| | | | - John Simeral
- Ctr. For Neurorestoration and Neurotechnology, Rehab. R&D Service, Dept. of VA Medical Center, School of Engineering, Brown University, Providence, RI, USA
| | - Reinhold Scherer
- Institute of Neural Engineering, BCI- Lab, Graz University of Technology, Graz, Austria
| | - Rüdiger Rupp
- Section Experimental Neurorehabilitation, Spinal Cord Injury Center, University Hospital in Heidelberg, Heidelberg, Germany
| | - Giulio Ruffini
- Neuroscience Business Unit, Starlab Barcelona SLU, Barcelona, Spain
- Neuroelectrics Inc., Boston, USA
| | - Douglas K. R. Robinson
- Institute: Laboratoire Interdisciplinaire Sciences Innovations Sociétés (LISIS), Université Paris-Est Marne-la-Vallée, MARNE-LA-VALLÉE, France
| | - Nick F. Ramsey
- Dept Neurology & Neurosurgery, Brain Center Rudolf Magnus, University Medical Center Utrecht, University of Utrecht, Utrecht, Netherlands
| | - Anton Nijholt
- Faculty EEMCS, Enschede, University of Twente, The Netherlands & Imagineering Institute, Iskandar, Malaysia
| | - Gernot Müller-Putz
- Institute of Neural Engineering, BCI- Lab, Graz University of Technology, Graz, Austria
| | - Dennis J. McFarland
- New York State Department of Health, National Center for Adaptive Neurotechnologies, Wadsworth Center, Albany, New York USA
| | - Donatella Mattia
- Clinical Neurophysiology, Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, IRCCS, Rome, Italy
| | - Brent J. Lance
- Human Research and Engineering Directorate, U.S. Army Research Laboratory, Aberdeen Proving Ground, Aberdeen, MD USA
| | | | - Iñaki Iturrate
- Defitech Chair in Brain–machine Interface (CNBI), Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, EPFL-STI-CNBI, Campus Biotech H4, Geneva, Switzerland
| | - Christian Herff
- Cognitive Systems Lab, University of Bremen, Bremen, Germany
| | - Disha Gupta
- Brain Mind Research Inst, Weill Cornell Medical College, Early Brain Injury and Recovery Lab, Burke Medical Research Inst, White Plains, New York, USA
| | - An H. Do
- Department of Neurology, UC Irvine Brain Computer Interface Lab, University of California, Irvine, CA, USA
| | - Jennifer L. Collinger
- Department of Physical Medicine and Rehabilitation, Department of Veterans Affairs, VA Pittsburgh Healthcare System, University of Pittsburgh, Pittsburgh, PA, USA
| | - Ricardo Chavarriaga
- Defitech Chair in Brain–machine Interface (CNBI), Center for Neuroprosthetics, École Polytechnique Fédérale de Lausanne, EPFL-STI-CNBI, Campus Biotech H4, Geneva, Switzerland
| | - Steven M. Chase
- Center for the Neural Basis of Cognition and Department Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Martin G. Bleichner
- Neuropsychology Lab, Department of Psychology, European Medical School, Cluster of Excellence Hearing4all, University of Oldenburg, Oldenburg, Germany
| | - Aaron Batista
- Department of Bioengineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA USA
| | - Charles W. Anderson
- Department of Computer Science, Colorado State University, Fort Collins, CO USA
| | - Erik J. Aarnoutse
- Brain Center Rudolf Magnus, Dept Neurology and Neurosurgery, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|
14
|
Ordikhani-Seyedlar M, Lebedev MA, Sorensen HBD, Puthusserypady S. Neurofeedback Therapy for Enhancing Visual Attention: State-of-the-Art and Challenges. Front Neurosci 2016; 10:352. [PMID: 27536212 PMCID: PMC4971093 DOI: 10.3389/fnins.2016.00352] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2016] [Accepted: 07/12/2016] [Indexed: 11/17/2022] Open
Abstract
We have witnessed a rapid development of brain-computer interfaces (BCIs) linking the brain to external devices. BCIs can be utilized to treat neurological conditions and even to augment brain functions. BCIs offer a promising treatment for mental disorders, including disorders of attention. Here we review the current state of the art and challenges of attention-based BCIs, with a focus on visual attention. Attention-based BCIs utilize electroencephalograms (EEGs) or other recording techniques to generate neurofeedback, which patients use to improve their attention, a complex cognitive function. Although progress has been made in the studies of neural mechanisms of attention, extraction of attention-related neural signals needed for BCI operations is a difficult problem. To attain good BCI performance, it is important to select the features of neural activity that represent attentional signals. BCI decoding of attention-related activity may be hindered by the presence of different neural signals. Therefore, BCI accuracy can be improved by signal processing algorithms that dissociate signals of interest from irrelevant activities. Notwithstanding recent progress, optimal processing of attentional neural signals remains a fundamental challenge for the development of efficient therapies for disorders of attention.
Collapse
Affiliation(s)
- Mehdi Ordikhani-Seyedlar
- Division of Biomedical Engineering, Department of Electrical Engineering, Technical University of Denmark Lyngby, Denmark
| | - Mikhail A Lebedev
- Department of Neurobiology, Duke UniversityDurham, NC, USA; Center for Neuroengineering, Duke UniversityDurham, NC, USA
| | - Helge B D Sorensen
- Division of Biomedical Engineering, Department of Electrical Engineering, Technical University of Denmark Lyngby, Denmark
| | - Sadasivan Puthusserypady
- Division of Biomedical Engineering, Department of Electrical Engineering, Technical University of Denmark Lyngby, Denmark
| |
Collapse
|
15
|
Iturrate I, Chavarriaga R, Montesano L, Minguez J, Millán JDR. Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control. Sci Rep 2015; 5:13893. [PMID: 26354145 PMCID: PMC4564803 DOI: 10.1038/srep13893] [Citation(s) in RCA: 71] [Impact Index Per Article: 7.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2014] [Accepted: 08/07/2015] [Indexed: 12/19/2022] Open
Abstract
Brain-machine interfaces (BMI) usually decode movement parameters from cortical activity to control neuroprostheses. This requires subjects to learn to modulate their brain activity to convey all necessary information, thus imposing natural limits on the complexity of tasks that can be performed. Here we demonstrate an alternative and complementary BMI paradigm that overcomes that limitation by decoding cognitive brain signals associated with monitoring processes relevant for achieving goals. In our approach the neuroprosthesis executes actions that the subject evaluates as erroneous or correct, and exploits the brain correlates of this assessment to learn suitable motor behaviours. Results show that, after a short user's training period, this teaching BMI paradigm operated three different neuroprostheses and generalized across several targets. Our results further support that these error-related signals reflect a task-independent monitoring mechanism in the brain, making this teaching paradigm scalable. We anticipate this BMI approach to become a key component of any neuroprosthesis that mimics natural motor control as it enables continuous adaptation in the absence of explicit information about goals. Furthermore, our paradigm can seamlessly incorporate other cognitive signals and conventional neuroprosthetic approaches, invasive or non-invasive, to enlarge the range and complexity of tasks that can be accomplished.
Collapse
Affiliation(s)
- Iñaki Iturrate
- Instituto de Investigación en Ingeniería de Aragón, Dpto. de Informática e Ingeniería de Sistemas, Universidad de Zaragoza, Spain
- Defitech Chair in Brain-Machine Interface, Center for Neuroprosthetics & Institute of Bioengineering, School of Engineering, Ecole Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland
| | - Ricardo Chavarriaga
- Defitech Chair in Brain-Machine Interface, Center for Neuroprosthetics & Institute of Bioengineering, School of Engineering, Ecole Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland
| | - Luis Montesano
- Instituto de Investigación en Ingeniería de Aragón, Dpto. de Informática e Ingeniería de Sistemas, Universidad de Zaragoza, Spain
| | - Javier Minguez
- Instituto de Investigación en Ingeniería de Aragón, Dpto. de Informática e Ingeniería de Sistemas, Universidad de Zaragoza, Spain
| | - José del R. Millán
- Defitech Chair in Brain-Machine Interface, Center for Neuroprosthetics & Institute of Bioengineering, School of Engineering, Ecole Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland
| |
Collapse
|
16
|
Iturrate I, Grizou J, Omedes J, Oudeyer PY, Lopes M, Montesano L. Exploiting Task Constraints for Self-Calibrated Brain-Machine Interface Control Using Error-Related Potentials. PLoS One 2015; 10:e0131491. [PMID: 26131890 PMCID: PMC4488878 DOI: 10.1371/journal.pone.0131491] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2015] [Accepted: 06/01/2015] [Indexed: 11/19/2022] Open
Abstract
This paper presents a new approach for self-calibration BCI for reaching tasks using error-related potentials. The proposed method exploits task constraints to simultaneously calibrate the decoder and control the device, by using a robust likelihood function and an ad-hoc planner to cope with the large uncertainty resulting from the unknown task and decoder. The method has been evaluated in closed-loop online experiments with 8 users using a previously proposed BCI protocol for reaching tasks over a grid. The results show that it is possible to have a usable BCI control from the beginning of the experiment without any prior calibration. Furthermore, comparisons with simulations and previous results obtained using standard calibration hint that both the quality of recorded signals and the performance of the system were comparable to those obtained with a standard calibration approach.
Collapse
Affiliation(s)
- Iñaki Iturrate
- Chair in Brain-Machine Interface (CNBI) and Center for Neuroprosthetics (CNP), École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
- Instituto de Investigación en Ingeniería de Sistemas (I3A), Universidad de Zaragoza, Zaragoza, Spain
- * E-mail:
| | - Jonathan Grizou
- Flowers Team, a joint Inria—Ensta ParisTech lab, Bourdeaux, France
| | - Jason Omedes
- Instituto de Investigación en Ingeniería de Sistemas (I3A), Universidad de Zaragoza, Zaragoza, Spain
| | | | - Manuel Lopes
- Flowers Team, a joint Inria—Ensta ParisTech lab, Bourdeaux, France
| | - Luis Montesano
- Instituto de Investigación en Ingeniería de Sistemas (I3A), Universidad de Zaragoza, Zaragoza, Spain
| |
Collapse
|