51
|
Velasco-Álvarez F, Sancha-Ros S, García-Garaluz E, Fernández-Rodríguez Á, Medina-Juliá MT, Ron-Angevin R. UMA-BCI Speller: An easily configurable P300 speller tool for end users. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2019; 172:127-138. [PMID: 30902124 DOI: 10.1016/j.cmpb.2019.02.015] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2018] [Revised: 02/08/2019] [Accepted: 02/27/2019] [Indexed: 06/09/2023]
Abstract
BACKGROUND AND OBJECTIVE Some neurodegenerative conditions can severely limit patients' capability to communicate because of the loss of muscular control. Brain-computer interfaces may help in the restoration of communication with these patients, bypassing the muscular activity, so that brain signals can be directly interpreted by a computer. There are many studies regarding brain-controlled spellers; however, these systems do not usually leap out of the lab because of technical and economic requirements. As a consequence, the potential end users do not benefit from these scientific advances in their daily life. The objective of this paper is to present a novel brain-controlled speller designed to be used by patients due to its versatility and ease of use. METHODS The brain-computer interface research group of the University of Málaga (UMA-BCI) has developed a speller application based on the well-known P300 potential which can be easily installed, configured and used. The application supports the common P300 paradigms: the Row-Column Paradigm and the Rapid Serial Visual Presentation Paradigm. The inner core of the application is implemented with a widely used and studied platform, BCI2000, which ensures its reliability and allows other researchers to apply modifications at will in order to test new features. Ten naïve volunteers carried out exercises using the application and completed usability tests for evaluation purposes. RESULTS New subjects using the application managed to set up and use the proposed speller in less than an hour. The positive results of the evaluation through the usability tests support this application's ease of use. CONCLUSIONS A new brain-controlled spelling tool has been presented whose aim is to be used by severely paralyzed patients in their daily lives, as well as by researchers to test new spelling features.
Collapse
Affiliation(s)
| | - Salvador Sancha-Ros
- ENESO Tecnología de Adaptación S.L., Parque Tecnológico de Andalucía, Málaga, Spain
| | | | | | | | - Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Universidad de Málaga, 29071 Málaga, Spain.
| |
Collapse
|
52
|
Megalingam RK, Rangan V, Veliyara P, Krishna RR, Prabhu R, Katoch R, Koppaka GSA, Sankaran R. Design, analysis and performance evaluation of a hand gesture platform for navigation. Technol Health Care 2019; 27:417-430. [PMID: 30909255 DOI: 10.3233/thc-181294] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Prevailing technological solutions that address the problems that are experienced by the infirm and elderly people in terms of locomotion needs, offer limited options when it comes to control mechanism and customization. For more than a decade, joysticks have served the purpose of steering and navigation of autonomous wheelchairs. An alternative gesture-based method for navigation of wheelchairs by the physically impaired could very well replace the conventional joystick controls. A prototype system, 'Mudra' (Sanskrit word for gesture), incorporates a gesture capture module, developed for recognition and identification of hand gestures. Mudra is a no-nonsense user-friendly system that enables control of the navigational platform, merely by four gestures of the human hand. This paper presents a comprehensive report on the implementation of the Mudra system's hardware and software, performance analysis and safety evaluation. Verification tests of the conceptual design show promising results, where 97.8% of the hand gestures were recognized accurately. Furthermore, the response timings of wheelchairs with Mudra controls were noticeably faster than the joystick-based wheelchairs, as affirmed by live testing with wheelchair-users. Pursuant to the positive feedback from the wheelchair-user experience, we conclude that Mudra's gesture controlled wheelchairs would be a preferable alternative to joystick-controlled wheelchairs.
Collapse
Affiliation(s)
- Rajesh Kannan Megalingam
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
| | - Venkat Rangan
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
| | - Pranav Veliyara
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
| | - Rithun Raj Krishna
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
| | - Raghavendra Prabhu
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
| | - Rocky Katoch
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India
| | | | - Ravi Sankaran
- Department of Physical Medicine and Rehabilitation, School of Medicine, Amrita Vishwa Vidyapeetham, Kochi, India
| |
Collapse
|
53
|
Yuan Y, Su W, Li Z, Shi G. Brain–Computer Interface-Based Stochastic Navigation and Control of a Semiautonomous Mobile Robot in Indoor Environments. IEEE Trans Cogn Dev Syst 2019. [DOI: 10.1109/tcds.2018.2885774] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
54
|
Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X. A comprehensive review of EEG-based brain–computer interface paradigms. J Neural Eng 2019; 16:011001. [DOI: 10.1088/1741-2552/aaf12e] [Citation(s) in RCA: 270] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
55
|
Choi JR, Kim SM, Ryu RH, Kim SP, Sohn JW. Implantable Neural Probes for Brain-Machine Interfaces - Current Developments and Future Prospects. Exp Neurobiol 2018; 27:453-471. [PMID: 30636899 PMCID: PMC6318554 DOI: 10.5607/en.2018.27.6.453] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2018] [Revised: 11/15/2018] [Accepted: 11/15/2018] [Indexed: 12/14/2022] Open
Abstract
A Brain-Machine interface (BMI) allows for direct communication between the brain and machines. Neural probes for recording neural signals are among the essential components of a BMI system. In this report, we review research regarding implantable neural probes and their applications to BMIs. We first discuss conventional neural probes such as the tetrode, Utah array, Michigan probe, and electroencephalography (ECoG), following which we cover advancements in next-generation neural probes. These next-generation probes are associated with improvements in electrical properties, mechanical durability, biocompatibility, and offer a high degree of freedom in practical settings. Specifically, we focus on three key topics: (1) novel implantable neural probes that decrease the level of invasiveness without sacrificing performance, (2) multi-modal neural probes that measure both electrical and optical signals, (3) and neural probes developed using advanced materials. Because safety and precision are critical for practical applications of BMI systems, future studies should aim to enhance these properties when developing next-generation neural probes.
Collapse
Affiliation(s)
- Jong-Ryul Choi
- Medical Device Development Center, Daegu-Gyeongbuk Medical Innovation Foundation (DGMIF), Daegu 41061, Korea
| | - Seong-Min Kim
- Department of Medical Science, College of Medicine, Catholic Kwandong University, Gangneung 25601, Korea.,Biomedical Research Institute, Catholic Kwandong University International St. Mary's Hospital, Incheon 21711, Korea
| | - Rae-Hyung Ryu
- Laboratory Animal Center, Daegu-Gyeongbuk Medical Innovation Foundation (DGMIF), Daegu 41061, Korea
| | - Sung-Phil Kim
- Department of Human Factors Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Korea
| | - Jeong-Woo Sohn
- Department of Medical Science, College of Medicine, Catholic Kwandong University, Gangneung 25601, Korea.,Biomedical Research Institute, Catholic Kwandong University International St. Mary's Hospital, Incheon 21711, Korea
| |
Collapse
|
56
|
Al-Qaysi ZT, Zaidan BB, Zaidan AA, Suzani MS. A review of disability EEG based wheelchair control system: Coherent taxonomy, open challenges and recommendations. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2018; 164:221-237. [PMID: 29958722 DOI: 10.1016/j.cmpb.2018.06.012] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2018] [Revised: 05/27/2018] [Accepted: 06/14/2018] [Indexed: 06/08/2023]
Abstract
CONTEXT Intelligent wheelchair technology has recently been utilised to address several mobility problems. Techniques based on brain-computer interface (BCI) are currently used to develop electric wheelchairs. Using human brain control in wheelchairs for people with disability has elicited widespread attention due to its flexibility. OBJECTIVE This study aims to determine the background of recent studies on wheelchair control based on BCI for disability and map the literature survey into a coherent taxonomy. The study intends to identify the most important aspects in this emerging field as an impetus for using BCI for disability in electric-powered wheelchair (EPW) control, which remains a challenge. The study also attempts to provide recommendations for solving other existing limitations and challenges. METHODS We systematically searched all articles about EPW control based on BCI for disability in three popular databases: ScienceDirect, IEEE and Web of Science. These databases contain numerous articles that considerably influenced this field and cover most of the relevant theoretical and technical issues. RESULTS We selected 100 articles on the basis of our inclusion and exclusion criteria. A large set of articles (55) discussed on developing real-time wheelchair control systems based on BCI for disability signals. Another set of articles (25) focused on analysing BCI for disability signals for wheelchair control. The third set of articles (14) considered the simulation of wheelchair control based on BCI for disability signals. Four articles designed a framework for wheelchair control based on BCI for disability signals. Finally, one article reviewed concerns regarding wheelchair control based on BCI for disability signals. DISCUSSION Since 2007, researchers have pursued the possibility of using BCI for disability in EPW control through different approaches. Regardless of type, articles have focused on addressing limitations that impede the full efficiency of BCI for disability and recommended solutions for these limitations. CONCLUSIONS Studies on wheelchair control based on BCI for disability considerably influence society due to the large number of people with disability. Therefore, we aim to provide researchers and developers with a clear understanding of this platform and highlight the challenges and gaps in the current and future studies.
Collapse
Affiliation(s)
- Z T Al-Qaysi
- Department of Computing, University Pendidikan Sultan Idris, Tanjong Malim, Perak, Malaysia; Department of Computer Science, Computer Science and Mathematics College, Tikrit University, Tikrit
| | - B B Zaidan
- Department of Computing, University Pendidikan Sultan Idris, Tanjong Malim, Perak, Malaysia
| | - A A Zaidan
- Department of Computing, University Pendidikan Sultan Idris, Tanjong Malim, Perak, Malaysia.
| | - M S Suzani
- Department of Computing, University Pendidikan Sultan Idris, Tanjong Malim, Perak, Malaysia
| |
Collapse
|
57
|
Tang J, Liu Y, Hu D, Zhou Z. Towards BCI-actuated smart wheelchair system. Biomed Eng Online 2018; 17:111. [PMID: 30126416 PMCID: PMC6102906 DOI: 10.1186/s12938-018-0545-x] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2018] [Accepted: 08/13/2018] [Indexed: 11/16/2022] Open
Abstract
Background Electroencephalogram-based brain–computer interfaces (BCIs) represent novel human machine interactive technology that allows people to communicate and interact with the external world without relying on their peripheral muscles and nervous system. Among BCI systems, brain-actuated wheelchairs are promising systems for the rehabilitation of severely motor disabled individuals who are unable to control a wheelchair by conventional interfaces. Previous related studies realized the easy use of brain-actuated wheelchairs that enable people to navigate the wheelchair through simple commands; however, these systems rely on offline calibration of the environment. Other systems do not rely on any prior knowledge; however, the control of the system is time consuming. In this paper, we have proposed an improved mobile platform structure equipped with an omnidirectional wheelchair, a lightweight robotic arm, a target recognition module and an auto-control module. Based on the you only look once (YOLO) algorithm, our system can, in real time, recognize and locate the targets in the environment, and the users confirm one target through a P300-based BCI. An expert system plans a proper solution for a specific target; for example, the planned solution for a door is opening the door and then passing through it, and the auto-control system then jointly controls the wheelchair and robotic arm to complete the operation. During the task execution, the target is also tracked by using an image tracking technique. Thus, we have formed an easy-to-use system that can provide accurate services to satisfy user requirements, and this system can accommodate different environments. Results To validate and evaluate our system, an experiment simulating the daily application was performed. The tasks included the user driving the system closer to a walking man and having a conversation with him; going to another room through a door; and picking up a bottle of water on the desk and drinking water. Three patients (cerebral infarction; spinal injury; and stroke) and four healthy subjects participated in the test and all completed the tasks. Conclusion This article presents a brain-actuated smart wheelchair system. The system is intelligent in that it provides efficient and considerate services for users. To test the system, three patients and four healthy subjects were recruited to participate in a test. The results demonstrate that the system works smartly and efficiently; with this system, users only need to issue small commands to get considerate services. This system is of significance for accelerating the application of BCIs in the practical environment, especially for patients who will use a BCI for rehabilitation applications.
Collapse
Affiliation(s)
- Jingsheng Tang
- College of Artificial Intelligence, National University of Defense Technology, Deya Road, Changsha, 410000, People's Republic of China
| | - Yadong Liu
- College of Artificial Intelligence, National University of Defense Technology, Deya Road, Changsha, 410000, People's Republic of China
| | - Dewen Hu
- College of Artificial Intelligence, National University of Defense Technology, Deya Road, Changsha, 410000, People's Republic of China
| | - ZongTan Zhou
- College of Artificial Intelligence, National University of Defense Technology, Deya Road, Changsha, 410000, People's Republic of China.
| |
Collapse
|
58
|
Yu Y, Zhou Z, Liu Y, Jiang J, Yin E, Zhang N, Wang Z, Liu Y, Wu X, Hu D. Self-Paced Operation of a Wheelchair Based on a Hybrid Brain-Computer Interface Combining Motor Imagery and P300 Potential. IEEE Trans Neural Syst Rehabil Eng 2018; 25:2516-2526. [PMID: 29220327 DOI: 10.1109/tnsre.2017.2766365] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This paper presents a hybrid brain-computer interface (BCI) that combines motor imagery (MI) and P300 potential for the asynchronous operation of a brain-controlled wheelchair whose design is based on a Mecanum wheel. This paradigm is completely user-centric. By sequentially performing MI tasks or paying attention to P300 flashing, the user can use eleven functions to control the wheelchair: move forward/backward, move left/right, move left45/right45, accelerate/decelerate, turn left/right, and stop. The practicality and effectiveness of the proposed approach were validated in eight subjects, all of whom achieved good performance. The preliminary results indicated that the proposed hybrid BCI system with different mental strategies operating sequentially is feasible and has potential applications for practical self-paced control.
Collapse
|
59
|
Wang H, Li T, Bezerianos A, Huang H, He Y, Chen P. The control of a virtual automatic car based on multiple patterns of motor imagery BCI. Med Biol Eng Comput 2018; 57:299-309. [PMID: 30101383 DOI: 10.1007/s11517-018-1883-3] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2017] [Accepted: 08/01/2018] [Indexed: 11/30/2022]
Abstract
Multiple degrees of freedom (DOF) commands are required for a brain-actuated virtual automatic car, which makes the brain-computer interface (BCI) control strategy a big challenge. In order to solve the challenging issue, a mixed model of BCI combining P300 potentials and motor imagery had been realized in our previous study. However, compared with single model BCI, more training procedures are needed for the mixed model and more mental workload for users to bear. In the present study, we propose a multiple patterns of motor imagery (MPMI) BCI method, which is based on the traditional two patterns of motor imagery. Our motor imagery BCI approach had been extended to multiple patterns: right-hand motor imagery, left-hand motor imagery, foot motor imagery, and both hands motor imagery resulting in turning right, turning left, acceleration, and deceleration for a virtual automatic car control. Ten healthy subjects participated in online experiments, the experimental results not only show the efficiency of our proposed MPMI-BCI strategy but also indicate that those users can control the virtual automatic car spontaneously and efficiently without any other visual attention. Furthermore, the metric of path length optimality ratio (1.23) is very encouraging and the time optimality ratio (1.28) is especially remarkable. Graphical Abstract The paradigm of multiple patterns of motor imagery detection and the relevant topographies of CSP weights for different MI patterns.
Collapse
Affiliation(s)
- Hongtao Wang
- School of Information Engineering, WuYi University, Jiangmen, 529020, China.
- Singapore Institute for Neurotechnology (SINAPSE), Center for Life Science, National University of Singapore, Singapore, 117456, Singapore.
| | - Ting Li
- School of Information Engineering, WuYi University, Jiangmen, 529020, China
| | - Anastasios Bezerianos
- Singapore Institute for Neurotechnology (SINAPSE), Center for Life Science, National University of Singapore, Singapore, 117456, Singapore
| | - Hui Huang
- School of Information Engineering, WuYi University, Jiangmen, 529020, China
| | - Yuebang He
- School of Information Engineering, WuYi University, Jiangmen, 529020, China
| | - Peng Chen
- School of Information Engineering, WuYi University, Jiangmen, 529020, China
| |
Collapse
|
60
|
Tariq M, Trivailo PM, Simic M. EEG-Based BCI Control Schemes for Lower-Limb Assistive-Robots. Front Hum Neurosci 2018; 12:312. [PMID: 30127730 PMCID: PMC6088276 DOI: 10.3389/fnhum.2018.00312] [Citation(s) in RCA: 88] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2018] [Accepted: 07/16/2018] [Indexed: 12/14/2022] Open
Abstract
Over recent years, brain-computer interface (BCI) has emerged as an alternative communication system between the human brain and an output device. Deciphered intents, after detecting electrical signals from the human scalp, are translated into control commands used to operate external devices, computer displays and virtual objects in the real-time. BCI provides an augmentative communication by creating a muscle-free channel between the brain and the output devices, primarily for subjects having neuromotor disorders, or trauma to nervous system, notably spinal cord injuries (SCI), and subjects with unaffected sensorimotor functions but disarticulated or amputated residual limbs. This review identifies the potentials of electroencephalography (EEG) based BCI applications for locomotion and mobility rehabilitation. Patients could benefit from its advancements such as wearable lower-limb (LL) exoskeletons, orthosis, prosthesis, wheelchairs, and assistive-robot devices. The EEG communication signals employed by the aforementioned applications that also provide feasibility for future development in the field are sensorimotor rhythms (SMR), event-related potentials (ERP) and visual evoked potentials (VEP). The review is an effort to progress the development of user's mental task related to LL for BCI reliability and confidence measures. As a novel contribution, the reviewed BCI control paradigms for wearable LL and assistive-robots are presented by a general control framework fitting in hierarchical layers. It reflects informatic interactions, between the user, the BCI operator, the shared controller, the robotic device and the environment. Each sub layer of the BCI operator is discussed in detail, highlighting the feature extraction, classification and execution methods employed by the various systems. All applications' key features and their interaction with the environment are reviewed for the EEG-based activity mode recognition, and presented in form of a table. It is suggested to structure EEG-BCI controlled LL assistive devices within the presented framework, for future generation of intent-based multifunctional controllers. Despite the development of controllers, for BCI-based wearable or assistive devices that can seamlessly integrate user intent, practical challenges associated with such systems exist and have been discerned, which can be constructive for future developments in the field.
Collapse
Affiliation(s)
| | | | - Milan Simic
- School of Engineering, RMIT University Melbourne, Melbourne, VIC, Australia
| |
Collapse
|
61
|
Antelis JM, Gudiño-Mendoza B, Falcón LE, Sanchez-Ante G, Sossa H. Dendrite morphological neural networks for motor task recognition from electroencephalographic signals. Biomed Signal Process Control 2018. [DOI: 10.1016/j.bspc.2018.03.010] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
62
|
Fernández-Rodríguez Á, Velasco-Álvarez F, Bonnet-Save M, Ron-Angevin R. Evaluation of Switch and Continuous Navigation Paradigms to Command a Brain-Controlled Wheelchair. Front Neurosci 2018; 12:438. [PMID: 30002615 PMCID: PMC6031925 DOI: 10.3389/fnins.2018.00438] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2017] [Accepted: 06/11/2018] [Indexed: 11/18/2022] Open
Abstract
A brain-computer interface (BCI) is a technology allowing patients with severe motor dysfunctions to use their electroencephalographic signals to create a communication channel to control devices. The objective of this paper is to study the feasibility of continuous and switch control modes for a brain-controlled wheelchair (BCW) using sensorimotor rhythms (SMR) modulated through a right-hand motor imagery task. Previous studies, which used a continuous navigation control with SMR, have reported the difficulty of maintaining the motor imagery task for a long time, especially for the forward command. The switch control has been presented as a proposal that may help to solve this issue since this task is only used temporary for either disabling or enabling the movement. Regarding the methodology, 10 of 15 able-bodied users, who had overcome the criterion of 30% error rate in the calibration phase, controlled the BCW using both paradigms. The navigation tasks consisted of a straight path divided in five sections: in three of them the users had to move forward, and in the other two the users had to maintain their position. To assess user performance in the device management, a usability approach was adopted, measuring the factors of effectiveness, efficiency, and satisfaction. Then, variables related to the time employed and commands selected by the user or parameters related to the confusion matrix were applied. In addition, the scores in NASA-TLX and two ad hoc questionnaires were considered to discuss the user experience controlling the wheelchair. Despite the results showed that the best system for a specific user relies on his/her abilities and preferences, the switch control mode obtained better accuracy (0.59 ± 0.17 for continuous and 0.72 ± 0.05 for switch). Furthermore, the switch paradigm can be recommended for the advance sections as with it users could complete the advance sections in less time (42.2 ± 28.7 s for continuous and 15.47 ± 3.43 s for switch), while the continuous mode seems to be better at keeping the wheelchair stopped (42.45 ± 16.01 s for continuous and 24.35 ± 10.94 s for switch).
Collapse
Affiliation(s)
| | | | - Manon Bonnet-Save
- IMS UMR Centre National de la Recherche Scientifique 5218, Cognitique et Ingénierie Humaine, Bordeaux INP-ENSC, Bordeaux, France
| | | |
Collapse
|
63
|
Argall BD. Autonomy in Rehabilitation Robotics: An Intersection. ANNUAL REVIEW OF CONTROL, ROBOTICS, AND AUTONOMOUS SYSTEMS 2018; 1:441-463. [PMID: 34316543 PMCID: PMC8313033 DOI: 10.1146/annurev-control-061417-041727] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Within the field of human rehabilitation, robotic machines are used both to rehabilitate the body and to perform functional tasks. Robotics autonomy able to perceive the external world and reason about high-level control decisions, however, seldom is present in these machines. For functional tasks in particular, autonomy could help to decrease the operational burden on the human and perhaps even to increase access-and this potential only grows as human motor impairments become more severe. There are however serious, and often subtle, considerations to introducing clinically-feasible robotics autonomy to rehabilitation robots and machines. Today the fields of robotics autonomy and rehabilitation robotics are largely separate. The topic of this article is at the intersection of these fields: the introduction of clinically-feasible autonomy solutions to rehabilitation robots, and opportunities for autonomy within the rehabilitation domain.
Collapse
Affiliation(s)
- Brenna D Argall
- McCormick School of Engineering and Feinberg School of Medicine, Northwestern University, Evanston, IL, USA, 60208
- Shirley Ryan AbilityLab (formerly the Rehabilitation Institute of Chicago), Chicago, IL, USA, 60611
| |
Collapse
|
64
|
Piña-Ramirez O, Valdes-Cristerna R, Yanez-Suarez O. Scenario Screen: A Dynamic and Context Dependent P300 Stimulator Screen Aimed at Wheelchair Navigation Control. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2018; 2018:7108906. [PMID: 29666663 PMCID: PMC5832133 DOI: 10.1155/2018/7108906] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/04/2017] [Revised: 12/21/2017] [Accepted: 01/04/2018] [Indexed: 12/02/2022]
Abstract
P300 spellers have been widely modified to implement nonspelling tasks. In this work, we propose a "scenario" stimulation screen that is a P300 speller variation to command a wheelchair. Our approach utilized a stimulation screen with an image background (scenario snapshot for a wheelchair) and stimulation markers arranged asymmetrically over relevant landmarks, such as suitable paths, doors, windows, and wall signs. Other scenario stimulation screen features were green/blue stimulation marker color scheme, variable Interstimulus Interval, single marker stimulus mode, and optimized stimulus sequence generator. Eighteen able-bodied subjects participated in the experiment; 78% had no experience in BCI usage. A waveform feature analysis and a Mann-Whitney U test performed over the pairs of target and nontarget coherent averages confirmed that 94% of the subjects elicit P300 (p < .005) on this modified stimulator. Least Absolute Shrinkage and Selection Operator optimization and Linear Discriminant Analysis were utilized for the automatic detection of P300. For evaluation with unseen data, target detection was computed (median sensitivity = 1.00 (0.78-1.00)), together with nontarget discrimination (median specificity = 1.00 (0.98-1.00)). The scenario screen adequately elicits P300 and seems suitable for commanding a wheelchair even when users have no previous experience on the BCI spelling task.
Collapse
Affiliation(s)
- Omar Piña-Ramirez
- Neuroimaging Research Laboratory, Electrical Engineering Department, Universidad Autonoma Metropolitana Unidad Iztapalapa, Av. San Rafael Atlixco 186, Col. Vicentina, Iztapalapa, 09340 Ciudad de México, Mexico
| | - Raquel Valdes-Cristerna
- Neuroimaging Research Laboratory, Electrical Engineering Department, Universidad Autonoma Metropolitana Unidad Iztapalapa, Av. San Rafael Atlixco 186, Col. Vicentina, Iztapalapa, 09340 Ciudad de México, Mexico
| | - Oscar Yanez-Suarez
- Neuroimaging Research Laboratory, Electrical Engineering Department, Universidad Autonoma Metropolitana Unidad Iztapalapa, Av. San Rafael Atlixco 186, Col. Vicentina, Iztapalapa, 09340 Ciudad de México, Mexico
| |
Collapse
|
65
|
Driving Control of a Powered Wheelchair Considering Uncertainty of Gaze Input in an Unknown Environment. APPLIED SCIENCES-BASEL 2018. [DOI: 10.3390/app8020267] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
66
|
|
67
|
EEG-based emergency braking intention prediction for brain-controlled driving considering one electrode falling-off. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:2494-2497. [PMID: 29060405 DOI: 10.1109/embc.2017.8037363] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
This paper proposes a novel method of electroencephalography (EEG)-based driver emergency braking intention detection system for brain-controlled driving considering one electrode falling-off. First, whether one electrode falls off is discriminated based on EEG potentials. Then, the missing signals are estimated by using the signals collected from other channels based on multivariate linear regression. Finally, a linear decoder is applied to classify driver intentions. Experimental results show that the falling-off discrimination accuracy is 99.63% on average and the correlation coefficient and root mean squared error (RMSE) between the estimated and experimental data are 0.90 and 11.43 μV, respectively, on average. Given one electrode falls off, the system accuracy of the proposed intention prediction method is significantly higher than that of the original method (95.12% VS 79.11%) and is close to that (95.95%) of the original system under normal situations (i. e., no electrode falling-off).
Collapse
|
68
|
Liu D, Chen W, Pei Z, Wang J. A brain-controlled lower-limb exoskeleton for human gait training. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2017; 88:104302. [PMID: 29092520 DOI: 10.1063/1.5006461] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
Brain-computer interfaces have been a novel approach to translate human intentions into movement commands in robotic systems. This paper describes an electroencephalogram-based brain-controlled lower-limb exoskeleton for gait training, as a proof of concept towards rehabilitation with human-in-the-loop. Instead of using conventional single electroencephalography correlates, e.g., evoked P300 or spontaneous motor imagery, we propose a novel framework integrated two asynchronous signal modalities, i.e., sensorimotor rhythms (SMRs) and movement-related cortical potentials (MRCPs). We executed experiments in a biologically inspired and customized lower-limb exoskeleton where subjects (N = 6) actively controlled the robot using their brain signals. Each subject performed three consecutive sessions composed of offline training, online visual feedback testing, and online robot-control recordings. Post hoc evaluations were conducted including mental workload assessment, feature analysis, and statistics test. An average robot-control accuracy of 80.16% ± 5.44% was obtained with the SMR-based method, while estimation using the MRCP-based method yielded an average performance of 68.62% ± 8.55%. The experimental results showed the feasibility of the proposed framework with all subjects successfully controlled the exoskeleton. The current paradigm could be further extended to paraplegic patients in clinical trials.
Collapse
Affiliation(s)
- Dong Liu
- School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
| | - Weihai Chen
- School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
| | - Zhongcai Pei
- School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
| | - Jianhua Wang
- School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
| |
Collapse
|
69
|
Huang Q, He S, Wang Q, Gu Z, Peng N, Li K, Zhang Y, Shao M, Li Y. An EOG-Based Human-Machine Interface for Wheelchair Control. IEEE Trans Biomed Eng 2017; 65:2023-2032. [PMID: 28767359 DOI: 10.1109/tbme.2017.2732479] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Nonmanual human-machine interfaces (HMIs) have been studied for wheelchair control with the aim of helping severely paralyzed individuals regain some mobility. The challenge is to rapidly, accurately, and sufficiently produce control commands, such as left and right turns, forward and backward motions, acceleration, deceleration, and stopping. In this paper, a novel electrooculogram (EOG) based HMI is proposed for wheelchair control. METHODS A total of 13 flashing buttons, each of which corresponds to a command, are presented in the graphical user interface. These buttons flash on a one-by-one manner in a predefined sequence. The user can select a button by blinking in sync with its flashes. The algorithm detects the eye blinks from a channel of vertical EOG data and determines the user's target button based on the synchronization between the detected blinks and the button's flashes. RESULTS For healthy subjects/patients with spinal cord injuries, the proposed HMI achieved an average accuracy of 96.7% / 91.7% and a response time of 3.53 s/3.67 s with 0 false positive rates (FPRs). CONCLUSION Using one channel of vertical EOG signals associated with eye blinks, the proposed HMI can accurately provide sufficient commands with a satisfactory response time. SIGNIFICANCE The proposed HMI provides a novel nonmanual approach for severely paralyzed individuals to control a wheelchair. Compared with a newly established EOG-based HMI, the proposed HMI can generate more commands with higher accuracy, lower FPR, and fewer electrodes.
Collapse
|
70
|
Libedinsky C, So RQ. Stop state classification in intracortical brain-machine-interface. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:1926-1929. [PMID: 29060269 DOI: 10.1109/embc.2017.8037225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Invasive brain-machine-interface (BMI) has the prospect to empower tetraplegic patients with independent mobility through the use of brain-controlled wheelchairs. For the practical and long-term use of such control systems, the system has to distinguish between stop and movement states and has to be robust to overcome non-stationarity in the brain signals. In this work, we investigates the non-stationarity of the stop state on neural data collected from a macaque trained to control a robotic platform to stop and move in left, right, forward directions We then propose a hybrid approach that employs both random forest and linear discriminant analysis (LDA). Using this approach, we performed offline decoding on 8 days of data collected over the course of three months during joystick control of the robotic platform. We compared the results of using the proposed approach with the use of LDA alone to perform direct classifications of stop, left, right and forward. The results showed an average performance increment of 22.7% using the proposed hybrid approach. The results yielded significant improvements during sessions where LDA showed a heavy bias towards the stop state. This suggests that the proposed hybrid approach addresses the non-stationarity in the stop state and subsequently facilitates a more accurate decoding of the movement states.
Collapse
|
71
|
Mao X, Li M, Li W, Niu L, Xian B, Zeng M, Chen G. Progress in EEG-Based Brain Robot Interaction Systems. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2017; 2017:1742862. [PMID: 28484488 PMCID: PMC5397651 DOI: 10.1155/2017/1742862] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/28/2016] [Accepted: 03/21/2017] [Indexed: 11/17/2022]
Abstract
The most popular noninvasive Brain Robot Interaction (BRI) technology uses the electroencephalogram- (EEG-) based Brain Computer Interface (BCI), to serve as an additional communication channel, for robot control via brainwaves. This technology is promising for elderly or disabled patient assistance with daily life. The key issue of a BRI system is to identify human mental activities, by decoding brainwaves, acquired with an EEG device. Compared with other BCI applications, such as word speller, the development of these applications may be more challenging since control of robot systems via brainwaves must consider surrounding environment feedback in real-time, robot mechanical kinematics, and dynamics, as well as robot control architecture and behavior. This article reviews the major techniques needed for developing BRI systems. In this review article, we first briefly introduce the background and development of mind-controlled robot technologies. Second, we discuss the EEG-based brain signal models with respect to generating principles, evoking mechanisms, and experimental paradigms. Subsequently, we review in detail commonly used methods for decoding brain signals, namely, preprocessing, feature extraction, and feature classification, and summarize several typical application examples. Next, we describe a few BRI applications, including wheelchairs, manipulators, drones, and humanoid robots with respect to synchronous and asynchronous BCI-based techniques. Finally, we address some existing problems and challenges with future BRI techniques.
Collapse
Affiliation(s)
- Xiaoqian Mao
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China
| | - Mengfan Li
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China
| | - Wei Li
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China
- Department of Computer & Electrical Engineering and Computer Science, California State University, Bakersfield, CA 93311, USA
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Shenyang, Liaoning 110016, China
| | - Linwei Niu
- Department of Math and Computer Science, West Virginia State University, Institute, WV 25112, USA
| | - Bin Xian
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China
| | - Ming Zeng
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China
| | - Genshe Chen
- Intelligent Fusion Technology, Inc., Germantown, MD 20876, USA
| |
Collapse
|
72
|
Puanhvuan D, Khemmachotikun S, Wechakarn P, Wijarn B, Wongsawat Y. Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cogn Neurodyn 2017; 11:117-134. [PMID: 28348644 PMCID: PMC5350091 DOI: 10.1007/s11571-017-9424-6] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2016] [Revised: 11/25/2016] [Accepted: 02/03/2017] [Indexed: 11/28/2022] Open
Abstract
Currently, electric wheelchairs are commonly used to improve mobility in disabled people. In severe cases, the user is unable to control the wheelchair by themselves because his/her motor functions are disabled. To restore mobility function, a brain-controlled wheelchair (BCW) would be a promising system that would allow the patient to control the wheelchair by their thoughts. P300 is a reliable brain electrical signal, a component of visual event-related potentials (ERPs), that could be used for interpreting user commands. This research aimed to propose a prototype BCW to allowed severe motor disabled patients to practically control a wheelchair for use in their home environment. The users were able to select from 9 possible destination commands in the automatic mode and from 4 directional commands (forward, backward, turn left and right) in the shared-control mode. These commands were selected via the designed P300 processing system. The wheelchair was steered to the desired location by the implemented navigation system. Safety of the user was ensured during wheelchair navigation due to the included obstacle detection and avoidance features. A combination of P300 and EOG was used as a hybrid BCW system. The user could fully operate the system such as enabling P300 detection system, mode shifting and stop/cancelation command by performing a different consecutive blinks to generate eye blinking patterns. The results revealed that the prototype BCW could be operated in either of the proposed modes. With the new design of the LED-based P300 stimulator, the average accuracies of the P300 detection algorithm in the shared-control and automatic modes were 95.31 and 83.42% with 3.09 and 3.79 bits/min, respectively. The P300 classification error was acceptable, as the user could cancel an incorrect command by blinking 2 times. Moreover, the proposed navigation system had a flexible design that could be interfaced with other assistive technologies. This research developed 3 alternative input modules: an eye tracker module and chin and hand controller modules. The user could select the most suitable assistive technology based on his/her level of disability. Other existing assistive technologies could also be connected to the proposed system in the future using the same protocol.
Collapse
Affiliation(s)
- Dilok Puanhvuan
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol Unversity, 25/25, Putthamonthol 4 Road, Salaya, Putthamonthol, Nakhon Pathom 73170 Thailand
| | - Sarawin Khemmachotikun
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol Unversity, 25/25, Putthamonthol 4 Road, Salaya, Putthamonthol, Nakhon Pathom 73170 Thailand
| | - Pongsakorn Wechakarn
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol Unversity, 25/25, Putthamonthol 4 Road, Salaya, Putthamonthol, Nakhon Pathom 73170 Thailand
| | - Boonyanuch Wijarn
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol Unversity, 25/25, Putthamonthol 4 Road, Salaya, Putthamonthol, Nakhon Pathom 73170 Thailand
| | - Yodchanan Wongsawat
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol Unversity, 25/25, Putthamonthol 4 Road, Salaya, Putthamonthol, Nakhon Pathom 73170 Thailand
| |
Collapse
|
73
|
Kaongoen N, Jo S. A novel hybrid auditory BCI paradigm combining ASSR and P300. J Neurosci Methods 2017; 279:44-51. [DOI: 10.1016/j.jneumeth.2017.01.011] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2016] [Accepted: 01/14/2017] [Indexed: 10/20/2022]
|
74
|
Zhang R, Wang Q, Li K, He S, Qin S, Feng Z, Chen Y, Song P, Yang T, Zhang Y, Yu Z, Hu Y, Shao M, Li Y. A BCI-based Environmental Control System for Patients with Severe Spinal Cord Injuries. IEEE Trans Biomed Eng 2017; 64:1959-1971. [PMID: 28092509 DOI: 10.1109/tbme.2016.2628861] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
This study proposes an event-related potential (ERP) BCI-based environmental control system that integrates household electrical appliances, a nursing bed, and an intelligent wheelchair to provide daily assistance to paralyzed patients with severe spinal cord injuries (SCIs). METHODS An asynchronous mode is used to switch the environmental control system on or off or to select a device (e.g., a TV) for achieving selfpaced control. In the asynchronous mode, we introduce several pseudo-keys and a verification mechanism to effectively reduce the false operation rate. By contrast, when the user selects a function of the device (e.g., a TV channel), a synchronous mode is used to improve the accuracy and speed of BCI detection. Two experiments involving six SCI patients were conducted separately in a nursing bed and a wheelchair, and the patients were instructed to control the nursing bed, the wheelchair, and household electrical appliances (an electric light, an air conditioner, and a TV). RESULTS The average false rate of BCI commands in the control state was 10.4%, whereas the average false operation ratio was 4.9% (a false BCI command might not necessarily result in a false operation according to our system design). During the idle state, there was an average of 0.97 false positives per minute, which did not result in any false operations. CONCLUSION All SCI patients could use the proposed ERP BCIbased environmental control system satisfactorily. SIGNIFICANCE The proposed ERP-based environmental control system could be used to assist patients with severe SCIs in their daily lives.
Collapse
|
75
|
Seanez-Gonzalez I, Pierella C, Farshchiansadegh A, Thorp EB, Abdollahi F, Pedersen JP, Sandro Mussa-Ivaldi FA. Static Versus Dynamic Decoding Algorithms in a Non-Invasive Body-Machine Interface. IEEE Trans Neural Syst Rehabil Eng 2016; 25:893-905. [PMID: 28092564 DOI: 10.1109/tnsre.2016.2640360] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In this study, we consider a non-invasive body-machine interface that captures body motions still available to people with spinal cord injury (SCI) and maps them into a set of signals for controlling a computer user interface while engaging in a sustained level of mobility and exercise. We compare the effectiveness of two decoding algorithms that transform a high-dimensional body-signal vector into a lower dimensional control vector on six subjects with high-level SCI and eight controls. One algorithm is based on a static map from current body signals to the current value of the control vector set through principal component analysis (PCA), the other on dynamic mapping a segment of body signals to the value and the temporal derivatives of the control vector set through a Kalman filter. SCI and control participants performed straighter and smoother cursor movements with the Kalman algorithm during center-out reaching, but their movements were faster and more precise when using PCA. All participants were able to use the BMI's continuous, two-dimensional control to type on a virtual keyboard and play pong, and performance with both algorithms was comparable. However, seven of eight control participants preferred PCA as their method of virtual wheelchair control. The unsupervised PCA algorithm was easier to train and seemed sufficient to achieve a higher degree of learnability and perceived ease of use.
Collapse
|
76
|
Libedinsky C, So R, Xu Z, Kyar TK, Ho D, Lim C, Chan L, Chua Y, Yao L, Cheong JH, Lee JH, Vishal KV, Guo Y, Chen ZN, Lim LK, Li P, Liu L, Zou X, Ang KK, Gao Y, Ng WH, Han BS, Chng K, Guan C, Je M, Yen SC. Independent Mobility Achieved through a Wireless Brain-Machine Interface. PLoS One 2016; 11:e0165773. [PMID: 27802344 PMCID: PMC5089763 DOI: 10.1371/journal.pone.0165773] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2015] [Accepted: 10/18/2016] [Indexed: 11/19/2022] Open
Abstract
Individuals with tetraplegia lack independent mobility, making them highly dependent on others to move from one place to another. Here, we describe how two macaques were able to use a wireless integrated system to control a robotic platform, over which they were sitting, to achieve independent mobility using the neuronal activity in their motor cortices. The activity of populations of single neurons was recorded using multiple electrode arrays implanted in the arm region of primary motor cortex, and decoded to achieve brain control of the platform. We found that free-running brain control of the platform (which was not equipped with any machine intelligence) was fast and accurate, resembling the performance achieved using joystick control. The decoding algorithms can be trained in the absence of joystick movements, as would be required for use by tetraplegic individuals, demonstrating that the non-human primate model is a good pre-clinical model for developing such a cortically-controlled movement prosthetic. Interestingly, we found that the response properties of some neurons differed greatly depending on the mode of control (joystick or brain control), suggesting different roles for these neurons in encoding movement intention and movement execution. These results demonstrate that independent mobility can be achieved without first training on prescribed motor movements, opening the door for the implementation of this technology in persons with tetraplegia.
Collapse
Affiliation(s)
- Camilo Libedinsky
- Department of Psychology, National University of Singapore, Singapore, Singapore
- Singapore Institute for Neurotechnology, National University of Singapore, Singapore, Singapore
- Singapore Institute for Clinical Sciences, A*STAR, Singapore, Singapore
| | - Rosa So
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | - Zhiming Xu
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | - Toe K. Kyar
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | - Duncun Ho
- National Neuroscience Institute, Singapore, Singapore
| | - Clement Lim
- Singapore Institute for Neurotechnology, National University of Singapore, Singapore, Singapore
| | - Louiza Chan
- Singapore Institute for Clinical Sciences, A*STAR, Singapore, Singapore
| | - Yuanwei Chua
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | - Lei Yao
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Jia Hao Cheong
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Jung Hyup Lee
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | | | - Yongxin Guo
- Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore
| | - Zhi Ning Chen
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
- Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore
| | - Lay K. Lim
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Peng Li
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Lei Liu
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Xiaodan Zou
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Kai K. Ang
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | - Yuan Gao
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Wai Hoe Ng
- National Neuroscience Institute, Singapore, Singapore
| | - Boon Siew Han
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
| | - Keefe Chng
- Singapore Institute for Clinical Sciences, A*STAR, Singapore, Singapore
| | - Cuntai Guan
- Institute for Infocomm Research, A*STAR, Singapore, Singapore
- School of Computer Science and Engineering, and School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore
| | - Minkyu Je
- Institute of Microelectronics, A*STAR, Singapore, Singapore
| | - Shih-Cheng Yen
- Singapore Institute for Neurotechnology, National University of Singapore, Singapore, Singapore
- Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore
- * E-mail:
| |
Collapse
|
77
|
Ben Taher F, Ben Amor N, Jallouli M, Ben Hammouda A, Dghim O. A multi levels data fusion approach for an electric wheelchair control. Biomed Eng Lett 2016. [DOI: 10.1007/s13534-016-0227-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022] Open
|
78
|
Fernández-Rodríguez Á, Velasco-Álvarez F, Ron-Angevin R. Review of real brain-controlled wheelchairs. J Neural Eng 2016; 13:061001. [PMID: 27739401 DOI: 10.1088/1741-2560/13/6/061001] [Citation(s) in RCA: 45] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
This paper presents a review of the state of the art regarding wheelchairs driven by a brain-computer interface. Using a brain-controlled wheelchair (BCW), disabled users could handle a wheelchair through their brain activity, granting autonomy to move through an experimental environment. A classification is established, based on the characteristics of the BCW, such as the type of electroencephalographic signal used, the navigation system employed by the wheelchair, the task for the participants, or the metrics used to evaluate the performance. Furthermore, these factors are compared according to the type of signal used, in order to clarify the differences among them. Finally, the trend of current research in this field is discussed, as well as the challenges that should be solved in the future.
Collapse
|
79
|
Toward brain-actuated car applications: Self-paced control with a motor imagery-based brain-computer interface. Comput Biol Med 2016; 77:148-55. [DOI: 10.1016/j.compbiomed.2016.08.010] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2016] [Revised: 08/11/2016] [Accepted: 08/11/2016] [Indexed: 11/17/2022]
|
80
|
Brain-Computer Interface for Control of Wheelchair Using Fuzzy Neural Networks. BIOMED RESEARCH INTERNATIONAL 2016; 2016:9359868. [PMID: 27777953 PMCID: PMC5061989 DOI: 10.1155/2016/9359868] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/05/2016] [Revised: 07/30/2016] [Accepted: 08/21/2016] [Indexed: 11/17/2022]
Abstract
The design of brain-computer interface for the wheelchair for physically disabled people is presented. The design of the proposed system is based on receiving, processing, and classification of the electroencephalographic (EEG) signals and then performing the control of the wheelchair. The number of experimental measurements of brain activity has been done using human control commands of the wheelchair. Based on the mental activity of the user and the control commands of the wheelchair, the design of classification system based on fuzzy neural networks (FNN) is considered. The design of FNN based algorithm is used for brain-actuated control. The training data is used to design the system and then test data is applied to measure the performance of the control system. The control of the wheelchair is performed under real conditions using direction and speed control commands of the wheelchair. The approach used in the paper allows reducing the probability of misclassification and improving the control accuracy of the wheelchair.
Collapse
|
81
|
Bi L, Lu Y, Fan X, Lian J, Liu Y. Queuing Network Modeling of Driver EEG Signals-Based Steering Control. IEEE Trans Neural Syst Rehabil Eng 2016; 25:1117-1124. [PMID: 27705860 DOI: 10.1109/tnsre.2016.2614003] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Directly using brain signals rather than limbs to steer a vehicle may not only help disabled people to control an assistive vehicle, but also provide a complementary means of control for a wider driving community. In this paper, to simulate and predict driver performance in steering a vehicle with brain signals, we propose a driver brain-controlled steering model by combining an extended queuing network-based driver model with a brain-computer interface (BCI) performance model. Experimental results suggest that the proposed driver brain-controlled steering model has performance close to that of real drivers with good performance in brain-controlled driving. The brain-controlled steering model has potential values in helping develop a brain-controlled assistive vehicle. Furthermore, this study provides some insights into the simulation and prediction of the performance of using BCI systems to control other external devices (e.g., mobile robots).
Collapse
|
82
|
A multi-command SSVEP-based BCI system based on single flickering frequency half-field steady-state visual stimulation. Med Biol Eng Comput 2016; 55:965-977. [PMID: 27651060 DOI: 10.1007/s11517-016-1560-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2015] [Accepted: 09/02/2016] [Indexed: 10/21/2022]
Abstract
Steady-state visual evoked potentials (SSVEPs) are widely employed in brain-computer interface (BCI) applications, especially to control machines. However, the use of SSVEPs leads to eye fatigue and causes lower accuracy over the long term, particularly when multi-commands are required. Therefore, this paper proposes a half-field steady-state visual stimulation pattern and paradigm to increase the limited number of commands that can be achieved with existing SSVEP-based BCI methods. Following the theory of vision perception and existing half-field SSVEP-based BCI systems, the new stimulation pattern generates four commands using only one frequency flickering stimulus and has an average classification accuracy of approximately 75 %. According to the proposed stimulus pattern, using only one frequency without requiring users to stare directly at the flickering stimulus allows users to easily focus on the system and experience less visual fatigue compared to existing systems. Furthermore, new half-field SSVEP-based BCI systems are proposed, incorporating our proposed feature extraction and decision-making algorithm. Extracting the signal from the occipital area and using a reference electrode position at the parietal area yielded better results compared to the central area. In addition, we recommend using an LED or LCD as the visual stimulus device (at the recommended size), which yielded comparable results to our proposed feature extraction and decision-making algorithm. Finally, an application of the proposed system is demonstrated for real-time television control.
Collapse
|
83
|
Abstract
A vital perception in designing human-machine interfaces for people with disabilities is hands-free control. It is important in setting up communication among a machine and the part of human body. Nowadays, hands-free control is extensively used to help disabled people to control the wheelchair. The hands-free control functions are not only helpful to elderly and disabled, but also useful for normal people who might require using their hands for other tasks at the same time. As a result, normal people are capable of manipulating multiple tasks at the same time. The wheelchair is manoeuvred using brainwaves, eyes and facial muscles movements, voice, tongue, gesture. The mobility using a powered wheelchair improves the quality of life of the aged people and the people with mobility impairments significantly. These powered wheelchairs can be actuated using various high-level control interfaces. In this review, the use of natural biological signals and non-biological signal to do the hands-free control of powered wheelchair is presented in this article.
Collapse
Affiliation(s)
- Sharmila Ashok
- a School of Electrical Engineering , VIT University , Vellore , India
| |
Collapse
|
84
|
Liu JC, Chou HC, Chen CH, Lin YT, Kuo CH. Time-Shift Correlation Algorithm for P300 Event Related Potential Brain-Computer Interface Implementation. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2016; 2016:3039454. [PMID: 27579033 PMCID: PMC4992804 DOI: 10.1155/2016/3039454] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/02/2016] [Revised: 05/23/2016] [Accepted: 06/22/2016] [Indexed: 11/17/2022]
Abstract
A high efficient time-shift correlation algorithm was proposed to deal with the peak time uncertainty of P300 evoked potential for a P300-based brain-computer interface (BCI). The time-shift correlation series data were collected as the input nodes of an artificial neural network (ANN), and the classification of four LED visual stimuli was selected as the output node. Two operating modes, including fast-recognition mode (FM) and accuracy-recognition mode (AM), were realized. The proposed BCI system was implemented on an embedded system for commanding an adult-size humanoid robot to evaluate the performance from investigating the ground truth trajectories of the humanoid robot. When the humanoid robot walked in a spacious area, the FM was used to control the robot with a higher information transfer rate (ITR). When the robot walked in a crowded area, the AM was used for high accuracy of recognition to reduce the risk of collision. The experimental results showed that, in 100 trials, the accuracy rate of FM was 87.8% and the average ITR was 52.73 bits/min. In addition, the accuracy rate was improved to 92% for the AM, and the average ITR decreased to 31.27 bits/min. due to strict recognition constraints.
Collapse
Affiliation(s)
- Ju-Chi Liu
- Department of Internal Medicine, School of Medicine, College of Medicine, Taipei Medical University, Taipei 110, Taiwan
- Division of Cardiology, Department of Internal Medicine, Shuang Ho Hospital, New Taipei City 235, Taiwan
| | - Hung-Chyun Chou
- Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan
| | - Chien-Hsiu Chen
- Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan
| | - Yi-Tseng Lin
- Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan
| | - Chung-Hsien Kuo
- Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan
- Department of Biomedical Engineering, National Defense Medical Center, Taipei 114, Taiwan
| |
Collapse
|
85
|
Kim KT, Suk HI, Lee SW. Commanding a Brain-Controlled Wheelchair Using Steady-State Somatosensory Evoked Potentials. IEEE Trans Neural Syst Rehabil Eng 2016; 26:654-665. [PMID: 27514060 DOI: 10.1109/tnsre.2016.2597854] [Citation(s) in RCA: 42] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In this work, we propose a novel brain-controlled wheelchair, one of the major applications of brain-machine interfaces (BMIs), that allows an individual with mobility impairments to perform daily living activities independently. Specifically, we propose to use a steady-state somatosensory evoked potential (SSSEP) paradigm, which elicits brain responses to tactile stimulation of specific frequencies, for a user's intention to control a wheelchair. In our system, a user had three possible commands by concentrating on one of three vibration stimuli, which were attached to the left-hand, right-hand, and right-foot, to selectively control the wheelchair. The three stimuli were associated with three wheelchair commands: turn-left, turn-right, and move-forward. From a machine learning perspective, we also devise a novel feature representation by combining spatial and spectral characteristics of brain signals. In order to validate the effectiveness of the proposed SSSEP-based system, we considered two different tasks: 1) a simple obstacle-avoidance task within a limited time and; 2) a driving task along the predefined trajectory of about 40 m length, where there were a narrow pathway, a door, and obstacles. In both experiments, we recruited 12 subjects and compared the average time of motor imagery (MI) and SSSEP-based controls to complete the task. With the SSSEP-based control, all subjects successfully completed the task without making any collision while four subjects failed it with MI-based control. It is also noteworthy that in terms of the average time to complete the task, the SSSEP-based control outperformed the MI-based control. In the other more challenging task, all subjects successfully reached the target location.
Collapse
|
86
|
Tidoni E, Gergondet P, Fusco G, Kheddar A, Aglioti SM. The Role of Audio-Visual Feedback in a Thought-Based Control of a Humanoid Robot: A BCI Study in Healthy and Spinal Cord Injured People. IEEE Trans Neural Syst Rehabil Eng 2016; 25:772-781. [PMID: 28113631 DOI: 10.1109/tnsre.2016.2597863] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The efficient control of our body and successful interaction with the environment are possible through the integration of multisensory information. Brain-computer interface (BCI) may allow people with sensorimotor disorders to actively interact in the world. In this study, visual information was paired with auditory feedback to improve the BCI control of a humanoid surrogate. Healthy and spinal cord injured (SCI) people were asked to embody a humanoid robot and complete a pick-and-place task by means of a visual evoked potentials BCI system. Participants observed the remote environment from the robot's perspective through a head mounted display. Human-footsteps and computer-beep sounds were used as synchronous/asynchronous auditory feedback. Healthy participants achieved better placing accuracy when listening to human footstep sounds relative to a computer-generated sound. SCI people demonstrated more difficulty in steering the robot during asynchronous auditory feedback conditions. Importantly, subjective reports highlighted that the BCI mask overlaying the display did not limit the observation of the scenario and the feeling of being in control of the robot. Overall, the data seem to suggest that sensorimotor-related information may improve the control of external devices. Further studies are required to understand how the contribution of residual sensory channels could improve the reliability of BCI systems.
Collapse
|
87
|
Zhang R, Li Y, Yan Y, Zhang H, Wu S. An intelligent wheelchair based on automated navigation and BCI techniques. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2016; 2014:1302-5. [PMID: 25570205 DOI: 10.1109/embc.2014.6943837] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In this paper, we propose an intelligent wheelchair system that relies on a brain computer interface (BCI) and automatic navigation. When in operation, candidate destinations and waypoints are automatically generated on the basis of the current environment. Then, the user selects a destination using a P300-based BCI. Finally, the navigation system plans a path and navigates the wheelchair to the determined destination. While the wheelchair is in motion, the user can issue a stop command with the BCI. Using our system, the mental burden of the user can be alleviated to a large degree. Furthermore, our system can adapt to changes in the environment. The experimental results demonstrated the effectiveness of our system.
Collapse
|
88
|
He S, Zhang R, Wang Q, Chen Y, Yang T, Feng Z, Zhang Y, Shao M, Li Y. A P300-Based Threshold-Free Brain Switch and Its Application in Wheelchair Control. IEEE Trans Neural Syst Rehabil Eng 2016; 25:715-725. [PMID: 27416603 DOI: 10.1109/tnsre.2016.2591012] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The key issue of electroencephalography (EEG)-based brain switches is to detect the control and idle states in an asynchronous manner. Most existing methods rely on a threshold. However, it is often time consuming to select a satisfactory threshold, and the chosen threshold might be inappropriate over a long period of time due to the variability of the EEG signals. This paper presents a new P300-based threshold-free brain switch. Specifically, one target button and three pseudo buttons, which are intensified in a random order to produce P300 potential, are set in the graphical user interface. The user can issue a switch command by focusing on the target button. Two support vector machine (SVM) classifiers, namely, SVM1 and SVM2, are used in the detection algorithm. During detection, we first obtained four SVM scores, corresponding to the four flashing buttons, by applying SVM1 to the ongoing EEG. If the SVM score corresponding to the target button was negative or not at the maximum, then an idle state was determined. Moreover, if the target button had a maximum and positive score, then we fed the four SVM scores as features into SVM2 to further discriminate the control and idle states. As an application, this brain switch was used to produce a start/stop command for an intelligent wheelchair, of which the left, right, forward, backward functions were carried out by an autonomous navigation system. Several experiments were conducted with eight healthy subjects and five patients with spinal cord injuries (SCIs). The experimental results not only demonstrated the effectiveness of our approach but also illustrated the potential application for patients with SCIs.
Collapse
|
89
|
UPADHYAY R, PADHY PK, KANKAR PK. APPLICATION OF S-TRANSFORM FOR AUTOMATED DETECTION OF VIGILANCE LEVEL USING EEG SIGNALS. J BIOL SYST 2016. [DOI: 10.1142/s0218339016500017] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
This paper presents an S-transform-based Electroencephalogram channel optimization and feature extraction methodology for monitoring mental vigilance level of humans. Vigilance level detection methodology consists of four steps. In the first stage, two types of Electroencephalogram signals (alert and drowsy) are acquired from 30 healthy subjects and decomposed into sub-bands using the S-transform. In the second stage, permutation entropy of the S-transform coefficients is calculated and Electroencephalogram channel optimization is performed. S-transform-based statistical features are computed from the optimized Electroencephalogram channels, in the third stage. In the fourth stage, artificial intelligence techniques such as Least Square-Support Vector Machine, Artificial Neural Network and Naive Bayes Classifier are used for the classification of Electroencephalogram signals using extracted features. The performance of the feature extraction methodology is tested on the Electroencephalogram data of 30 healthy subjects. Experimental results ensured the effectiveness of proposed methodology for the estimation of mental vigilance level by using Electroencephalogram signals. It is observed that the Artificial Neural Network classifier is a good candidate for pre-emptive automatic vigilance level detection system for Brain-Computer Interface applications.
Collapse
Affiliation(s)
- R. UPADHYAY
- Electronics and Communication Engineering Discipline, PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India
| | - P. K. PADHY
- Electronics and Communication Engineering Discipline, PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India
| | - P. K. KANKAR
- Mechanical Engineering Discipline, PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India
| |
Collapse
|
90
|
Xu R, Jiang N, Dosen S, Lin C, Mrachacz-Kersting N, Dremstrup K, Farina D. Endogenous Sensory Discrimination and Selection by a Fast Brain Switch for a High Transfer Rate Brain-Computer Interface. IEEE Trans Neural Syst Rehabil Eng 2016; 24:901-10. [PMID: 26849869 DOI: 10.1109/tnsre.2016.2523565] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In this study, we present a novel multi-class brain-computer interface (BCI) for communication and control. In this system, the information processing is shared by the algorithm (computer) and the user (human). Specifically, an electro-tactile cycle was presented to the user, providing the choice (class) by delivering timely sensory input. The user discriminated these choices by his/her endogenous sensory ability and selected the desired choice with an intuitive motor task. This selection was detected by a fast brain switch based on real-time detection of movement-related cortical potentials from scalp EEG. We demonstrated the feasibility of such a system with a four-class BCI, yielding a true positive rate of ∼ 80% and ∼ 70%, and an information transfer rate of ∼ 7 bits/min and ∼ 5 bits/min, for the movement and imagination selection command, respectively. Furthermore, when the system was extended to eight classes, the throughput of the system was improved, demonstrating the capability of accommodating a large number of classes. Combining the endogenous sensory discrimination with the fast brain switch, the proposed system could be an effective, multi-class, gaze-independent BCI system for communication and control applications.
Collapse
|
91
|
Padir T. Towards personalized smart wheelchairs: Lessons learned from discovery interviews. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2016; 2015:5016-9. [PMID: 26737418 DOI: 10.1109/embc.2015.7319518] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
We posit that it is necessary to investigate the personalization of smart wheelchairs in three aspects interfaces for interaction, controllers for action (top-level, middle-level, and low-level), and feedback in interaction. Our team has been selected as an Innovation Corps (I-Corps) Team by the National Science Foundation to pursue customer discovery research to explore the commercial viability of smart wheelchairs. Through the process, our team has performed more than 110 interviews with powered wheelchair users, manufacturers, therapists, policy makers, and non-profit organization staff. Our findings revealed that the acceptability of fully autonomous systems by the users is still challenging and highly-dependent on the severity of the disability. Furthermore, the cost, ease-of-use and personalization are the most important factors in commercializing smart wheelchair technologies.
Collapse
|
92
|
Li M, Li W, Zhou H. Increasing N200 Potentials Via Visual Stimulus Depicting Humanoid Robot Behavior. Int J Neural Syst 2016; 26:1550039. [DOI: 10.1142/s0129065715500392] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Achieving recognizable visual event-related potentials plays an important role in improving the success rate in telepresence control of a humanoid robot via N200 or P300 potentials. The aim of this research is to intensively investigate ways to induce N200 potentials with obvious features by flashing robot images (images with meaningful information) and by flashing pictures containing only solid color squares (pictures with incomprehensible information). Comparative studies have shown that robot images evoke N200 potentials with recognizable negative peaks at approximately 260[Formula: see text]ms in the frontal and central areas. The negative peak amplitudes increase, on average, from [Formula: see text]V, induced by flashing the squares, to [Formula: see text]V, induced by flashing the robot images. The data analyses support that the N200 potentials induced by the robot image stimuli exhibit recognizable features. Compared with the square stimuli, the robot image stimuli increase the average accuracy rate by 9.92%, from 83.33% to 93.25%, and the average information transfer rate by 24.56[Formula: see text]bits/min, from 72.18[Formula: see text]bits/min to 96.74 bits/min, in a single repetition. This finding implies that the robot images might provide the subjects with more information to understand the visual stimuli meanings and help them more effectively concentrate on their mental activities.
Collapse
Affiliation(s)
- Mengfan Li
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, P. R. China
| | - Wei Li
- School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, P. R. China
- Department of Computer & Electrical Engineering and Computer Science, California State University, 9001 Stockdale Highway, Bakersfield, California 93311, USA
| | - Huihui Zhou
- McGovern Institute for Brain Research Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Ave, Shenzhen, Guangdong 518055, P. R. China
| |
Collapse
|
93
|
Morris S, Hirata M, Sugata H, Goto T, Matsushita K, Yanagisawa T, Saitoh Y, Kishima H, Yoshimine T. Patient-specific cortical electrodes for sulcal and gyral implantation. IEEE Trans Biomed Eng 2015; 62:1034-41. [PMID: 25029330 DOI: 10.1109/tbme.2014.2329812] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
PURPOSE Noninvasive localization of certain brain functions may be mapped on a millimetre level. However, the interelectrode spacing of common clinical brain surface electrodes still remains around 10 mm. Here, we present details on development of electrodes for attaining higher quality electrocorticographic signals for use in functional brain mapping and brain-machine interface (BMI) technologies. METHODS We used platinum-plate-electrodes of 1-mm diameter to produce sheet electrodes after the creation of individualized molds using a 3-D printer and a press system that sandwiched the electrodes between personalized silicone sheets. RESULTS We created arrays to fit the surface curvature of the brain and inside the central sulcus, with interelectrode distances of 2.5 mm (a density of 16 times previous standard types). Rat experiments undertaken indicated no long term toxicity. We were also able to custom design, rapidly manufacture, safely implant, and confirm the efficacy of personalized electrodes, including the capability to attain meaningful high-gamma-band information in an amyotrophic lateral sclerosis patient. CONCLUSION We developed cortical sheet electrodes with a high-spatial resolution, tailor-made to match an individual's brain. SIGNIFICANCE This sheet electrode may contribute to the higher performance of BMI's.
Collapse
|
94
|
Iturrate I, Montesano L, Minguez J. Shared-control brain-computer interface for a two dimensional reaching task using EEG error-related potentials. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2015; 2013:5258-62. [PMID: 24110922 DOI: 10.1109/embc.2013.6610735] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
One of the main problems of EEG-based brain computer interfaces (BCIs) is their low information rate, thus for complex tasks the user needs large amounts of time to solve the task. In an attempt to reduce this time and improve the application robustness, recent works have explored shared-control strategies where the device does not only execute the decoded commands, but it is also involved in executing the task. This work proposes a shared-control BCI using error potentials for a 2D reaching task with discrete actions and states. The proposed system has several interesting properties: the system is scalable without increasing the complexity of the user's mental task; the interaction is natural for the user, as the mental task is to monitor the device performance to promote its task learning (in this context the reaching task); and the system has the potential to be combined with additional brain signals to recover or learn from interaction errors. Online control experiments were performed with four subjects, showing that it was possible to reach a goal location from any starting point within a 5×5 grid in around 23 actions (about 19 seconds of EEG signal), both with fixed goals and goals freely chosen by the users.
Collapse
|
95
|
Omedes J, Iturrate I, Minguez J, Montesano L. Analysis and asynchronous detection of gradually unfolding errors during monitoring tasks. J Neural Eng 2015; 12:056001. [DOI: 10.1088/1741-2560/12/5/056001] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
96
|
Carlson T, Tonin L, Perdikis S, Leeb R, del R Millán J. A hybrid BCI for enhanced control of a telepresence robot. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2015; 2013:3097-100. [PMID: 24110383 DOI: 10.1109/embc.2013.6610196] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Motor-disabled end users have successfully driven a telepresence robot in a complex environment using a Brain-Computer Interface (BCI). However, to facilitate the interaction aspect that underpins the notion of telepresence, users must be able to voluntarily and reliably stop the robot at any moment, not just drive from point to point. In this work, we propose to exploit the user's residual muscular activity to provide a fast and reliable control channel, which can start/stop the telepresence robot at any moment. Our preliminary results show that not only does this hybrid approach increase the accuracy, but it also helps to reduce the workload and was the preferred control paradigm of all the participants.
Collapse
|
97
|
Human-Machine Interface for the Control of Multi-Function Systems Based on Electrocutaneous Menu: Application to Multi-Grasp Prosthetic Hands. PLoS One 2015; 10:e0127528. [PMID: 26069961 PMCID: PMC4466571 DOI: 10.1371/journal.pone.0127528] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2014] [Accepted: 04/16/2015] [Indexed: 11/19/2022] Open
Abstract
Modern assistive devices are very sophisticated systems with multiple degrees of freedom. However, an effective and user-friendly control of these systems is still an open problem since conventional human-machine interfaces (HMI) cannot easily accommodate the system’s complexity. In HMIs, the user is responsible for generating unique patterns of command signals directly triggering the device functions. This approach can be difficult to implement when there are many functions (necessitating many command patterns) and/or the user has a considerable impairment (limited number of available signal sources). In this study, we propose a novel concept for a general-purpose HMI where the controller and the user communicate bidirectionally to select the desired function. The system first presents possible choices to the user via electro-tactile stimulation; the user then acknowledges the desired choice by generating a single command signal. Therefore, the proposed approach simplifies the user communication interface (one signal to generate), decoding (one signal to recognize), and allows selecting from a number of options. To demonstrate the new concept the method was used in one particular application, namely, to implement the control of all the relevant functions in a state of the art commercial prosthetic hand without using any myoelectric channels. We performed experiments in healthy subjects and with one amputee to test the feasibility of the novel approach. The results showed that the performance of the novel HMI concept was comparable or, for some outcome measures, better than the classic myoelectric interfaces. The presented approach has a general applicability and the obtained results point out that it could be used to operate various assistive systems (e.g., prosthesis vs. wheelchair), or it could be integrated into other control schemes (e.g., myoelectric control, brain-machine interfaces) in order to improve the usability of existing low-bandwidth HMIs.
Collapse
|
98
|
Thorp EB, Abdollahi F, Chen D, Farshchiansadegh A, Lee MH, Pedersen JP, Pierella C, Roth EJ, Seanez Gonzalez I, Mussa-Ivaldi FA. Upper Body-Based Power Wheelchair Control Interface for Individuals With Tetraplegia. IEEE Trans Neural Syst Rehabil Eng 2015; 24:249-60. [PMID: 26054071 DOI: 10.1109/tnsre.2015.2439240] [Citation(s) in RCA: 41] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Many power wheelchair control interfaces are not sufficient for individuals with severely limited upper limb mobility. The majority of controllers that do not rely on coordinated arm and hand movements provide users a limited vocabulary of commands and often do not take advantage of the user's residual motion. We developed a body-machine interface (BMI) that leverages the flexibility and customizability of redundant control by using high dimensional changes in shoulder kinematics to generate proportional control commands for a power wheelchair. In this study, three individuals with cervical spinal cord injuries were able to control a power wheelchair safely and accurately using only small shoulder movements. With the BMI, participants were able to achieve their desired trajectories and, after five sessions driving, were able to achieve smoothness that was similar to the smoothness with their current joystick. All participants were twice as slow using the BMI however improved with practice. Importantly, users were able to generalize training controlling a computer to driving a power wheelchair, and employed similar strategies when controlling both devices. Overall, this work suggests that the BMI can be an effective wheelchair control interface for individuals with high-level spinal cord injuries who have limited arm and hand control.
Collapse
|
99
|
Zhang R, Li Y, Yan Y, Zhang H, Wu S, Yu T, Gu Z. Control of a Wheelchair in an Indoor Environment Based on a Brain-Computer Interface and Automated Navigation. IEEE Trans Neural Syst Rehabil Eng 2015; 24:128-39. [PMID: 26054072 DOI: 10.1109/tnsre.2015.2439298] [Citation(s) in RCA: 88] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The concept of controlling a wheelchair using brain signals is promising. However, the continuous control of a wheelchair based on unstable and noisy electroencephalogram signals is unreliable and generates a significant mental burden for the user. A feasible solution is to integrate a brain-computer interface (BCI) with automated navigation techniques. This paper presents a brain-controlled intelligent wheelchair with the capability of automatic navigation. Using an autonomous navigation system, candidate destinations and waypoints are automatically generated based on the existing environment. The user selects a destination using a motor imagery (MI)-based or P300-based BCI. According to the determined destination, the navigation system plans a short and safe path and navigates the wheelchair to the destination. During the movement of the wheelchair, the user can issue a stop command with the BCI. Using our system, the mental burden of the user can be substantially alleviated. Furthermore, our system can adapt to changes in the environment. Two experiments based on MI and P300 were conducted to demonstrate the effectiveness of our system.
Collapse
|
100
|
Pasqualotto E, Matuz T, Federici S, Ruf CA, Bartl M, Olivetti Belardinelli M, Birbaumer N, Halder S. Usability and Workload of Access Technology for People With Severe Motor Impairment. Neurorehabil Neural Repair 2015; 29:950-7. [PMID: 25753951 DOI: 10.1177/1545968315575611] [Citation(s) in RCA: 46] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Background. Eye trackers are widely used among people with amyotrophic lateral sclerosis, and their benefits to quality of life have been previously shown. On the contrary, Brain-computer interfaces (BCIs) are still quite a novel technology, which also serves as an access technology for people with severe motor impairment. Objective. To compare a visual P300-based BCI and an eye tracker in terms of information transfer rate (ITR), usability, and cognitive workload in users with motor impairments. Methods. Each participant performed 3 spelling tasks, over 4 total sessions, using an Internet browser, which was controlled by a spelling interface that was suitable for use with either the BCI or the eye tracker. At the end of each session, participants evaluated usability and cognitive workload of the system. Results. ITR and System Usability Scale (SUS) score were higher for the eye tracker (Wilcoxon signed-rank test: ITR T = 9, P = .016; SUS T = 12.50, P = .035). Cognitive workload was higher for the BCI ( T = 4; P = .003). Conclusions. Although BCIs could be potentially useful for people with severe physical disabilities, we showed that the usability of BCIs based on the visual P300 remains inferior to eye tracking. We suggest that future research on visual BCIs should use eye tracking–based control as a comparison to evaluate performance or focus on nonvisual paradigms for persons who have lost gaze control.
Collapse
Affiliation(s)
| | | | - Stefano Federici
- University of Perugia, Perugia, Italy
- Sapienza Università di Roma, Rome, Italy
| | | | | | | | - Niels Birbaumer
- Eberhard Karls Universität, Tübingen, Germany
- Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), Venezia Lido, Italy
| | - Sebastian Halder
- Universität Würzburg, Würzburg, Germany
- National Rehabilitation Center for Persons with Disabilities, Tokorozawa, Japan
| |
Collapse
|