1
|
Kim M, Choi MS, Jang GR, Bae JH, Park HS. EEG-controlled tele-grasping for undefined objects. Front Neurorobot 2023; 17:1293878. [PMID: 38186671 PMCID: PMC10770246 DOI: 10.3389/fnbot.2023.1293878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 11/29/2023] [Indexed: 01/09/2024] Open
Abstract
This paper presents a teleoperation system of robot grasping for undefined objects based on a real-time EEG (Electroencephalography) measurement and shared autonomy. When grasping an undefined object in an unstructured environment, real-time human decision is necessary since fully autonomous grasping may not handle uncertain situations. The proposed system allows involvement of a wide range of human decisions throughout the entire grasping procedure, including 3D movement of the gripper, selecting proper grasping posture, and adjusting the amount of grip force. These multiple decision-making procedures of the human operator have been implemented with six flickering blocks for steady-state visually evoked potentials (SSVEP) by dividing the grasping task into predefined substeps. Each substep consists of approaching the object, selecting posture and grip force, grasping, transporting to the desired position, and releasing. The graphical user interface (GUI) displays the current substep and simple symbols beside each flickering block for quick understanding. The tele-grasping of various objects by using real-time human decisions of selecting among four possible postures and three levels of grip force has been demonstrated. This system can be adapted to other sequential EEG-controlled teleoperation tasks that require complex human decisions.
Collapse
Affiliation(s)
- Minki Kim
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Myoung-Su Choi
- Applied Robot R&D Department, Korea Institute of Industrial Technology, Ansan, Republic of Korea
| | - Ga-Ram Jang
- Applied Robot R&D Department, Korea Institute of Industrial Technology, Ansan, Republic of Korea
| | - Ji-Hun Bae
- Applied Robot R&D Department, Korea Institute of Industrial Technology, Ansan, Republic of Korea
| | - Hyung-Soon Park
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| |
Collapse
|
2
|
Mai X, Ai J, Wei Y, Zhu X, Meng J. Phase-Locked Time-Shift Data Augmentation Method for SSVEP Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4096-4105. [PMID: 37815966 DOI: 10.1109/tnsre.2023.3323351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/12/2023]
Abstract
Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) have achieved an information transfer rate (ITR) of over 300 bits/min, but abundant training data is required. The performance of SSVEP algorithms deteriorates greatly under limited data, and the existing time-shift data augmentation method fails to improve it because the phase-locked requirement between training samples is violated. To address this issue, this study proposes a novel augmentation method, namely phase-locked time-shift (PLTS), for SSVEP-BCI. The similarity between epochs at different time moments was evaluated, and a unique time-shift step was calculated for each class to augment additional data epochs in each trial. The results showed that the PLTS significantly improved the classification performance of SSVEP algorithms on the BETA SSVEP datasets. Moreover, under the condition of one calibration block, by slightly prolonging the calibration duration (from 48 s to 51.5 s), the ITR increased from 40.88±4.54 bits/min to 122.61±7.05 bits/min with the PLTS. This study provides a new perspective on augmenting data epochs for training-based SSVEP-BCI, promotes the classification accuracy and ITR under limited training data, and thus facilitates the real-life applications of SSVEP-based brain spellers.
Collapse
|
3
|
Mai X, Sheng X, Shu X, Ding Y, Zhu X, Meng J. A Calibration-Free Hybrid Approach Combining SSVEP and EOG for Continuous Control. IEEE Trans Neural Syst Rehabil Eng 2023; 31:3480-3491. [PMID: 37610901 DOI: 10.1109/tnsre.2023.3307814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/25/2023]
Abstract
While SSVEP-BCI has been widely developed to control external devices, most of them rely on the discrete control strategy. The continuous SSVEP-BCI enables users to continuously deliver commands and receive real-time feedback from the devices, but it suffers from the transition state problem, a period the erroneous recognition, when users shift their gazes between targets. To resolve this issue, we proposed a novel calibration-free Bayesian approach by hybridizing SSVEP and electrooculography (EOG). First, canonical correlation analysis (CCA) was applied to detect the evoked SSVEPs, and saccade during the gaze shift was detected by EOG data using an adaptive threshold method. Then, the new target after the gaze shift was recognized based on a Bayesian optimization approach, which combined the detection of SSVEP and saccade together and calculated the optimized probability distribution of the targets. Eighteen healthy subjects participated in the offline and online experiments. The offline experiments showed that the proposed hybrid BCI had significantly higher overall continuous accuracy and shorter gaze-shifting time compared to FBCCA, CCA, MEC, and PSDA. In online experiments, the proposed hybrid BCI significantly outperformed CCA-based SSVEP-BCI in terms of continuous accuracy (77.61 ± 1.36%vs. 68.86 ± 1.08% and gaze-shifting time (0.93 ± 0.06s vs. 1.94 ± 0.08s). Additionally, participants also perceived a significant improvement over the CCA-based SSVEP-BCI when the newly proposed decoding approach was used. These results validated the efficacy of the proposed hybrid Bayesian approach for the BCI continuous control without any calibration. This study provides an effective framework for combining SSVEP and EOG, and promotes the potential applications of plug-and-play BCIs in continuous control.
Collapse
|
4
|
Xu B, Liu D, Xue M, Miao M, Hu C, Song A. Continuous shared control of a mobile robot with brain-computer interface and autonomous navigation for daily assistance. Comput Struct Biotechnol J 2023; 22:3-16. [PMID: 37600142 PMCID: PMC10433001 DOI: 10.1016/j.csbj.2023.07.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2023] [Revised: 07/04/2023] [Accepted: 07/22/2023] [Indexed: 08/22/2023] Open
Abstract
Although the electroencephalography (EEG) based brain-computer interface (BCI) has been successfully developed for rehabilitation and assistance, it is still challenging to achieve continuous control of a brain-actuated mobile robot system. In this study, we propose a continuous shared control strategy combining continuous BCI and autonomous navigation for a mobile robot system. The weight of shared control is designed to dynamically adjust the fusion of continuous BCI control and autonomous navigation. During this process, the system uses the visual-based simultaneous localization and mapping (SLAM) method to construct environmental maps. After obtaining the global optimal path, the system utilizes the brain-based shared control dynamic window approach (BSC-DWA) to evaluate safe and reachable trajectories while considering shared control velocity. Eight subjects participated in two-stage training, and six of these eight subjects participated in online shared control experiments. The training results demonstrated that naïve subjects could achieve continuous control performance with an average percent valid correct rate of approximately 97 % and an average total correct rate of over 80 %. The results of online shared control experiments showed that all of the subjects could complete navigation tasks in an unknown corridor with continuous shared control. Therefore, our experiments verified the feasibility and effectiveness of the proposed system combining continuous BCI, shared control, autonomous navigation, and visual SLAM. The proposed continuous shared control framework shows great promise in BCI-driven tasks, especially navigation tasks for brain-driven assistive mobile robots and wheelchairs in daily applications.
Collapse
Affiliation(s)
- Baoguo Xu
- State Key Laboratory of Bioelectronics, Jiangsu Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Deping Liu
- State Key Laboratory of Bioelectronics, Jiangsu Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Muhui Xue
- State Key Laboratory of Bioelectronics, Jiangsu Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Minmin Miao
- School of Information Engineering, Huzhou University, Huzhou 313000, China
| | - Cong Hu
- Guangxi Key Laboratory of Automatic Detecting Technology and Instruments, Guilin University of Electronic Technology, Guilin 541004, China
| | - Aiguo Song
- State Key Laboratory of Bioelectronics, Jiangsu Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| |
Collapse
|
5
|
Yadav H, Maini S. Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-45. [PMID: 37362726 PMCID: PMC10157593 DOI: 10.1007/s11042-023-15653-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 07/17/2022] [Accepted: 04/22/2023] [Indexed: 06/28/2023]
Abstract
Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.
Collapse
Affiliation(s)
- Hitesh Yadav
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| | - Surita Maini
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| |
Collapse
|
6
|
Li M, Wei R, Zhang Z, Zhang P, Xu G, Liao W. CVT-Based Asynchronous BCI for Brain-Controlled Robot Navigation. CYBORG AND BIONIC SYSTEMS 2023; 4:0024. [PMID: 37223547 PMCID: PMC10202181 DOI: 10.34133/cbsystems.0024] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 03/20/2023] [Indexed: 08/24/2024] Open
Abstract
Brain-computer interface (BCI) is a typical direction of integration of human intelligence and robot intelligence. Shared control is an essential form of combining human and robot agents in a common task, but still faces a lack of freedom for the human agent. This paper proposes a Centroidal Voronoi Tessellation (CVT)-based road segmentation approach for brain-controlled robot navigation by means of asynchronous BCI. An electromyogram-based asynchronous mechanism is introduced into the BCI system for self-paced control. A novel CVT-based road segmentation method is provided to generate optional navigation goals in the road area for arbitrary goal selection. An event-related potential of the BCI is designed for target selection to communicate with the robot. The robot has an autonomous navigation function to reach the human selected goals. A comparison experiment in the single-step control pattern is executed to verify the effectiveness of the CVT-based asynchronous (CVT-A) BCI system. Eight subjects participated in the experiment, and they were instructed to control the robot to navigate toward a destination with obstacle avoidance tasks. The results show that the CVT-A BCI system can shorten the task duration, decrease the command times, and optimize navigation path, compared with the single-step pattern. Moreover, this shared control mechanism of the CVT-A BCI system contributes to the promotion of human and robot agent integration control in unstructured environments.
Collapse
Affiliation(s)
- Mengfan Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment,
School of Health Sciences and Biomedical Engineering, Hebei University of Technology, 300132 Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, 300132 Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, 300132 Tianjin, China
| | - Ran Wei
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment,
School of Health Sciences and Biomedical Engineering, Hebei University of Technology, 300132 Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, 300132 Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, 300132 Tianjin, China
| | - Ziqi Zhang
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment,
School of Health Sciences and Biomedical Engineering, Hebei University of Technology, 300132 Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, 300132 Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, 300132 Tianjin, China
| | - Pengfei Zhang
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment,
School of Health Sciences and Biomedical Engineering, Hebei University of Technology, 300132 Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, 300132 Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, 300132 Tianjin, China
| | - Guizhi Xu
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment,
School of Health Sciences and Biomedical Engineering, Hebei University of Technology, 300132 Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, 300132 Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, 300132 Tianjin, China
| | - Wenzhe Liao
- School of Artificial Intelligence, Hebei University of Technology, 300132 Tianjin, China
| |
Collapse
|
7
|
Zhao SN, Cui Y, He Y, He Z, Diao Z, Peng F, Cheng C. Teleoperation control of a wheeled mobile robot based on Brain-machine Interface. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:3638-3660. [PMID: 36899597 DOI: 10.3934/mbe.2023170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
This paper presents a novel teleoperation system using Electroencephalogram (EEG) to control the motion of a wheeled mobile robot (WMR). Different from the other traditional motion controlling method, the WMR is braked with the EEG classification results. Furthermore, the EEG will be induced by using the online BMI (Brain Machine Interface) system, and adopting the non-intrusion induced mode SSVEP (steady state visually evoked potentials). Then, user's motion intention can be recognized by canonical correlation analysis (CCA) classifier, which will be converted into motion commands of the WMR. Finally, the teleoperation technique is utilized to manage the information of the movement scene and adjust the control instructions based on the real-time information. Bezier curve is used to parameterize the path planning of the robot, and the trajectory can be adjusted in real time by EEG recognition results. A motion controller based on error model is proposed to track the planned trajectory by using velocity feedback control, providing excellent track tracking performance. Finally, the feasibility and performance of the proposed teleoperation brain-controlled WMR system are verified using demonstration experiments.
Collapse
Affiliation(s)
- Su-Na Zhao
- College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China
| | - Yingxue Cui
- College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China
| | - Yan He
- College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China
| | - Zhendong He
- College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China
| | - Zhihua Diao
- College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China
| | - Fang Peng
- Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan 528402, China
| | - Chao Cheng
- Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Changchun 130022, China
- Weihai Institute for Bionics, Jilin University, Weihai 264402, China
| |
Collapse
|
8
|
Driving Mode Selection through SSVEP-Based BCI and Energy Consumption Analysis. SENSORS 2022; 22:s22155631. [PMID: 35957188 PMCID: PMC9371069 DOI: 10.3390/s22155631] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/12/2022] [Revised: 07/20/2022] [Accepted: 07/25/2022] [Indexed: 12/04/2022]
Abstract
Background: The brain–computer interface (BCI) is a highly cross-discipline technology and its successful application in various domains has received increasing attention. However, the BCI-enabled automobile industry is has been comparatively less investigated. In particular, there are currently no studies focusing on brain-controlled driving mode selection. Specifically, different driving modes indicate different driving styles which can be selected according to the road condition or the preference of individual drivers. Methods: In this paper, a steady-state visual-evoked potential (SSVEP)-based driving mode selection system is proposed. Upon this system, drivers can select the intended driving modes by only gazing at the corresponding SSVEP stimuli. A novel EEG processing algorithm named inter-trial distance minimization analysis (ITDMA) is proposed to enhance SSVEP detection. Both offline and real-time experiments were carried out to validate the effectiveness of the proposed system. Conclusion: The results show that a high selection accuracy of up to 92.3% can be realized, although this depends on the specific choice of flickering duration, the number of EEG channels, and the number of training signals. Additionally, energy consumption is investigated in terms of which the proposed brain-controlled system considerably differs from a traditional driving mode selection system, and the main reason is shown to be the existence of a detection error.
Collapse
|
9
|
Application of Neuroengineering Based on EEG Features in the Industrial Design of Comfort. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:4667689. [PMID: 35720909 PMCID: PMC9205692 DOI: 10.1155/2022/4667689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 04/23/2022] [Accepted: 04/26/2022] [Indexed: 11/17/2022]
Abstract
The smart wheelchair is a service robot that can be used as a means of transportation for the elderly and the disabled. The patients were given an intelligent wheelchair designed by electroencephalogram (EEG), which was used for more than 8 hours and tested continuously for 1 month. By ridit analysis, the difference between the two groups was statistically significant (U = 3.72, P < 0.01). The scores of visual analogue scale (VAS) and joint ground visuality (JGV) in the observation group were significantly better than those in the control group. The modules of physiological function (PF), physical pain (PP), overall health (OH), vitality (VT), social function (SF), emotional function (EF), and mental health (MH) in the SF-36 scores of the two groups were significantly improved (P < 0.05), and the improvement of each module in the observation group was significantly better than that in the control group (P < 0.05). The levels of serum IL-6, IL-10, and superoxide dismutase (SOD) in the two groups were significantly improved (P < 0.05), and the improvement of serum IL-6, IL-10, and SOD in the observation group was significantly better than that in the control group (P < 0.05). It is suggested that neural engineering based on EEG characteristics can be well applied in comfort industrial design.
Collapse
|
10
|
Zhao X, Jin J, Xu R, Li S, Sun H, Wang X, Cichocki A. A Regional Smoothing Block Sparse Bayesian Learning Method With Temporal Correlation for Channel Selection in P300 Speller. Front Hum Neurosci 2022; 16:875851. [PMID: 35754766 PMCID: PMC9231363 DOI: 10.3389/fnhum.2022.875851] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Accepted: 05/18/2022] [Indexed: 11/13/2022] Open
Abstract
The P300-based brain-computer interfaces (BCIs) enable participants to communicate by decoding the electroencephalography (EEG) signal. Different regions of the brain correspond to various mental activities. Therefore, removing weak task-relevant and noisy channels through channel selection is necessary when decoding a specific type of activity from EEG. It can improve the recognition accuracy and reduce the training time of the subsequent models. This study proposes a novel block sparse Bayesian-based channel selection method for the P300 speller. In this method, we introduce block sparse Bayesian learning (BSBL) into the channel selection of P300 BCI for the first time and propose a regional smoothing BSBL (RSBSBL) by combining the spatial distribution properties of EEG. The RSBSBL can determine the number of channels adaptively. To ensure practicality, we design an automatic selection iteration strategy model to reduce the time cost caused by the inverse operation of the large-size matrix. We verified the proposed method on two public P300 datasets and on our collected datasets. The experimental results show that the proposed method can remove the inferior channels and work with the classifier to obtain high-classification accuracy. Hence, RSBSBL has tremendous potential for channel selection in P300 tasks.
Collapse
Affiliation(s)
- Xueqing Zhao
- The Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Jing Jin
- The Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, China
- Shenzhen Research Institute of East China University of Technology, Shenzhen, China
| | - Ren Xu
- g.tec medical engineering GmbH, Graz, Austria
| | - Shurui Li
- The Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Hao Sun
- The Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Xingyu Wang
- The Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai, China
| | - Andrzej Cichocki
- Skolkovo Institute of Science and Technology, Moscow, Russia
- Systems Research Institute of Polish Academy of Science, Warsaw, Poland
- Department of Informatics, Nicolaus Copernicus University, Toruń, Poland
| |
Collapse
|
11
|
Wang J, Chen YH, Yang J, Sawan M. Intelligent Classification Technique of Hand Motor Imagery Using EEG Beta Rebound Follow-Up Pattern. BIOSENSORS 2022; 12:bios12060384. [PMID: 35735532 PMCID: PMC9221354 DOI: 10.3390/bios12060384] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 05/18/2022] [Accepted: 05/20/2022] [Indexed: 11/16/2022]
Abstract
To apply EEG-based brain-machine interfaces during rehabilitation, separating various tasks during motor imagery (MI) and assimilating MI into motor execution (ME) are needed. Previous studies were focusing on classifying different MI tasks based on complex algorithms. In this paper, we implement intelligent, straightforward, comprehensible, time-efficient, and channel-reduced methods to classify ME versus MI and left- versus right-hand MI. EEG of 30 healthy participants undertaking motional tasks is recorded to investigate two classification tasks. For the first task, we first propose a “follow-up” pattern based on the beta rebound. This method achieves an average classification accuracy of 59.77% ± 11.95% and can be up to 89.47% for finger-crossing. Aside from time-domain information, we map EEG signals to feature space using extraction methods including statistics, wavelet coefficients, average power, sample entropy, and common spatial patterns. To evaluate their practicability, we adopt a support vector machine as an intelligent classifier model and sparse logistic regression as a feature selection technique and achieve 79.51% accuracy. Similar approaches are taken for the second classification reaching 75.22% accuracy. The classifiers we propose show high accuracy and intelligence. The achieved results make our approach highly suitable to be applied to the rehabilitation of paralyzed limbs.
Collapse
Affiliation(s)
- Jiachen Wang
- Center of Excellence in Biomedical Research on Advanced Integrated-on-Chips Neurotechnologies (CenBRAIN Neurotech), School of Engineering, Westlake University, Hangzhou 310024, China; (J.W.); (J.Y.)
| | - Yun-Hsuan Chen
- Center of Excellence in Biomedical Research on Advanced Integrated-on-Chips Neurotechnologies (CenBRAIN Neurotech), School of Engineering, Westlake University, Hangzhou 310024, China; (J.W.); (J.Y.)
- Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou 310024, China
- Correspondence: (Y.-H.C.); (M.S.)
| | - Jie Yang
- Center of Excellence in Biomedical Research on Advanced Integrated-on-Chips Neurotechnologies (CenBRAIN Neurotech), School of Engineering, Westlake University, Hangzhou 310024, China; (J.W.); (J.Y.)
- Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou 310024, China
| | - Mohamad Sawan
- Center of Excellence in Biomedical Research on Advanced Integrated-on-Chips Neurotechnologies (CenBRAIN Neurotech), School of Engineering, Westlake University, Hangzhou 310024, China; (J.W.); (J.Y.)
- Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou 310024, China
- Correspondence: (Y.-H.C.); (M.S.)
| |
Collapse
|
12
|
Zhou Y, Yang B, Guan C. Task-Related Component Analysis Combining Paired Character Decoding for Miniature Asymmetric Visual Evoked Potentials. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1331-1339. [PMID: 35576428 DOI: 10.1109/tnsre.2022.3175307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Brain-computer interface (BCI) technology based on event-related potentials (ERP) of electroencephalography (EEG) is widely used in daily life and medical treatment. However, the research of identifying the miniature and more informative asymmetric visual evoked potentials (aVEPs), which belongs to ERP, needs further exploration. Herein, a task-related component analysis combining paired character decoding (TRCA-PCD) method, which can enhance reproducibility of aVEPs in multiple trials and strengthen the features of different samples, was designed to realize fast decoding of aVEPs. The BCI performance and the influence of repetition times between the TRCA-PCD method, the discriminative canonical pattern matching (DCPM) method and traditional task-related component analysis (TRCA) method were compared using a 32-class aVEPs dataset recorded from 32 subjects. The highest average recognition accuracy and information transfer rate (ITR) of TRCA-PCD after parameter selection were 70.37 ± 2.49% (DCPM: 64.91 ± 2.81%, TRCA: 44.01 ± 3.25%) with the peak value of 97.92% and 28.90 ± 3.83 bits/min (DCPM: 21.29 ± 3.35 bits/min, TRCA: 11.54 ± 2.81 bits/min) with the peak value of 94.55 bits/min respectively. Statistical analysis indicated that the highest average recognition rate could be obtained when the repetition time was six, and the highest ITR could be obtained when the repetition time was one. Overall, the results verified the effectiveness and superiority of TRCA-PCD in recognition of aVEPs and provided a reference for parameter selection. Therefore, the TRCA-PCD method can promote the further application of aVEPs in the BCI speller field.
Collapse
|
13
|
Mendez Guerra I, Barsakcioglu DY, Vujaklija I, Wetmore DZ, Farina D. Far-field electric potentials provide access to the output from the spinal cord from wrist-mounted sensors. J Neural Eng 2022; 19. [PMID: 35303732 DOI: 10.1088/1741-2552/ac5f1a] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Accepted: 03/18/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Neural interfaces need to become more unobtrusive and socially acceptable to appeal to general consumers outside rehabilitation settings. APPROACH We developed a non-invasive neural interface that provides access to spinal motor neuron activities from the wrist, which is the preferred location for a wearable. The interface decodes far-field potentials present at the tendon endings of the forearm muscles using blind source separation. First, we evaluated the reliability of the interface to detect motor neuron firings based on far-field potentials, and thereafter we used the decoded motor neuron activity for the prediction of finger contractions in offline and real-time conditions. MAIN RESULTS The results showed that motor neuron activity decoded from the far-field potentials at the wrist accurately predicted individual and combined finger commands and therefore allowed for highly accurate real-time task classification. SIGNIFICANCE These findings demonstrate the feasibility of a non-invasive, neural interface at the wrist for precise real-time control based on the output of the spinal cord.
Collapse
Affiliation(s)
- Irene Mendez Guerra
- Department of Bioengineering, Imperial College London, 80 Wood Lane, London, W12 7TA, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Deren Yusuf Barsakcioglu
- Department of Bioengineering, Imperial College London, 80 Wood Lane, London, W12 7TA, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Ivan Vujaklija
- Department of Electrical Engineering and Automation, Aalto-yliopisto, Otakaari 3 (F306), Espoo, 00076, FINLAND
| | - Daniel Z Wetmore
- Meta Inc, 770 Broadway, New York City, New York, 10003, UNITED STATES
| | - Dario Farina
- Department of Bioengineering, Imperial College London, 80 Wood Lane, London, W12 7TA, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| |
Collapse
|
14
|
Peng F, Li M, Zhao SN, Xu Q, Xu J, Wu H. Control of a Robotic Arm With an Optimized Common Template-Based CCA Method for SSVEP-Based BCI. Front Neurorobot 2022; 16:855825. [PMID: 35370596 PMCID: PMC8965569 DOI: 10.3389/fnbot.2022.855825] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2022] [Accepted: 02/11/2022] [Indexed: 11/16/2022] Open
Abstract
Recently, the robotic arm control system based on a brain-computer interface (BCI) has been employed to help the disabilities to improve their interaction abilities without body movement. However, it's the main challenge to implement the desired task by a robotic arm in a three-dimensional (3D) space because of the instability of electroencephalogram (EEG) signals and the interference by the spontaneous EEG activities. Moreover, the free motion control of a manipulator in 3D space is a complicated operation that requires more output commands and higher accuracy for brain activity recognition. Based on the above, a steady-state visual evoked potential (SSVEP)-based synchronous BCI system with six stimulus targets was designed to realize the motion control function of the seven degrees of freedom (7-DOF) robotic arm. Meanwhile, a novel template-based method, which builds the optimized common templates (OCTs) from various subjects and learns spatial filters from the common templates and the multichannel EEG signal, was applied to enhance the SSVEP recognition accuracy, called OCT-based canonical correlation analysis (OCT-CCA). The comparison results of offline experimental based on a public benchmark dataset indicated that the proposed OCT-CCA method achieved significant improvement of detection accuracy in contrast to CCA and individual template-based CCA (IT-CCA), especially using a short data length. In the end, online experiments with five healthy subjects were implemented for achieving the manipulator real-time control system. The results showed that all five subjects can accomplish the tasks of controlling the manipulator to reach the designated position in the 3D space independently.
Collapse
Affiliation(s)
- Fang Peng
- Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan, China
| | - Ming Li
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Su-na Zhao
- College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou, China
- *Correspondence: Su-na Zhao
| | - Qinyi Xu
- School of Automation, Guangdong University of Technology, Guangzhou, China
| | - Jiajun Xu
- Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan, China
| | - Haozhen Wu
- Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan, China
| |
Collapse
|
15
|
Li P, Li C, Bore JC, Si Y, Li F, Cao Z, Zhang Y, Wang G, Zhang Z, Yao D, Xu P. L1-norm based time-varying brain neural network and its application to dynamic analysis for motor imagery. J Neural Eng 2022; 19. [PMID: 35234668 DOI: 10.1088/1741-2552/ac59a4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 03/01/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE EEG-based motor imagery (MI) brain-computer interface offers a promising way to improve the efficiency of motor rehabilitation and motor skill learning. In recent years, the power of dynamic network analysis for MI classification has been proved. In fact, its usability mainly depends on the accurate estimation of brain connection. However, traditional dynamic network estimation strategies such as adaptive directed transfer function (ADTF) are designed in the L2-norm. Usually, they estimate a series of pseudo connections caused by outliers, which results in biased features and further limits its online application. Thus, how to accurately infer dynamic causal relationship under outlier influence is urgent. APPROACH In this work, we proposed a novel ADTF, which solves the dynamic system in the L1-norm space (L1-ADTF), so as to restrict the outlier influence. To enhance its convergence, we designed an iteration strategy with the alternating direction method of multipliers (ADMM), which could be used for the solution of the dynamic state-space model restricted in the L1-norm space. Furthermore, we compared L1-ADTF to traditional ADTF and its dual extension across both simulation and real EEG experiments. MAIN RESULTS A quantitative comparison between L1-ADTF and other ADTFs in simulation studies demonstrates that fewer bias errors and more desirable dynamic state transformation patterns can be captured by the L1-ADTF. Application to real MI EEG datasets seriously noised by ocular artifacts also reveals the efficiency of the proposed L1-ADTF approach to extract the time-varying brain neural network patterns, even when more complex noises are involved. SIGNIFICANCE The L1-ADTF may not only be capable of tracking time-varying brain network state drifts robustly but may also be useful in solving a wide range of dynamic systems such as trajectory tracking problems and dynamic neural networks.
Collapse
Affiliation(s)
- Peiyang Li
- School of Bioinformatics, Chongqing University of Posts and Telecommunications, NO.2,Chongwen Road,Nan'an District, Chongqing, China, Chongqing, 400065, CHINA
| | - Cunbo Li
- University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, Sichuan, China, Chengdu, 611731, CHINA
| | - Joyce Chelangat Bore
- University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, Sichuan, China, Chengdu, 611731, CHINA
| | - Yajing Si
- Department of Psychology, Xinxiang Medical University, No. 601, Jinsui Avenue, Hongqi District, Xinxiang, Henan, 453003, CHINA
| | - Fali Li
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, Sichuan, China, Chengdu, 610054, CHINA
| | - Zehong Cao
- University of South Australia, Adelaide, SA 5095, Australia, Adelaide, South Australia, 5001, AUSTRALIA
| | - Yangsong Zhang
- Southwest University of Science and Technology, 59 Qinglong Road, Mianyang,Sichuan, P.R.China, Mianyang, 621010, CHINA
| | - Gang Wang
- University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, Sichuan, China, Chengdu, 610054, CHINA
| | - Zhijun Zhang
- South China University of Technology, 777 Xingye Avenue East, Panyu District, Guangzhou, Guangzhou, 510640, CHINA
| | - Dezhong Yao
- University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, Sichuan, China, Chengdu, 611731, CHINA
| | - Peng Xu
- University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, Sichuan, China, Chengdu, 611731, CHINA
| |
Collapse
|
16
|
Sivakanthan S, Candiotti JL, Sundaram AS, Duvall JA, Sergeant JJG, Cooper R, Satpute S, Turner RL, Cooper RA. Mini-review: Robotic wheelchair taxonomy and readiness. Neurosci Lett 2022; 772:136482. [PMID: 35104618 PMCID: PMC8887066 DOI: 10.1016/j.neulet.2022.136482] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Revised: 01/21/2022] [Accepted: 01/24/2022] [Indexed: 01/05/2023]
Abstract
Robotic wheelchair research and development is a growing sector. This article introduces a robotic wheelchair taxonomy, and a readiness model supported by a mini-review. The taxonomy is constructed by power wheelchair and, mobile robot standards, the ICF and, PHAATE models. The mini-review of 2797 articles spanning 7 databases produced 205 articles and 4 review articles that matched inclusion/exclusion criteria. The review and analysis illuminate how innovations in robotic wheelchair research progressed and have been slow to translate into the marketplace.
Collapse
Affiliation(s)
- Sivashankar Sivakanthan
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA
| | - Jorge L Candiotti
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA
| | - Andrea S Sundaram
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA
| | - Jonathan A Duvall
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA
| | | | - Rosemarie Cooper
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA
| | - Shantanu Satpute
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA; Department of Bioengineering, Swanson School of Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Rose L Turner
- Health Science Library System, University of Pittsburgh, Pittsburgh, PA, USA
| | - Rory A Cooper
- Human Engineering Research Laboratories, VA Pittsburgh Healthcare System, Pittsburgh, PA, USA; Human Engineering Research Laboratories, School of Health and Rehabilitation Sciences, Pittsburgh, PA, USA.
| |
Collapse
|
17
|
A multiscale siamese convolutional neural network with cross-channel fusion for motor imagery decoding. J Neurosci Methods 2022; 367:109426. [PMID: 34902364 DOI: 10.1016/j.jneumeth.2021.109426] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2021] [Revised: 10/28/2021] [Accepted: 11/22/2021] [Indexed: 11/22/2022]
Abstract
BACKGROUND Recently, convolutional neural networks (CNN) are widely applied in motor imagery electroencephalography (MI-EEG) signal classification tasks. However, a simple CNN framework is challenging to satisfy the complex MI-EEG signal decoding. NEW METHOD In this study, we propose a multiscale Siamese convolutional neural network with cross-channel fusion (MSCCF-Net) for MI-EEG classification tasks. The proposed network consists of three parts: Siamese cross-channel fusion streams, similarity module and classification module. Each Siamese cross-channel fusion stream contains multiple branches, and each branch is supplemented by cross-channel fusion modules to improve multiscale temporal feature representation capability. The similarity module is adopted to measure the feature similarity between multiple branches. At the same time, the classification module provides a strong constraint to classify the features from all Siamese cross-channel fusion streams. The combination of the similarity module and classification module constitutes a new joint training strategy to further optimize the network performance. RESULTS The experiment is conducted on the public BCI Competition IV 2a and 2b datasets, and the results show that the proposed network achieves an average accuracy of 87.36% and 87.33%, respectively. COMPARISON WITH EXISTING METHODS AND CONCLUSIONS The proposed network adopts cross-channel fusion to learn multiscale temporal characteristics and joint training strategy to optimize the training process. Therefore, the performance outperforms other state-of-the-art MI-EEG signal classification methods.
Collapse
|
18
|
Steady-State Visual Evoked Potential Classification Using Complex Valued Convolutional Neural Networks. SENSORS 2021; 21:s21165309. [PMID: 34450751 PMCID: PMC8398418 DOI: 10.3390/s21165309] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Revised: 07/24/2021] [Accepted: 08/03/2021] [Indexed: 11/30/2022]
Abstract
The steady-state visual evoked potential (SSVEP), which is a kind of event-related potential in electroencephalograms (EEGs), has been applied to brain–computer interfaces (BCIs). SSVEP-based BCIs currently perform the best in terms of information transfer rate (ITR) among various BCI implementation methods. Canonical component analysis (CCA) or spectrum estimation, such as the Fourier transform, and their extensions have been used to extract features of SSVEPs. However, these signal extraction methods have a limitation in the available stimulation frequency; thus, the number of commands is limited. In this paper, we propose a complex valued convolutional neural network (CVCNN) to overcome the limitation of SSVEP-based BCIs. The experimental results demonstrate that the proposed method overcomes the limitation of the stimulation frequency, and it outperforms conventional SSVEP feature extraction methods.
Collapse
|
19
|
Tensor-based dynamic brain functional network for motor imagery classification. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102940] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
20
|
Jamil N, Belkacem AN, Ouhbi S, Lakas A. Noninvasive Electroencephalography Equipment for Assistive, Adaptive, and Rehabilitative Brain-Computer Interfaces: A Systematic Literature Review. SENSORS (BASEL, SWITZERLAND) 2021; 21:4754. [PMID: 34300492 PMCID: PMC8309653 DOI: 10.3390/s21144754] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/06/2021] [Revised: 06/28/2021] [Accepted: 07/09/2021] [Indexed: 11/30/2022]
Abstract
Humans interact with computers through various devices. Such interactions may not require any physical movement, thus aiding people with severe motor disabilities in communicating with external devices. The brain-computer interface (BCI) has turned into a field involving new elements for assistive and rehabilitative technologies. This systematic literature review (SLR) aims to help BCI investigator and investors to decide which devices to select or which studies to support based on the current market examination. This examination of noninvasive EEG devices is based on published BCI studies in different research areas. In this SLR, the research area of noninvasive BCIs using electroencephalography (EEG) was analyzed by examining the types of equipment used for assistive, adaptive, and rehabilitative BCIs. For this SLR, candidate studies were selected from the IEEE digital library, PubMed, Scopus, and ScienceDirect. The inclusion criteria (IC) were limited to studies focusing on applications and devices of the BCI technology. The data used herein were selected using IC and exclusion criteria to ensure quality assessment. The selected articles were divided into four main research areas: education, engineering, entertainment, and medicine. Overall, 238 papers were selected based on IC. Moreover, 28 companies were identified that developed wired and wireless equipment as means of BCI assistive technology. The findings of this review indicate that the implications of using BCIs for assistive, adaptive, and rehabilitative technologies are encouraging for people with severe motor disabilities and healthy people. With an increasing number of healthy people using BCIs, other research areas, such as the motivation of players when participating in games or the security of soldiers when observing certain areas, can be studied and collaborated using the BCI technology. However, such BCI systems must be simple (wearable), convenient (sensor fabrics and self-adjusting abilities), and inexpensive.
Collapse
Affiliation(s)
- Nuraini Jamil
- Department of Computer Science and Software Engineering, College of Information Technology, United Arab Emirates University, Al Ain P.O. Box 15551, United Arab Emirates; (N.J.); (S.O.)
| | - Abdelkader Nasreddine Belkacem
- Department of Computer and Network Engineering, College of Information Technology, United Arab Emirates University, Al Ain P.O. Box 15551, United Arab Emirates;
| | - Sofia Ouhbi
- Department of Computer Science and Software Engineering, College of Information Technology, United Arab Emirates University, Al Ain P.O. Box 15551, United Arab Emirates; (N.J.); (S.O.)
| | - Abderrahmane Lakas
- Department of Computer and Network Engineering, College of Information Technology, United Arab Emirates University, Al Ain P.O. Box 15551, United Arab Emirates;
| |
Collapse
|
21
|
Garcia A. DE, Sierra M. SD, Gomez-Vargas D, Jiménez MF, Múnera M, Cifuentes CA. Semi-Remote Gait Assistance Interface: A Joystick with Visual Feedback Capabilities for Therapists. SENSORS 2021; 21:s21103521. [PMID: 34069340 PMCID: PMC8158774 DOI: 10.3390/s21103521] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 05/03/2021] [Accepted: 05/06/2021] [Indexed: 11/21/2022]
Abstract
The constant growth of pathologies affecting human mobility has led to developing of different assistive devices to provide physical and cognitive assistance. Smart walkers are a particular type of these devices since they integrate navigation systems, path-following algorithms, and user interaction modules to ensure natural and intuitive interaction. Although these functionalities are often implemented in rehabilitation scenarios, there is a need to actively involve the healthcare professionals in the interaction loop while guaranteeing safety for them and patients. This work presents the validation of two visual feedback strategies for the teleoperation of a simulated robotic walker during an assisted navigation task. For this purpose, a group of 14 clinicians from the rehabilitation area formed the validation group. A simple path-following task was proposed, and the feedback strategies were assessed through the kinematic estimation error (KTE) and a usability survey. A KTE of 0.28 m was obtained for the feedback strategy on the joystick. Additionally, significant differences were found through a Mann–Whitney–Wilcoxon test for the perception of behavior and confidence towards the joystick according to the modes of interaction (p-values of 0.04 and 0.01, respectively). The use of visual feedback with this tool contributes to research areas such as remote management of therapies and monitoring rehabilitation of people’s mobility.
Collapse
Affiliation(s)
- Daniel E. Garcia A.
- Department of Biomedical Engineering, Colombian School of Engineering Julio Garavito, Bogota 111166, Colombia; (D.E.G.A.); (S.D.S.M.); (D.G.-V.); (M.M.)
| | - Sergio D. Sierra M.
- Department of Biomedical Engineering, Colombian School of Engineering Julio Garavito, Bogota 111166, Colombia; (D.E.G.A.); (S.D.S.M.); (D.G.-V.); (M.M.)
| | - Daniel Gomez-Vargas
- Department of Biomedical Engineering, Colombian School of Engineering Julio Garavito, Bogota 111166, Colombia; (D.E.G.A.); (S.D.S.M.); (D.G.-V.); (M.M.)
| | - Mario F. Jiménez
- School of Engineering, Science and Technology, Universidad del Rosario, Bogotá 111711, Colombia
- Correspondence: (M.F.J.); (C.A.C.); Tel.: +57-(1)-297-0200 (M.F.J.); +57-(031)-668-3600 (C.A.C.)
| | - Marcela Múnera
- Department of Biomedical Engineering, Colombian School of Engineering Julio Garavito, Bogota 111166, Colombia; (D.E.G.A.); (S.D.S.M.); (D.G.-V.); (M.M.)
| | - Carlos A. Cifuentes
- Department of Biomedical Engineering, Colombian School of Engineering Julio Garavito, Bogota 111166, Colombia; (D.E.G.A.); (S.D.S.M.); (D.G.-V.); (M.M.)
- Correspondence: (M.F.J.); (C.A.C.); Tel.: +57-(1)-297-0200 (M.F.J.); +57-(031)-668-3600 (C.A.C.)
| |
Collapse
|
22
|
Cao L, Li G, Xu Y, Zhang H, Shu X, Zhang D. A brain-actuated robotic arm system using non-invasive hybrid brain-computer interface and shared control strategy. J Neural Eng 2021; 18. [PMID: 33862607 DOI: 10.1088/1741-2552/abf8cb] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Accepted: 04/16/2021] [Indexed: 01/20/2023]
Abstract
Objective.The electroencephalography (EEG)-based brain-computer interfaces (BCIs) have been used in the control of robotic arms. The performance of non-invasive BCIs may not be satisfactory due to the poor quality of EEG signals, so the shared control strategies were tried as an alternative solution. However, most of the existing shared control methods set the arbitration rules manually, which highly depended on the specific tasks and developer's experience. In this study, we proposed a novel shared control model that automatically optimized the control commands in a dynamical way based on the context in real-time control. Besides, we employed the hybrid BCI to better allocate commands with multiple functions. The system allowed non-invasive BCI users to manipulate a robotic arm moving in a three-dimensional (3D) space and complete a pick-place task of multiple objects.Approach.Taking the scene information obtained by computer vision as a knowledge base, a machine agent was designed to infer the user's intention and generate automatic commands. Based on the inference confidence and user's characteristic, the proposed shared control model fused the machine autonomy and human intention dynamically for robotic arm motion optimization during the online control. In addition, we introduced a hybrid BCI scheme that applied steady-state visual evoked potentials and motor imagery to the divided primary and secondary BCI interfaces to better allocate the BCI resources (e.g. decoding computing power, screen occupation) and realize the multi-dimensional control of the robotic arm.Main results.Eleven subjects participated in the online experiments of picking and placing five objects that scattered at different positions in a 3D workspace. The results showed that most of the subjects could control the robotic arm to complete accurate and robust picking task with an average success rate of approximately 85% under the shared control strategy, while the average success rate of placing task controlled by pure BCI was 50% approximately.Significance.In this paper, we proposed a novel shared controller for motion automatic optimization, together with a hybrid BCI control scheme that allocated paradigms according to the importance of commands to realize multi-dimensional and effective control of a robotic arm. Our study indicated that the shared control strategy with hybrid BCI could greatly improve the performance of the brain-actuated robotic arm system.
Collapse
Affiliation(s)
- Linfeng Cao
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Guangye Li
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Yang Xu
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Heng Zhang
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xiaokang Shu
- State Key Laboratory of Mechanical Systems and Vibrations, Institute of Robotics, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Dingguo Zhang
- Department of Electronic and Electrical Engineering, University of Bath, Bath, United Kingdom
| |
Collapse
|
23
|
Saha S, Mamun KA, Ahmed K, Mostafa R, Naik GR, Darvishi S, Khandoker AH, Baumert M. Progress in Brain Computer Interface: Challenges and Opportunities. Front Syst Neurosci 2021; 15:578875. [PMID: 33716680 PMCID: PMC7947348 DOI: 10.3389/fnsys.2021.578875] [Citation(s) in RCA: 82] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Accepted: 01/06/2021] [Indexed: 12/13/2022] Open
Abstract
Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.
Collapse
Affiliation(s)
- Simanto Saha
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Khondaker A. Mamun
- Advanced Intelligent Multidisciplinary Systems (AIMS) Lab, Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh
| | - Khawza Ahmed
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Raqibul Mostafa
- Department of Electrical and Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Ganesh R. Naik
- Adelaide Institute for Sleep Health, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia
| | - Sam Darvishi
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
| | - Ahsan H. Khandoker
- Healthcare Engineering Innovation Center, Department of Biomedical Engineering, Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates
| | - Mathias Baumert
- School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide, SA, Australia
| |
Collapse
|
24
|
A Novel Approach to Shadow Boundary Detection Based on an Adaptive Direction-Tracking Filter for Brain-Machine Interface Applications. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10196761] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
In this paper, a Brain-Machine Interface (BMI) system is proposed to automatically control the navigation of wheelchairs by detecting the shadows on their route. In this context, a new algorithm to detect shadows in a single image is proposed. Specifically, a novel adaptive direction tracking filter (ADT) is developed to extract feature information along the direction of shadow boundaries. The proposed algorithm avoids extraction of features around all directions of pixels, which significantly improves the efficiency and accuracy of shadow features extraction. Higher-order statistics (HOS) features such as skewness and kurtosis in addition to other optical features are used as input to different Machine Learning (ML) based classifiers, specifically, a Multilayer Perceptron (MLP), Autoencoder (AE), 1D-Convolutional Neural Network (1D-CNN) and Support Vector Machine (SVM), to perform the shadow boundaries detection task. Comparative results demonstrate that the proposed MLP-based system outperforms all the other state-of-the-art approaches, reporting accuracy rates up to 84.63%.
Collapse
|