1
|
Zhang J, Zhang Y, Zhang X, Xu B, Zhao H, Sun T, Wang J, Lu S, Shen X. A high-performance general computer cursor control scheme based on a hybrid BCI combining motor imagery and eye-tracking. iScience 2024; 27:110164. [PMID: 38974471 PMCID: PMC11225862 DOI: 10.1016/j.isci.2024.110164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Revised: 03/21/2024] [Accepted: 05/29/2024] [Indexed: 07/09/2024] Open
Abstract
This study introduces a novel virtual cursor control system designed to empower individuals with neuromuscular disabilities in the digital world. By combining eye-tracking with motor imagery (MI) in a hybrid brain-computer interface (BCI), the system enhances cursor control accuracy and simplicity. Real-time classification accuracy reaches 87.92% (peak of 93.33%), with cursor stability in the gazing state at 96.1%. Integrated into common operating systems, it enables tasks like text entry, online chatting, email, web surfing, and picture dragging, with an average text input rate of 53.2 characters per minute (CPM). This technology facilitates fundamental computing tasks for patients, fostering their integration into the online community and paving the way for future developments in BCI systems.
Collapse
Affiliation(s)
- Jiakai Zhang
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Yuqi Zhang
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Xinlong Zhang
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Boyang Xu
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Huanqing Zhao
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Tinghui Sun
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Ju Wang
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Shaojie Lu
- School of Information Science and Technology, Nantong University, Nantong 226019, China
| | - Xiaoyan Shen
- School of Information Science and Technology, Nantong University, Nantong 226019, China
- Nantong Research Institute for Advanced Communication Technologies, Nantong University, Nantong 226019, China
| |
Collapse
|
2
|
Wang F, Wen Y, Bi J, Li H, Sun J. A portable SSVEP-BCI system for rehabilitation exoskeleton in augmented reality environment. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104664] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
|
3
|
Lii NY, Pereira A, Dietl J, Stillfried G, Schmidt A, Beik-Mohammadi H, Baker T, Maier A, Pleintinger B, Chen Z, Elawad A, Mentzer L, Pineault A, Reisich P, Albu-Schäffer A. Exodex Adam—A Reconfigurable Dexterous Haptic User Interface for the Whole Hand. Front Robot AI 2022; 8:716598. [PMID: 35309724 PMCID: PMC8927287 DOI: 10.3389/frobt.2021.716598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Accepted: 12/24/2021] [Indexed: 11/13/2022] Open
Abstract
Applications for dexterous robot teleoperation and immersive virtual reality are growing. Haptic user input devices need to allow the user to intuitively command and seamlessly “feel” the environment they work in, whether virtual or a remote site through an avatar. We introduce the DLR Exodex Adam, a reconfigurable, dexterous, whole-hand haptic input device. The device comprises multiple modular, three degrees of freedom (3-DOF) robotic fingers, whose placement on the device can be adjusted to optimize manipulability for different user hand sizes. Additionally, the device is mounted on a 7-DOF robot arm to increase the user’s workspace. Exodex Adam uses a front-facing interface, with robotic fingers coupled to two of the user’s fingertips, the thumb, and two points on the palm. Including the palm, as opposed to only the fingertips as is common in existing devices, enables accurate tracking of the whole hand without additional sensors such as a data glove or motion capture. By providing “whole-hand” interaction with omnidirectional force-feedback at the attachment points, we enable the user to experience the environment with the complete hand instead of only the fingertips, thus realizing deeper immersion. Interaction using Exodex Adam can range from palpation of objects and surfaces to manipulation using both power and precision grasps, all while receiving haptic feedback. This article details the concept and design of the Exodex Adam, as well as use cases where it is deployed with different command modalities. These include mixed-media interaction in a virtual environment, gesture-based telemanipulation, and robotic hand–arm teleoperation using adaptive model-mediated teleoperation. Finally, we share the insights gained during our development process and use case deployments.
Collapse
Affiliation(s)
- Neal Y. Lii
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
- *Correspondence: Neal Y. Lii,
| | - Aaron Pereira
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Julian Dietl
- Faculty of Mechanical Engineering, Munich University of Applied Science, Munich, Germany
| | - Georg Stillfried
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Annika Schmidt
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
- Faculty of Informatics, Technical University of Munich, Munich, Germany
| | - Hadi Beik-Mohammadi
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Thomas Baker
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Annika Maier
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Benedikt Pleintinger
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Zhaopeng Chen
- Department of Informatics, Faculty of Mathematics, Informatics and Natural Science, University of Hamburg, Hamburg, Germany
| | - Amal Elawad
- Department of Electrical Engineering, Chalmers University of Technology, Göteborg, Sweden
| | - Lauren Mentzer
- Department of Computer Science and Electrical Engineering, Stanford University, Stanford, CA, United States
| | - Austin Pineault
- Department of Computer Science and Electrical Engineering, Stanford University, Stanford, CA, United States
| | - Philipp Reisich
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| | - Alin Albu-Schäffer
- Institute of Robotics and Mechatronics, German Aerospace Center (DLR), Wessling, Germany
| |
Collapse
|
4
|
Chu C, Luo J, Tian X, Han X, Guo S. A P300 Brain-Computer Interface Paradigm Based on Electric and Vibration Simple Command Tactile Stimulation. Front Hum Neurosci 2021; 15:641357. [PMID: 33935672 PMCID: PMC8081187 DOI: 10.3389/fnhum.2021.641357] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Accepted: 03/09/2021] [Indexed: 12/15/2022] Open
Abstract
This paper proposed a novel tactile-stimuli P300 paradigm for Brain-Computer Interface (BCI), which potentially targeted at people with less learning ability or difficulty in maintaining attention. The new paradigm using only two types of stimuli was designed, and different targets were distinguished by frequency and spatial information. The classification algorithm was developed by introducing filters for frequency bands selection and conducting optimization with common spatial pattern (CSP) on the tactile evoked EEG signals. It features a combination of spatial and frequency information, with the spatial information distinguishing the sites of stimuli and frequency information identifying target stimuli and disturbances. We investigated both electrical stimuli and vibration stimuli, in which only one target site was stimulated in each block. The results demonstrated an average accuracy of 94.88% for electrical stimuli and 95.21% for vibration stimuli, respectively.
Collapse
Affiliation(s)
- Chenxi Chu
- Institute of Artificial Intelligence (AI) and Robotics, Academy for Engineering and Technology, Fudan University, as well as Engineering Research Center of AI & Robotics, Ministry of Education, Shanghai, China
- Guanghua Lingang Engineering Application and Technology R&D (Shanghai) Co., Ltd., Shanghai, China
| | - Jingjing Luo
- Institute of Artificial Intelligence (AI) and Robotics, Academy for Engineering and Technology, Fudan University, as well as Engineering Research Center of AI & Robotics, Ministry of Education, Shanghai, China
- Jihua Laboratory, Guangzhou, China
| | - Xiwei Tian
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and The Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
| | - Xiangke Han
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and The Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
| | - Shijie Guo
- Institute of Artificial Intelligence (AI) and Robotics, Academy for Engineering and Technology, Fudan University, as well as Engineering Research Center of AI & Robotics, Ministry of Education, Shanghai, China
- Guanghua Lingang Engineering Application and Technology R&D (Shanghai) Co., Ltd., Shanghai, China
- Department of the State Key Laboratory of Reliability and Intelligence of Electrical Equipment and The Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin, China
| |
Collapse
|