1
|
Niu L, Bin J, Wang JKS, Zhan G, Jia J, Zhang L, Gan Z, Kang X. Effect of 3D paradigm synchronous motion for SSVEP-based hybrid BCI-VR system. Med Biol Eng Comput 2023; 61:2481-2495. [PMID: 37191865 DOI: 10.1007/s11517-023-02845-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 05/05/2023] [Indexed: 05/17/2023]
Abstract
A brain-computer interface (BCI) system and virtual reality (VR) are integrated as a more interactive hybrid system (BCI-VR) that allows the user to manipulate the car. A virtual scene in the VR system that is the same as the physical environment is built, and the object's movement can be observed in the VR scene. The four-class three-dimensional (3D) paradigm is designed and moves synchronously in virtual reality. The dynamic paradigm may affect their attention according to the experimenters' feedback. Fifteen subjects in our experiment steered the car according to a specified motion trajectory. According to our online experimental result, different motion trajectories of the paradigm have various effects on the system's performance, and training can mitigate this adverse effect. Moreover, the hybrid system using frequencies between 5 and 10 Hz indicates better performance than those using lower or higher stimulation frequencies. The experiment results show a maximum average accuracy of 0.956 and a maximum information transfer rate (ITR) of 41.033 bits/min. It suggests that a hybrid system provides a high-performance way of brain-computer interaction. This research could encourage more interesting applications involving BCI and VR technologies.
Collapse
Affiliation(s)
- Lan Niu
- Laboratory for Neural Interface and Brain Computer Interface, Engineering Research Center of AI & Robotics, Shanghai Engineering Research Center of AI & Robotics, MOE Frontiers Center for Brain Science, State Key Laboratory of Medical Neurobiology, Institute of AI & Robotics, Institute of Meta-Medical, Academy for Engineering & Technology, Ministry of Education, FudanUniversity, Shanghai, China
- Ji Hua Laboratory, Foshan, 528000, Guangdong Province, China
| | - Jianxiong Bin
- Laboratory for Neural Interface and Brain Computer Interface, Engineering Research Center of AI & Robotics, Shanghai Engineering Research Center of AI & Robotics, MOE Frontiers Center for Brain Science, State Key Laboratory of Medical Neurobiology, Institute of AI & Robotics, Institute of Meta-Medical, Academy for Engineering & Technology, Ministry of Education, FudanUniversity, Shanghai, China
- Ji Hua Laboratory, Foshan, 528000, Guangdong Province, China
| | | | - Gege Zhan
- Ji Hua Laboratory, Foshan, 528000, Guangdong Province, China
| | - Jie Jia
- Department of Rehabilitation Medicine, Huashan Hospital, Fudan University, Shanghai, 200040, China
| | - Lihua Zhang
- Laboratory for Neural Interface and Brain Computer Interface, Engineering Research Center of AI & Robotics, Shanghai Engineering Research Center of AI & Robotics, MOE Frontiers Center for Brain Science, State Key Laboratory of Medical Neurobiology, Institute of AI & Robotics, Institute of Meta-Medical, Academy for Engineering & Technology, Ministry of Education, FudanUniversity, Shanghai, China
- Ji Hua Laboratory, Foshan, 528000, Guangdong Province, China
| | - Zhongxue Gan
- Laboratory for Neural Interface and Brain Computer Interface, Engineering Research Center of AI & Robotics, Shanghai Engineering Research Center of AI & Robotics, MOE Frontiers Center for Brain Science, State Key Laboratory of Medical Neurobiology, Institute of AI & Robotics, Institute of Meta-Medical, Academy for Engineering & Technology, Ministry of Education, FudanUniversity, Shanghai, China
- Ji Hua Laboratory, Foshan, 528000, Guangdong Province, China
| | - Xiaoyang Kang
- Laboratory for Neural Interface and Brain Computer Interface, Engineering Research Center of AI & Robotics, Shanghai Engineering Research Center of AI & Robotics, MOE Frontiers Center for Brain Science, State Key Laboratory of Medical Neurobiology, Institute of AI & Robotics, Institute of Meta-Medical, Academy for Engineering & Technology, Ministry of Education, FudanUniversity, Shanghai, China.
- Ji Hua Laboratory, Foshan, 528000, Guangdong Province, China.
- Yiwu Research Institute of Fudan University, Chengbei Road, Yiwu City, 322000, Zhejiang, China.
- Research Center for Intelligent Sensing, Zhejiang Lab, Hangzhou, 311100, China.
| |
Collapse
|
2
|
Niu L, Bin J, kong shuai Wang J, Zhan G, Zhang L, Gan Z, Kang X. A dynamically optimized time-window length for SSVEP based hybrid BCI-VR system. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104826] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/14/2023]
|
3
|
Hazarika A, Rahmati M. Towards an Evolved Immersive Experience: Exploring 5G- and Beyond-Enabled Ultra-Low-Latency Communications for Augmented and Virtual Reality. SENSORS (BASEL, SWITZERLAND) 2023; 23:3682. [PMID: 37050742 PMCID: PMC10098798 DOI: 10.3390/s23073682] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 03/24/2023] [Accepted: 03/31/2023] [Indexed: 06/19/2023]
Abstract
Augmented reality and virtual reality technologies are witnessing an evolutionary change in the 5G and Beyond (5GB) network due to their promising ability to enable an immersive and interactive environment by coupling the virtual world with the real one. However, the requirement of low-latency connectivity, which is defined as the end-to-end delay between the action and the reaction, is very crucial to leverage these technologies for a high-quality immersive experience. This paper provides a comprehensive survey and detailed insight into various advantageous approaches from the hardware and software perspectives, as well as the integration of 5G technology, towards 5GB, in enabling a low-latency environment for AR and VR applications. The contribution of 5GB systems as an outcome of several cutting-edge technologies, such as massive multiple-input, multiple-output (mMIMO) and millimeter wave (mmWave), along with the utilization of artificial intelligence (AI) and machine learning (ML) techniques towards an ultra-low-latency communication system, is also discussed in this paper. The potential of using a visible-light communications (VLC)-guided beam through a learning algorithm for a futuristic, evolved immersive experience of augmented and virtual reality with the ultra-low-latency transmission of multi-sensory tracking information with an optimal scheduling policy is discussed in this paper.
Collapse
|
4
|
Marek K, Zubrycki I, Miller E. Immersion Therapy with Head-Mounted Display for Rehabilitation of the Upper Limb after Stroke-Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:9962. [PMID: 36560328 PMCID: PMC9785384 DOI: 10.3390/s22249962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/07/2022] [Revised: 12/08/2022] [Accepted: 12/14/2022] [Indexed: 06/17/2023]
Abstract
Immersive virtual therapy technology is a new method that uses head-mounted displays for rehabilitation purposes. It offers a realistic experience that puts the user in a virtual reality. This new type of therapy is used in the rehabilitation of stroke patients. Many patients after this disease have complications related to the upper extremities that limit independence in their everyday life, which affects the functioning of society. Conventional neurological rehabilitation can be supplemented by the use of immersive virtual therapy. The system allows patients with upper limb dysfunction to perform a motor and task-oriented training in virtual reality that is individually tailored to their performance. The complete immersion therapy itself is researched and evaluated by medical teams to determine the suitability for rehabilitation of the upper limb after a stroke. The purpose of this article is to provide an overview of the latest research (2019-2022) on immersive virtual reality with head-mounted displays using in rehabilitation of the upper extremities of stroke patients.
Collapse
Affiliation(s)
- Klaudia Marek
- Department of Neurological Rehabilitation, Medical University of Lodz, Milionowa 14, 93-113 Lodz, Poland
| | - Igor Zubrycki
- Institute of Automatic Control, Lodz University of Technology, Stefanowskiego 18, 90-537 Lodz, Poland
| | - Elżbieta Miller
- Department of Neurological Rehabilitation, Medical University of Lodz, Milionowa 14, 93-113 Lodz, Poland
| |
Collapse
|
5
|
Ultra-Reliable Low-Latency Communications: Unmanned Aerial Vehicles Assisted Systems. INFORMATION 2022. [DOI: 10.3390/info13090430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Ultra-reliable low-latency communication (uRLLC) is a group of fifth-generation and sixth-generation (5G/6G) cellular applications with special requirements regarding latency, reliability, and availability. Most of the announced 5G/6G applications are uRLLC that require an end-to-end latency of milliseconds and ultra-high reliability of communicated data. Such systems face many challenges since traditional networks cannot meet such requirements. Thus, novel network structures and technologies have been introduced to enable such systems. Since uRLLC is a promising paradigm that covers many applications, this work considers reviewing the current state of the art of the uRLLC. This includes the main applications, specifications, and main requirements of ultra-reliable low-latency (uRLL) applications. The design challenges of uRLLC systems are discussed, and promising solutions are introduced. The virtual and augmented realities (VR/AR) are considered the main use case of uRLLC, and the current proposals for VR and AR are discussed. Moreover, unmanned aerial vehicles (UAVs) are introduced as enablers of uRLLC. The current research directions and the existing proposals are discussed.
Collapse
|