1
|
Eidel M, Pfeiffer M, Ziebell P, Kübler A. Recording the tactile P300 with the cEEGrid for potential use in a brain-computer interface. Front Hum Neurosci 2024; 18:1371631. [PMID: 38957693 PMCID: PMC11218745 DOI: 10.3389/fnhum.2024.1371631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Accepted: 05/27/2024] [Indexed: 07/04/2024] Open
Abstract
Brain-computer interfaces (BCIs) are scientifically well established, but they rarely arrive in the daily lives of potential end-users. This could be in part because electroencephalography (EEG), a prevalent method to acquire brain activity for BCI operation, is considered too impractical to be applied in daily life of end-users with physical impairment as an assistive device. Hence, miniaturized EEG systems such as the cEEGrid have been developed. While they promise to be a step toward bridging the gap between BCI development, lab demonstrations, and home use, they still require further validation. Encouragingly, the cEEGrid has already demonstrated its ability to record visually and auditorily evoked event-related potentials (ERP), which are important as input signal for many BCIs. With this study, we aimed at evaluating the cEEGrid in the context of a BCI based on tactually evoked ERPs. To compare the cEEGrid with a conventional scalp EEG, we recorded brain activity with both systems simultaneously. Forty healthy participants were recruited to perform a P300 oddball task based on vibrotactile stimulation at four different positions. This tactile paradigm has been shown to be feasible for BCI repeatedly but has never been tested with the cEEGrid. We found distinct P300 deflections in the cEEGrid data, particularly at vertical bipolar channels. With an average of 63%, the cEEGrid classification accuracy was significantly above the chance level (25%) but significantly lower than the 81% reached with the EEG cap. Likewise, the P300 amplitude was significantly lower (cEEGrid R2-R7: 1.87 μV, Cap Cz: 3.53 μV). These results indicate that a tactile BCI using the cEEGrid could potentially be operated, albeit with lower efficiency. Additionally, participants' somatosensory sensitivity was assessed, but no correlation to the accuracy of either EEG system was shown. Our research contributes to the growing amount of literature comparing the cEEGrid to conventional EEG systems and provides first evidence that the tactile P300 can be recorded behind the ear. A BCI based on a thus simplified EEG system might be more readily accepted by potential end-users, provided the accuracy can be substantially increased, e.g., by training and improved classification.
Collapse
Affiliation(s)
- M. Eidel
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | | | | | | |
Collapse
|
2
|
Papadopoulos S, Szul MJ, Congedo M, Bonaiuto JJ, Mattout J. Beta bursts question the ruling power for brain-computer interfaces. J Neural Eng 2024; 21:016010. [PMID: 38167234 DOI: 10.1088/1741-2552/ad19ea] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 01/02/2024] [Indexed: 01/05/2024]
Abstract
Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.
Collapse
Affiliation(s)
- Sotirios Papadopoulos
- University Lyon 1, Lyon, France
- Lyon Neuroscience Research Center, CRNL, INSERM U1028, CNRS, UMR5292, Lyon, France
- Institut de Sciences Cognitives Marc Jeannerod, CNRS, UMR5229, Lyon, France
| | - Maciej J Szul
- University Lyon 1, Lyon, France
- Institut de Sciences Cognitives Marc Jeannerod, CNRS, UMR5229, Lyon, France
| | - Marco Congedo
- GIPSA-lab, University Grenoble Alpes, CNRS, Grenoble-INP, Grenoble, France
| | - James J Bonaiuto
- University Lyon 1, Lyon, France
- Institut de Sciences Cognitives Marc Jeannerod, CNRS, UMR5229, Lyon, France
| | - Jérémie Mattout
- University Lyon 1, Lyon, France
- Lyon Neuroscience Research Center, CRNL, INSERM U1028, CNRS, UMR5292, Lyon, France
| |
Collapse
|
3
|
Tai P, Ding P, Wang F, Gong A, Li T, Zhao L, Su L, Fu Y. Brain-computer interface paradigms and neural coding. Front Neurosci 2024; 17:1345961. [PMID: 38287988 PMCID: PMC10822902 DOI: 10.3389/fnins.2023.1345961] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 12/28/2023] [Indexed: 01/31/2024] Open
Abstract
Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.
Collapse
Affiliation(s)
- Pengrui Tai
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Peng Ding
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Fan Wang
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Anmin Gong
- School of Information Engineering, Chinese People’s Armed Police Force Engineering University, Xi’an, China
| | - Tianwen Li
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
- Faculty of Science, Kunming University of Science and Technology, Kunming, China
| | - Lei Zhao
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
- Faculty of Science, Kunming University of Science and Technology, Kunming, China
| | - Lei Su
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| | - Yunfa Fu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, China
| |
Collapse
|
4
|
Huang Z, Duan X, Zhu G, Zhang S, Wang R, Wang Z. Assessing the data quality of AdHawk MindLink eye-tracking glasses. Behav Res Methods 2024:10.3758/s13428-023-02310-2. [PMID: 38168041 DOI: 10.3758/s13428-023-02310-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/30/2023] [Indexed: 01/05/2024]
Abstract
Most commercially available eye-tracking devices rely on video cameras and image processing algorithms to track gaze. Despite this, emerging technologies are entering the field, making high-speed, cameraless eye-tracking more accessible. In this study, a series of tests were conducted to compare the data quality of MEMS-based eye-tracking glasses (AdHawk MindLink) with three widely used camera-based eye-tracking devices (EyeLink Portable Duo, Tobii Pro Glasses 2, and SMI Eye Tracking Glasses 2). The data quality measures assessed in these tests included accuracy, precision, data loss, and system latency. The results suggest that, overall, the data quality of the eye-tracking glasses was lower compared to that of a desktop EyeLink Portable Duo eye-tracker. Among the eye-tracking glasses, the accuracy and precision of the MindLink eye-tracking glasses were either higher or on par with those of Tobii Pro Glasses 2 and SMI Eye Tracking Glasses 2. The system latency of MindLink was approximately 9 ms, significantly lower than that of camera-based eye-tracking devices found in VR goggles. These results suggest that the MindLink eye-tracking glasses show promise for research applications where high sampling rates and low latency are preferred.
Collapse
Affiliation(s)
- Zehao Huang
- Center for Psychological Sciences, Zhejiang University, 148 Tianmushan Rd., Hangzhou, 310028, China
| | - Xiaoting Duan
- Center for Psychological Sciences, Zhejiang University, 148 Tianmushan Rd., Hangzhou, 310028, China
| | - Gancheng Zhu
- Center for Psychological Sciences, Zhejiang University, 148 Tianmushan Rd., Hangzhou, 310028, China
| | - Shuai Zhang
- Center for Psychological Sciences, Zhejiang University, 148 Tianmushan Rd., Hangzhou, 310028, China
| | - Rong Wang
- Center for Psychological Sciences, Zhejiang University, 148 Tianmushan Rd., Hangzhou, 310028, China
| | - Zhiguo Wang
- Center for Psychological Sciences, Zhejiang University, 148 Tianmushan Rd., Hangzhou, 310028, China.
| |
Collapse
|
5
|
Almajidy RK, Mottaghi S, Ajwad AA, Boudria Y, Mankodiya K, Besio W, Hofmann UG. A case for hybrid BCIs: combining optical and electrical modalities improves accuracy. Front Hum Neurosci 2023; 17:1162712. [PMID: 37351363 PMCID: PMC10282188 DOI: 10.3389/fnhum.2023.1162712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 05/18/2023] [Indexed: 06/24/2023] Open
Abstract
Near-infrared spectroscopy (NIRS) is a promising research tool that found its way into the field of brain-computer interfacing (BCI). BCI is crucially dependent on maximized usability thus demanding lightweight, compact, and low-cost hardware. We designed, built, and validated a hybrid BCI system incorporating one optical and two electrical modalities ameliorating usability issues. The novel hardware consisted of a NIRS device integrated with an electroencephalography (EEG) system that used two different types of electrodes: Regular gelled gold disk electrodes and tri-polar concentric ring electrodes (TCRE). BCI experiments with 16 volunteers implemented a two-dimensional motor imagery paradigm in off- and online sessions. Various non-canonical signal processing methods were used to extract and classify useful features from EEG, tEEG (EEG through TCRE electrodes), and NIRS. Our analysis demonstrated evidence of improvement in classification accuracy when using the TCRE electrodes compared to disk electrodes and the NIRS system. Based on our synchronous hybrid recording system, we could show that the combination of NIRS-EEG-tEEG performed significantly better than either single modality only.
Collapse
Affiliation(s)
- Rand Kasim Almajidy
- Faculty of Medicine, University of Freiburg, Freiburg im Breisgau, Germany
- Section for Neuroelectronic Systems, Department of Neurosurgery, Medical Center University of Freiburg, Freiburg im Breisgau, Germany
| | - Soheil Mottaghi
- Roche Diagnostics Automation Solutions GmbH, Ludwigsburg, Germany
| | - Asmaa A. Ajwad
- College of Medicine, University of Diyala, Baqubah, Iraq
| | - Yacine Boudria
- Electro Standards Laboratories, Cranston, RI, United States
| | - Kunal Mankodiya
- Electrical, Computer and Biomedical Engineering, Kingston, RI, United States
| | - Walter Besio
- Electrical, Computer and Biomedical Engineering, Kingston, RI, United States
| | - Ulrich G. Hofmann
- Faculty of Medicine, University of Freiburg, Freiburg im Breisgau, Germany
- Section for Neuroelectronic Systems, Department of Neurosurgery, Medical Center University of Freiburg, Freiburg im Breisgau, Germany
| |
Collapse
|
6
|
Yadav H, Maini S. Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-45. [PMID: 37362726 PMCID: PMC10157593 DOI: 10.1007/s11042-023-15653-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 07/17/2022] [Accepted: 04/22/2023] [Indexed: 06/28/2023]
Abstract
Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.
Collapse
Affiliation(s)
- Hitesh Yadav
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| | - Surita Maini
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| |
Collapse
|
7
|
Lyu X, Ding P, Li S, Dong Y, Su L, Zhao L, Gong A, Fu Y. Human factors engineering of BCI: an evaluation for satisfaction of BCI based on motor imagery. Cogn Neurodyn 2023; 17:105-118. [PMID: 36704636 PMCID: PMC9871150 DOI: 10.1007/s11571-022-09808-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 03/04/2022] [Accepted: 04/01/2022] [Indexed: 01/29/2023] Open
Abstract
Existing brain-computer interface (BCI) research has made great progress in improving the accuracy and information transfer rate (ITR) of BCI systems. However, the practicability of BCI is still difficult to achieve. One of the important reasons for this difficulty is that human factors are not fully considered in the research and development of BCI. As a result, BCI systems have not yet reached users' expectations. In this study, we investigate a BCI system of motor imagery for lower limb synchronous rehabilitation as an example. From the perspective of human factors engineering of BCI, a comprehensive evaluation method of BCI system development is proposed based on the concept of human-centered design and evaluation. Subjects' satisfaction ratings for BCI sensors, visual analog scale (VAS), subjects' satisfaction rating of the BCI system, and the mental workload rating for subjects manipulating the BCI system, as well as interview/follow-up comprehensive evaluation of motor imagery of BCI (MI-BCI) system satisfaction were used. The methods and concepts proposed in this study provide useful insights for the design of personalized MI-BCI. We expect that the human factors engineering of BCI could be applied to the design and satisfaction evaluation of MI-BCI, so as to promote the practical application of this kind of BCI.
Collapse
Affiliation(s)
- Xiaotong Lyu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, Yunnan China
| | - Peng Ding
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, Yunnan China
| | - Siyu Li
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, Yunnan China
| | - Yuyang Dong
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, Yunnan China
| | - Lei Su
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan China
| | - Lei Zhao
- Faculty of Science, Kunming University of Science and Technology, Kunming, Yunnan China
| | - Anmin Gong
- School of Information Engineering, Chinese People’s Armed Police Force Engineering University, Xian, Shanxi China
| | - Yunfa Fu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan China
- Brain Cognition and Brain-Computer Intelligence Integration Group, Kunming University of Science and Technology, Kunming, Yunnan China
| |
Collapse
|
8
|
Identifying Thematics in a Brain-Computer Interface Research. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2023; 2023:2793211. [PMID: 36643889 PMCID: PMC9833923 DOI: 10.1155/2023/2793211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 12/21/2022] [Accepted: 12/24/2022] [Indexed: 01/05/2023]
Abstract
This umbrella review is motivated to understand the shift in research themes on brain-computer interfacing (BCI) and it determined that a shift away from themes that focus on medical advancement and system development to applications that included education, marketing, gaming, safety, and security has occurred. The background of this review examined aspects of BCI categorisation, neuroimaging methods, brain control signal classification, applications, and ethics. The specific area of BCI software and hardware development was not examined. A search using One Search was undertaken and 92 BCI reviews were selected for inclusion. Publication demographics indicate the average number of authors on review papers considered was 4.2 ± 1.8. The results also indicate a rapid increase in the number of BCI reviews from 2003, with only three reviews before that period, two in 1972, and one in 1996. While BCI authors were predominantly Euro-American in early reviews, this shifted to a more global authorship, which China dominated by 2020-2022. The review revealed six disciplines associated with BCI systems: life sciences and biomedicine (n = 42), neurosciences and neurology (n = 35), and rehabilitation (n = 20); (2) the second domain centred on the theme of functionality: computer science (n = 20), engineering (n = 28) and technology (n = 38). There was a thematic shift from understanding brain function and modes of interfacing BCI systems to more applied research novel areas of research-identified surround artificial intelligence, including machine learning, pre-processing, and deep learning. As BCI systems become more invasive in the lives of "normal" individuals, it is expected that there will be a refocus and thematic shift towards increased research into ethical issues and the need for legal oversight in BCI application.
Collapse
|
9
|
Saichoo T, Boonbrahm P, Punsawad Y. Investigating User Proficiency of Motor Imagery for EEG-Based BCI System to Control Simulated Wheelchair. SENSORS (BASEL, SWITZERLAND) 2022; 22:9788. [PMID: 36560158 PMCID: PMC9781917 DOI: 10.3390/s22249788] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 12/01/2022] [Accepted: 12/12/2022] [Indexed: 06/17/2023]
Abstract
The research on the electroencephalography (EEG)-based brain-computer interface (BCI) is widely utilized for wheelchair control. The ability of the user is one factor of BCI efficiency. Therefore, we focused on BCI tasks and protocols to yield high efficiency from the robust EEG features of individual users. This study proposes a task-based brain activity to gain the power of the alpha band, which included eyes closed for alpha response at the occipital area, attention to an upward arrow for alpha response at the frontal area, and an imagined left/right motor for alpha event-related desynchronization at the left/right motor cortex. An EPOC X neuroheadset was used to acquire the EEG signals. We also proposed user proficiency in motor imagery sessions with limb movement paradigms by recommending motor imagination tasks. Using the proposed system, we verified the feature extraction algorithms and command translation. Twelve volunteers participated in the experiment, and the conventional paradigm of motor imagery was used to compare the efficiencies. With utilized user proficiency in motor imagery, an average accuracy of 83.7% across the left and right commands was achieved. The recommended MI paradigm via user proficiency achieved an approximately 4% higher accuracy than the conventional MI paradigm. Moreover, the real-time control results of a simulated wheelchair revealed a high efficiency based on the time condition. The time results for the same task as the joystick-based control were still approximately three times longer. We suggest that user proficiency be used to recommend an individual MI paradigm for beginners. Furthermore, the proposed BCI system can be used for electric wheelchair control by people with severe disabilities.
Collapse
Affiliation(s)
- Theerat Saichoo
- School of Informatics, Walailak University, Nakhon Si Thammarat 80160, Thailand
| | - Poonpong Boonbrahm
- School of Informatics, Walailak University, Nakhon Si Thammarat 80160, Thailand
| | - Yunyong Punsawad
- School of Informatics, Walailak University, Nakhon Si Thammarat 80160, Thailand
- Informatics Innovative Center of Excellence, Walailak University, Nakhon Si Thammarat 80160, Thailand
| |
Collapse
|
10
|
Galiotta V, Quattrociocchi I, D'Ippolito M, Schettini F, Aricò P, Sdoia S, Formisano R, Cincotti F, Mattia D, Riccio A. EEG-based Brain-Computer Interfaces for people with Disorders of Consciousness: Features and applications. A systematic review. Front Hum Neurosci 2022; 16:1040816. [PMID: 36545350 PMCID: PMC9760911 DOI: 10.3389/fnhum.2022.1040816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 11/17/2022] [Indexed: 12/11/2022] Open
Abstract
Background Disorders of Consciousness (DoC) are clinical conditions following a severe acquired brain injury (ABI) characterized by absent or reduced awareness, known as coma, Vegetative State (VS)/Unresponsive Wakefulness Syndrome (VS/UWS), and Minimally Conscious State (MCS). Misdiagnosis rate between VS/UWS and MCS is attested around 40% due to the clinical and behavioral fluctuations of the patients during bedside consciousness assessments. Given the large body of evidence that some patients with DoC possess "covert" awareness, revealed by neuroimaging and neurophysiological techniques, they are candidates for intervention with brain-computer interfaces (BCIs). Objectives The aims of the present work are (i) to describe the characteristics of BCI systems based on electroencephalography (EEG) performed on DoC patients, in terms of control signals adopted to control the system, characteristics of the paradigm implemented, classification algorithms and applications (ii) to evaluate the performance of DoC patients with BCI. Methods The search was conducted on Pubmed, Web of Science, Scopus and Google Scholar. The PRISMA guidelines were followed in order to collect papers published in english, testing a BCI and including at least one DoC patient. Results Among the 527 papers identified with the first run of the search, 27 papers were included in the systematic review. Characteristics of the sample of participants, behavioral assessment, control signals employed to control the BCI, the classification algorithms, the characteristics of the paradigm, the applications and performance of BCI were the data extracted from the study. Control signals employed to operate the BCI were: P300 (N = 19), P300 and Steady-State Visual Evoked Potentials (SSVEP; hybrid system, N = 4), sensorimotor rhythms (SMRs; N = 5) and brain rhythms elicited by an emotional task (N = 1), while assessment, communication, prognosis, and rehabilitation were the possible applications of BCI in DoC patients. Conclusion Despite the BCI is a promising tool in the management of DoC patients, supporting diagnosis and prognosis evaluation, results are still preliminary, and no definitive conclusions may be drawn; even though neurophysiological methods, such as BCI, are more sensitive to covert cognition, it is suggested to adopt a multimodal approach and a repeated assessment strategy.
Collapse
Affiliation(s)
- Valentina Galiotta
- Neuroelectric Imaging and Brain-Computer Interface Laboratory, Fondazione Santa Lucia (IRCCS), Rome, Italy,Department of Psychology, Sapienza University of Rome, Rome, Italy
| | - Ilaria Quattrociocchi
- Neuroelectric Imaging and Brain-Computer Interface Laboratory, Fondazione Santa Lucia (IRCCS), Rome, Italy,Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Rome, Italy
| | - Mariagrazia D'Ippolito
- Neuroelectric Imaging and Brain-Computer Interface Laboratory, Fondazione Santa Lucia (IRCCS), Rome, Italy,*Correspondence: Mariagrazia D'Ippolito
| | - Francesca Schettini
- Neuroelectric Imaging and Brain-Computer Interface Laboratory, Fondazione Santa Lucia (IRCCS), Rome, Italy,Servizio di Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Pietro Aricò
- Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Rome, Italy,Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy,BrainSigns srl, Rome, Italy
| | - Stefano Sdoia
- Department of Psychology, Sapienza University of Rome, Rome, Italy
| | - Rita Formisano
- Neurorehabilitation 2 and Post-Coma Unit, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Febo Cincotti
- Department of Computer, Control, and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Rome, Italy
| | - Donatella Mattia
- Neuroelectric Imaging and Brain-Computer Interface Laboratory, Fondazione Santa Lucia (IRCCS), Rome, Italy,Servizio di Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Angela Riccio
- Neuroelectric Imaging and Brain-Computer Interface Laboratory, Fondazione Santa Lucia (IRCCS), Rome, Italy,Servizio di Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| |
Collapse
|
11
|
Mussi MG, Adams KD. EEG hybrid brain-computer interfaces: A scoping review applying an existing hybrid-BCI taxonomy and considerations for pediatric applications. Front Hum Neurosci 2022; 16:1007136. [DOI: 10.3389/fnhum.2022.1007136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2022] [Accepted: 10/27/2022] [Indexed: 11/18/2022] Open
Abstract
Most hybrid brain-computer interfaces (hBCI) aim at improving the performance of single-input BCI. Many combinations are possible to configure an hBCI, such as using multiple brain input signals, different stimuli or more than one input system. Multiple studies have been done since 2010 where such interfaces have been tested and analyzed. Results and conclusions are promising but little has been discussed as to what is the best approach for the pediatric population, should they use hBCI as an assistive technology. Children might face greater challenges when using BCI and might benefit from less complex interfaces. Hence, in this scoping review we included 42 papers that developed hBCI systems for the purpose of control of assistive devices or communication software, and we analyzed them through the lenses of potential use in clinical settings and for children. We extracted taxonomic categories proposed in previous studies to describe the types of interfaces that have been developed. We also proposed interface characteristics that could be observed in different hBCI, such as type of target, number of targets and number of steps before selection. Then, we discussed how each of the extracted characteristics could influence the overall complexity of the system and what might be the best options for applications for children. Effectiveness and efficiency were also collected and included in the analysis. We concluded that the least complex hBCI interfaces might involve having a brain inputs and an external input, with a sequential role of operation, and visual stimuli. Those interfaces might also use a minimal number of targets of the strobic type, with one or two steps before the final selection. We hope this review can be used as a guideline for future hBCI developments and as an incentive to the design of interfaces that can also serve children who have motor impairments.
Collapse
|
12
|
Mughal NE, Khan MJ, Khalil K, Javed K, Sajid H, Naseer N, Ghafoor U, Hong KS. EEG-fNIRS-based hybrid image construction and classification using CNN-LSTM. Front Neurorobot 2022; 16:873239. [PMID: 36119719 PMCID: PMC9472125 DOI: 10.3389/fnbot.2022.873239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 07/20/2022] [Indexed: 11/13/2022] Open
Abstract
The constantly evolving human–machine interaction and advancement in sociotechnical systems have made it essential to analyze vital human factors such as mental workload, vigilance, fatigue, and stress by monitoring brain states for optimum performance and human safety. Similarly, brain signals have become paramount for rehabilitation and assistive purposes in fields such as brain–computer interface (BCI) and closed-loop neuromodulation for neurological disorders and motor disabilities. The complexity, non-stationary nature, and low signal-to-noise ratio of brain signals pose significant challenges for researchers to design robust and reliable BCI systems to accurately detect meaningful changes in brain states outside the laboratory environment. Different neuroimaging modalities are used in hybrid settings to enhance accuracy, increase control commands, and decrease the time required for brain activity detection. Functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) measure the hemodynamic and electrical activity of the brain with a good spatial and temporal resolution, respectively. However, in hybrid settings, where both modalities enhance the output performance of BCI, their data compatibility due to the huge discrepancy between their sampling rate and the number of channels remains a challenge for real-time BCI applications. Traditional methods, such as downsampling and channel selection, result in important information loss while making both modalities compatible. In this study, we present a novel recurrence plot (RP)-based time-distributed convolutional neural network and long short-term memory (CNN-LSTM) algorithm for the integrated classification of fNIRS EEG for hybrid BCI applications. The acquired brain signals are first projected into a non-linear dimension with RPs and fed into the CNN to extract essential features without performing any downsampling. Then, LSTM is used to learn the chronological features and time-dependence relation to detect brain activity. The average accuracies achieved with the proposed model were 78.44% for fNIRS, 86.24% for EEG, and 88.41% for hybrid EEG-fNIRS BCI. Moreover, the maximum accuracies achieved were 85.9, 88.1, and 92.4%, respectively. The results confirm the viability of the RP-based deep-learning algorithm for successful BCI systems.
Collapse
Affiliation(s)
- Nabeeha Ehsan Mughal
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Muhammad Jawad Khan
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
- National Center of Artificial Intelligence (NCAI) – NUST, Islamabad, Pakistan
| | - Khurram Khalil
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Kashif Javed
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Hasan Sajid
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
- National Center of Artificial Intelligence (NCAI) – NUST, Islamabad, Pakistan
| | - Noman Naseer
- Department of Mechatronics and Biomedical Engineering, Air University, Islamabad, Pakistan
| | - Usman Ghafoor
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
- *Correspondence: Keum-Shik Hong
| |
Collapse
|
13
|
Yue Z, Wu Q, Ren SY, Li M, Shi B, Pan Y, Wang J. A novel multiple time-frequency sequential coding strategy for hybrid brain-computer interface. Front Hum Neurosci 2022; 16:859259. [PMID: 35966991 PMCID: PMC9372511 DOI: 10.3389/fnhum.2022.859259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Accepted: 06/28/2022] [Indexed: 11/18/2022] Open
Abstract
Background For brain-computer interface (BCI) communication, electroencephalography provides a preferable choice due to its high temporal resolution and portability over other neural recording techniques. However, current BCIs are unable to sufficiently use the information from time and frequency domains simultaneously. Thus, we proposed a novel hybrid time-frequency paradigm to investigate better ways of using the time and frequency information. Method We adopt multiple omitted stimulus potential (OSP) and steady-state motion visual evoked potential (SSMVEP) to design the hybrid paradigm. A series of pre-experiments were undertaken to study factors that would influence the feasibility of the hybrid paradigm and the interaction between multiple features. After that, a novel Multiple Time-Frequencies Sequential Coding (MTFSC) strategy was introduced and explored in experiments. Results Omissions with multiple short and long durations could effectively elicit time and frequency features, including the multi-OSP, ERP, and SSVEP in this hybrid paradigm. The MTFSC was feasible and efficient. The preliminary online analysis showed that the accuracy and the ITR of the nine-target stimulator over thirteen subjects were 89.04% and 36.37 bits/min. Significance This study first combined the SSMVEP and multi-OSP in a hybrid paradigm to produce robust and abundant time features for coding BCI. Meanwhile, the MTFSC proved feasible and showed great potential in improving performance, such as expanding the number of BCI targets by better using time information in specific stimulated frequencies. This study holds promise for designing better BCI systems with a novel coding method.
Collapse
Affiliation(s)
- Zan Yue
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Qiong Wu
- Beijing Tsinghua Changgeng Hospital, Tsinghua University, Beijing, China
| | - Shi-Yuan Ren
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Man Li
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Bin Shi
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
| | - Yu Pan
- Beijing Tsinghua Changgeng Hospital, Tsinghua University, Beijing, China
- *Correspondence: Yu Pan
| | - Jing Wang
- Institute of Robotics and Intelligent Systems, Xi'an Jiaotong University, Xi'an, China
- Jing Wang
| |
Collapse
|
14
|
Peters B, Eddy B, Galvin-McLaughlin D, Betz G, Oken B, Fried-Oken M. A systematic review of research on augmentative and alternative communication brain-computer interface systems for individuals with disabilities. Front Hum Neurosci 2022; 16:952380. [PMID: 35966988 PMCID: PMC9374067 DOI: 10.3389/fnhum.2022.952380] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Augmentative and alternative communication brain-computer interface (AAC-BCI) systems are intended to offer communication access to people with severe speech and physical impairment (SSPI) without requiring volitional movement. As the field moves toward clinical implementation of AAC-BCI systems, research involving participants with SSPI is essential. Research has demonstrated variability in AAC-BCI system performance across users, and mixed results for comparisons of performance for users with and without disabilities. The aims of this systematic review were to (1) describe study, system, and participant characteristics reported in BCI research, (2) summarize the communication task performance of participants with disabilities using AAC-BCI systems, and (3) explore any differences in performance for participants with and without disabilities. Electronic databases were searched in May, 2018, and March, 2021, identifying 6065 records, of which 73 met inclusion criteria. Non-experimental study designs were common and sample sizes were typically small, with approximately half of studies involving five or fewer participants with disabilities. There was considerable variability in participant characteristics, and in how those characteristics were reported. Over 60% of studies reported an average selection accuracy ≤70% for participants with disabilities in at least one tested condition. However, some studies excluded participants who did not reach a specific system performance criterion, and others did not state whether any participants were excluded based on performance. Twenty-nine studies included participants both with and without disabilities, but few reported statistical analyses comparing performance between the two groups. Results suggest that AAC-BCI systems show promise for supporting communication for people with SSPI, but they remain ineffective for some individuals. The lack of standards in reporting outcome measures makes it difficult to synthesize data across studies. Further research is needed to demonstrate efficacy of AAC-BCI systems for people who experience SSPI of varying etiologies and severity levels, and these individuals should be included in system design and testing. Consensus in terminology and consistent participant, protocol, and performance description will facilitate the exploration of user and system characteristics that positively or negatively affect AAC-BCI use, and support innovations that will make this technology more useful to a broader group of people. Clinical trial registration https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018095345, PROSPERO: CRD42018095345.
Collapse
Affiliation(s)
- Betts Peters
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| | - Brandon Eddy
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
- Speech and Hearing Sciences Department, Portland State University, Portland, OR, United States
| | - Deirdre Galvin-McLaughlin
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| | - Gail Betz
- Health Sciences and Human Services Library, University of Maryland, Baltimore, MD, United States
| | - Barry Oken
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- Department of Neurology, Oregon Health & Science University, Portland, OR, United States
| | - Melanie Fried-Oken
- Consortium for Accessible Multimodal Brain-Body Interfaces, United States
- REKNEW Projects, Institute on Development and Disability, Department of Pediatrics, Oregon Health and Science University, Portland, OR, United States
| |
Collapse
|
15
|
Riccio A, Schettini F, Galiotta V, Giraldi E, Grasso MG, Cincotti F, Mattia D. Usability of a Hybrid System Combining P300-Based Brain-Computer Interface and Commercial Assistive Technologies to Enhance Communication in People With Multiple Sclerosis. Front Hum Neurosci 2022; 16:868419. [PMID: 35721361 PMCID: PMC9204311 DOI: 10.3389/fnhum.2022.868419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Accepted: 04/26/2022] [Indexed: 11/25/2022] Open
Abstract
Brain-computer interface (BCI) can provide people with motor disabilities with an alternative channel to access assistive technology (AT) software for communication and environmental interaction. Multiple sclerosis (MS) is a chronic disease of the central nervous system that mostly starts in young adulthood and often leads to a long-term disability, possibly exacerbated by the presence of fatigue. Patients with MS have been rarely considered as potential BCI end-users. In this pilot study, we evaluated the usability of a hybrid BCI (h-BCI) system that enables both a P300-based BCI and conventional input devices (i.e., muscular dependent) to access mainstream applications through the widely used AT software for communication “Grid 3.” The evaluation was performed according to the principles of the user-centered design (UCD) with the aim of providing patients with MS with an alternative control channel (i.e., BCI), potentially less sensitive to fatigue. A total of 13 patients with MS were enrolled. In session I, participants were presented with a widely validated P300-based BCI (P3-speller); in session II, they had to operate Grid 3 to access three mainstream applications with (1) an AT conventional input device and (2) the h-BCI. Eight patients completed the protocol. Five out of eight patients with MS were successfully able to access the Grid 3 via the BCI, with a mean online accuracy of 83.3% (± 14.6). Effectiveness (online accuracy), satisfaction, and workload were comparable between the conventional AT inputs and the BCI channel in controlling the Grid 3. As expected, the efficiency (time for correct selection) resulted to be significantly lower for the BCI with respect to the AT conventional channels (Z = 0.2, p < 0.05). Although cautious due to the limited sample size, these preliminary findings indicated that the BCI control channel did not have a detrimental effect with respect to conventional AT channels on the ability to operate an AT software (Grid 3). Therefore, we inferred that the usability of the two access modalities was comparable. The integration of BCI with commercial AT input devices to access a widely used AT software represents an important step toward the introduction of BCIs into the AT centers’ daily practice.
Collapse
Affiliation(s)
- Angela Riccio
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
- Servizio Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
- *Correspondence: Angela Riccio,
| | - Francesca Schettini
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
- Servizio Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Valentina Galiotta
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Enrico Giraldi
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | | | - Febo Cincotti
- Department of Computer, Control and Management Engineering Antonio Ruberti, Sapienza University of Rome, Rome, Italy
| | - Donatella Mattia
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
- Servizio Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| |
Collapse
|
16
|
Chandler JA, Van der Loos KI, Boehnke S, Beaudry JS, Buchman DZ, Illes J. Brain Computer Interfaces and Communication Disabilities: Ethical, Legal, and Social Aspects of Decoding Speech From the Brain. Front Hum Neurosci 2022; 16:841035. [PMID: 35529778 PMCID: PMC9069963 DOI: 10.3389/fnhum.2022.841035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2021] [Accepted: 03/03/2022] [Indexed: 11/28/2022] Open
Abstract
A brain-computer interface technology that can decode the neural signals associated with attempted but unarticulated speech could offer a future efficient means of communication for people with severe motor impairments. Recent demonstrations have validated this approach. Here we assume that it will be possible in future to decode imagined (i.e., attempted but unarticulated) speech in people with severe motor impairments, and we consider the characteristics that could maximize the social utility of a BCI for communication. As a social interaction, communication involves the needs and goals of both speaker and listener, particularly in contexts that have significant potential consequences. We explore three high-consequence legal situations in which neurally-decoded speech could have implications: Testimony, where decoded speech is used as evidence; Consent and Capacity, where it may be used as a means of agency and participation such as consent to medical treatment; and Harm, where such communications may be networked or may cause harm to others. We then illustrate how design choices might impact the social and legal acceptability of these technologies.
Collapse
Affiliation(s)
- Jennifer A. Chandler
- Bertram Loeb Research Chair, Faculty of Law, University of Ottawa, Ottawa, ON, Canada
- *Correspondence: Jennifer A. Chandler,
| | | | - Susan Boehnke
- Centre for Neuroscience Studies, Queen’s University, Kingston, ON, Canada
| | - Jonas S. Beaudry
- Institute for Health and Social Policy (IHSP) and Faculty of Law, McGill University, Montreal, QC, Canada
| | - Daniel Z. Buchman
- Centre for Addiction and Mental Health, Dalla Lana School of Public Health, Krembil Research Institute, University of Toronto Joint Centre for Bioethics, Toronto, ON, Canada
| | - Judy Illes
- Division of Neurology, Department of Medicine, University of British Columbia, Vancouver, BC, Canada
| |
Collapse
|
17
|
Zaunseder S, Vehkaoja A, Fleischhauer V, Hoog Antink C. Signal-to-noise ratio is more important than sampling rate in beat-to-beat interval estimation from optical sensors. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103538] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
|
18
|
Han Y, Ziebell P, Riccio A, Halder S. Two sides of the same coin: adaptation of BCIs to internal states with user-centered design and electrophysiological features. BRAIN-COMPUTER INTERFACES 2022. [DOI: 10.1080/2326263x.2022.2041294] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Affiliation(s)
- Yiyuan Han
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
| | - Philipp Ziebell
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - Angela Riccio
- Neuroelectrical Imaging and Brain Computer Interface Laboratory,Fondazione Santa Lucia, Irccs, Rome, Italy
| | - Sebastian Halder
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
| |
Collapse
|
19
|
Poststroke Cognitive Impairment Research Progress on Application of Brain-Computer Interface. BIOMED RESEARCH INTERNATIONAL 2022; 2022:9935192. [PMID: 35252458 PMCID: PMC8896931 DOI: 10.1155/2022/9935192] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Revised: 12/20/2021] [Accepted: 12/23/2021] [Indexed: 12/19/2022]
Abstract
Brain-computer interfaces (BCIs), a new type of rehabilitation technology, pick up nerve cell signals, identify and classify their activities, and convert them into computer-recognized instructions. This technique has been widely used in the rehabilitation of stroke patients in recent years and appears to promote motor function recovery after stroke. At present, the application of BCI in poststroke cognitive impairment is increasing, which is a common complication that also affects the rehabilitation process. This paper reviews the promise and potential drawbacks of using BCI to treat poststroke cognitive impairment, providing a solid theoretical basis for the application of BCI in this area.
Collapse
|
20
|
Papadopoulos S, Bonaiuto J, Mattout J. An Impending Paradigm Shift in Motor Imagery Based Brain-Computer Interfaces. Front Neurosci 2022; 15:824759. [PMID: 35095410 PMCID: PMC8789741 DOI: 10.3389/fnins.2021.824759] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Accepted: 12/21/2021] [Indexed: 01/11/2023] Open
Abstract
The development of reliable assistive devices for patients that suffer from motor impairments following central nervous system lesions remains a major challenge in the field of non-invasive Brain-Computer Interfaces (BCIs). These approaches are predominated by electroencephalography and rely on advanced signal processing and machine learning methods to extract neural correlates of motor activity. However, despite tremendous and still ongoing efforts, their value as effective clinical tools remains limited. We advocate that a rather overlooked research avenue lies in efforts to question neurophysiological markers traditionally targeted in non-invasive motor BCIs. We propose an alternative approach grounded by recent fundamental advances in non-invasive neurophysiology, specifically subject-specific feature extraction of sensorimotor bursts of activity recorded via (possibly magnetoencephalography-optimized) electroencephalography. This path holds promise in overcoming a significant proportion of existing limitations, and could foster the wider adoption of online BCIs in rehabilitation protocols.
Collapse
Affiliation(s)
- Sotirios Papadopoulos
- University Lyon 1, Lyon, France
- Lyon Neuroscience Research Center, CRNL, INSERM, U1028, CNRS, UMR 5292, Lyon, France
- Institut des Sciences Cognitives Marc Jeannerod, CNRS, UMR 5229, Bron, France
- *Correspondence: Sotirios Papadopoulos,
| | - James Bonaiuto
- University Lyon 1, Lyon, France
- Institut des Sciences Cognitives Marc Jeannerod, CNRS, UMR 5229, Bron, France
| | - Jérémie Mattout
- University Lyon 1, Lyon, France
- Lyon Neuroscience Research Center, CRNL, INSERM, U1028, CNRS, UMR 5292, Lyon, France
| |
Collapse
|
21
|
Colamarino E, de Seta V, Masciullo M, Cincotti F, Mattia D, Pichiorri F, Toppi J. Corticomuscular and Intermuscular Coupling in Simple Hand Movements to Enable a Hybrid Brain-Computer Interface. Int J Neural Syst 2021; 31:2150052. [PMID: 34590990 DOI: 10.1142/s0129065721500520] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Hybrid Brain-Computer Interfaces (BCIs) for upper limb rehabilitation after stroke should enable the reinforcement of "more normal" brain and muscular activity. Here, we propose the combination of corticomuscular coherence (CMC) and intermuscular coherence (IMC) as control features for a novel hybrid BCI for rehabilitation purposes. Multiple electroencephalographic (EEG) signals and surface electromyography (EMG) from 5 muscles per side were collected in 20 healthy participants performing finger extension (Ext) and grasping (Grasp) with both dominant and non-dominant hand. Grand average of CMC and IMC patterns showed a bilateral sensorimotor area as well as multiple muscles involvement. CMC and IMC values were used as features to classify each task versus rest and Ext versus Grasp. We demonstrated that a combination of CMC and IMC features allows for classification of both movements versus rest with better performance (Area Under the receiver operating characteristic Curve, AUC) for the Ext movement (0.97) with respect to Grasp (0.88). Classification of Ext versus Grasp also showed high performances (0.99). All in all, these preliminary findings indicate that the combination of CMC and IMC could provide for a comprehensive framework for simple hand movements to eventually be employed in a hybrid BCI system for post-stroke rehabilitation.
Collapse
Affiliation(s)
- Emma Colamarino
- Department of Computer, Control and Management Engineering, Sapienza University of Rome, Via Ariosto 25, Rome 00185, Italy.,Fondazione Santa Lucia IRCCS, Via Ardeatina 306-354, Rome 00179, Italy
| | - Valeria de Seta
- Department of Computer, Control and Management Engineering, Sapienza University of Rome, Via Ariosto 25, Rome 00185, Italy.,Fondazione Santa Lucia IRCCS, Via Ardeatina 306-354, Rome 00179, Italy
| | | | - Febo Cincotti
- Department of Computer, Control and Management Engineering, Sapienza University of Rome, Via Ariosto 25, Rome 00185, Italy.,Fondazione Santa Lucia IRCCS, Via Ardeatina 306-354, Rome 00179, Italy
| | - Donatella Mattia
- Fondazione Santa Lucia IRCCS, Via Ardeatina 306-354, Rome 00179, Italy
| | | | - Jlenia Toppi
- Department of Computer, Control and Management Engineering, Sapienza University of Rome, Via Ariosto 25, Rome 00185, Italy.,Fondazione Santa Lucia IRCCS, Via Ardeatina 306-354, Rome 00179, Italy
| |
Collapse
|
22
|
Antonietti A, Balachandran P, Hossaini A, Hu Y, Valeriani D. The BCI Glossary: a first proposal for a community review. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.1969789] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Alberto Antonietti
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy
| | | | - Ali Hossaini
- Department of Engineering, King’s College London, London, UK
| | - Yaoping Hu
- Department of Electrical and Software Engineering,University of Calgary, Calgary, AB, Canada
| | - Davide Valeriani
- Department of Otolaryngology Head and Neck Surgery, Massachusetts Eye and Ear, Boston, MA, USA
- Department of Otolaryngology Head and Neck Surgery, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
23
|
Portillo-Lara R, Tahirbegi B, Chapman CAR, Goding JA, Green RA. Mind the gap: State-of-the-art technologies and applications for EEG-based brain-computer interfaces. APL Bioeng 2021; 5:031507. [PMID: 34327294 PMCID: PMC8294859 DOI: 10.1063/5.0047237] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2021] [Accepted: 05/19/2021] [Indexed: 11/14/2022] Open
Abstract
Brain-computer interfaces (BCIs) provide bidirectional communication between the brain and output devices that translate user intent into function. Among the different brain imaging techniques used to operate BCIs, electroencephalography (EEG) constitutes the preferred method of choice, owing to its relative low cost, ease of use, high temporal resolution, and noninvasiveness. In recent years, significant progress in wearable technologies and computational intelligence has greatly enhanced the performance and capabilities of EEG-based BCIs (eBCIs) and propelled their migration out of the laboratory and into real-world environments. This rapid translation constitutes a paradigm shift in human-machine interaction that will deeply transform different industries in the near future, including healthcare and wellbeing, entertainment, security, education, and marketing. In this contribution, the state-of-the-art in wearable biosensing is reviewed, focusing on the development of novel electrode interfaces for long term and noninvasive EEG monitoring. Commercially available EEG platforms are surveyed, and a comparative analysis is presented based on the benefits and limitations they provide for eBCI development. Emerging applications in neuroscientific research and future trends related to the widespread implementation of eBCIs for medical and nonmedical uses are discussed. Finally, a commentary on the ethical, social, and legal concerns associated with this increasingly ubiquitous technology is provided, as well as general recommendations to address key issues related to mainstream consumer adoption.
Collapse
Affiliation(s)
- Roberto Portillo-Lara
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Bogachan Tahirbegi
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Christopher A. R. Chapman
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Josef A. Goding
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| | - Rylie A. Green
- Department of Bioengineering, Imperial College London, Royal School of Mines, London SW7 2AZ, United Kingdom
| |
Collapse
|
24
|
Fedotchev A, Parin S, Polevaya S, Zemlianaia A. EEG-based musical neurointerfaces in the correction of stress-induced states. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.1964874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Alexander Fedotchev
- Department of Psychophysiology, Lobachevsky State University of Nizhni Novgorod, Nizhny Novgorod, Russia
- Department of Mechanisms of Reception, Institute of Cell Biophysics, Russian Academy of Sciences, Pushchino, Moscow Region, Russia
| | - Sergey Parin
- Department of Psychophysiology, Lobachevsky State University of Nizhni Novgorod, Nizhny Novgorod, Russia
| | - Sofia Polevaya
- Department of Psychophysiology, Lobachevsky State University of Nizhni Novgorod, Nizhny Novgorod, Russia
| | - Anna Zemlianaia
- Department of Psychophysiology, Moscow Research Institute of Psychiatry, Branch of the Serbsky‘ National Medical Research Center of Psychiatry and Narcology, Russian Ministry of Health, Moscow, Russia
| |
Collapse
|
25
|
Solving the SSVEP Paradigm Using the Nonlinear Canonical Correlation Analysis Approach. SENSORS 2021; 21:s21165308. [PMID: 34450750 PMCID: PMC8439358 DOI: 10.3390/s21165308] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Revised: 07/31/2021] [Accepted: 08/01/2021] [Indexed: 01/16/2023]
Abstract
This paper presents the implementation of nonlinear canonical correlation analysis (NLCCA) approach to detect steady-state visual evoked potentials (SSVEP) quickly. The need for the fast recognition of proper stimulus to help end an SSVEP task in a BCI system is justified due to the flickering external stimulus exposure that causes users to start to feel fatigued. Measuring the accuracy and exposure time can be carried out through the information transfer rate-ITR, which is defined as a relationship between the precision, the number of stimuli, and the required time to obtain a result. NLCCA performance was evaluated by comparing it with two other approaches-the well-known canonical correlation analysis (CCA) and the least absolute reduction and selection operator (LASSO), both commonly used to solve the SSVEP paradigm. First, the best average ITR value was found from a dataset comprising ten healthy users with an average age of 28, where an exposure time of one second was obtained. In addition, the time sliding window responses were observed immediately after and around 200 ms after the flickering exposure to obtain the phase effects through the coefficient of variation (CV), where NLCCA obtained the lowest value. Finally, in order to obtain statistical significance to demonstrate that all approaches differ, the accuracy and ITR from the time sliding window responses was compared using a statistical analysis of variance per approach to identify differences between them using Tukey's test.
Collapse
|
26
|
Orlandi S, House SC, Karlsson P, Saab R, Chau T. Brain-Computer Interfaces for Children With Complex Communication Needs and Limited Mobility: A Systematic Review. Front Hum Neurosci 2021; 15:643294. [PMID: 34335203 PMCID: PMC8319030 DOI: 10.3389/fnhum.2021.643294] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Accepted: 05/18/2021] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interfaces (BCIs) represent a new frontier in the effort to maximize the ability of individuals with profound motor impairments to interact and communicate. While much literature points to BCIs' promise as an alternative access pathway, there have historically been few applications involving children and young adults with severe physical disabilities. As research is emerging in this sphere, this article aims to evaluate the current state of translating BCIs to the pediatric population. A systematic review was conducted using the Scopus, PubMed, and Ovid Medline databases. Studies of children and adolescents that reported BCI performance published in English in peer-reviewed journals between 2008 and May 2020 were included. Twelve publications were identified, providing strong evidence for continued research in pediatric BCIs. Research evidence was generally at multiple case study or exploratory study level, with modest sample sizes. Seven studies focused on BCIs for communication and five on mobility. Articles were categorized and grouped based on type of measurement (i.e., non-invasive and invasive), and the type of brain signal (i.e., sensory evoked potentials or movement-related potentials). Strengths and limitations of studies were identified and used to provide requirements for clinical translation of pediatric BCIs. This systematic review presents the state-of-the-art of pediatric BCIs focused on developing advanced technology to support children and youth with communication disabilities or limited manual ability. Despite a few research studies addressing the application of BCIs for communication and mobility in children, results are encouraging and future works should focus on customizable pediatric access technologies based on brain activity.
Collapse
Affiliation(s)
- Silvia Orlandi
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Sarah C. House
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Petra Karlsson
- Cerebral Palsy Alliance, The University of Sydney, Sydney, NSW, Australia
| | - Rami Saab
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Tom Chau
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
- Institute of Biomedical Engineering (BME), University of Toronto, Toronto, ON, Canada
| |
Collapse
|
27
|
Singh HP, Kumar P. Developments in the human machine interface technologies and their applications: a review. J Med Eng Technol 2021; 45:552-573. [PMID: 34184601 DOI: 10.1080/03091902.2021.1936237] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
Human-machine interface (HMI) techniques use bioelectrical signals to gain real-time synchronised communication between the human body and machine functioning. HMI technology not only provides a real-time control access but also has the ability to control multiple functions at a single instance of time with modest human inputs and increased efficiency. The HMI technologies yield advanced control access on numerous applications such as health monitoring, medical diagnostics, development of prosthetic and assistive devices, automotive and aerospace industry, robotic controls and many more fields. In this paper, various physiological signals, their acquisition and processing techniques along with their respective applications in different HMI technologies have been discussed.
Collapse
Affiliation(s)
- Harpreet Pal Singh
- Department of Mechanical Engineering, Punjabi University, Patiala, India
| | - Parlad Kumar
- Department of Mechanical Engineering, Punjabi University, Patiala, India
| |
Collapse
|
28
|
Arif S, Khan MJ, Naseer N, Hong KS, Sajid H, Ayaz Y. Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface. Front Hum Neurosci 2021; 15:658444. [PMID: 33994983 PMCID: PMC8121150 DOI: 10.3389/fnhum.2021.658444] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Accepted: 03/09/2021] [Indexed: 11/13/2022] Open
Abstract
A passive brain-computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects' data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.
Collapse
Affiliation(s)
- Saad Arif
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan
| | - Muhammad Jawad Khan
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan.,National Center of Artificial Intelligence (NCAI), Islamabad, Pakistan
| | - Noman Naseer
- Department of Mechatronics Engineering, Air University, Islamabad, Pakistan
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
| | - Hasan Sajid
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan.,National Center of Artificial Intelligence (NCAI), Islamabad, Pakistan
| | - Yasar Ayaz
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad, Pakistan.,National Center of Artificial Intelligence (NCAI), Islamabad, Pakistan
| |
Collapse
|
29
|
Kinney-Lang E, Kelly D, Floreani ED, Jadavji Z, Rowley D, Zewdie ET, Anaraki JR, Bahari H, Beckers K, Castelane K, Crawford L, House S, Rauh CA, Michaud A, Mussi M, Silver J, Tuck C, Adams K, Andersen J, Chau T, Kirton A. Advancing Brain-Computer Interface Applications for Severely Disabled Children Through a Multidisciplinary National Network: Summary of the Inaugural Pediatric BCI Canada Meeting. Front Hum Neurosci 2020; 14:593883. [PMID: 33343318 PMCID: PMC7744376 DOI: 10.3389/fnhum.2020.593883] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Accepted: 11/10/2020] [Indexed: 11/15/2022] Open
Abstract
Thousands of youth suffering from acquired brain injury or other early-life neurological disease live, mature, and learn with only limited communication and interaction with their world. Such cognitively capable children are ideal candidates for brain-computer interfaces (BCI). While BCI systems are rapidly evolving, a fundamental gap exists between technological innovators and the patients and families who stand to benefit. Forays into translating BCI systems to children in recent years have revealed that kids can learn to operate simple BCI with proficiency akin to adults. BCI could bring significant boons to the lives of many children with severe physical impairment, supporting their complex physical and social needs. However, children have been neglected in BCI research and a collaborative BCI research community is required to unite and push pediatric BCI development forward. To this end, the pediatric BCI Canada collaborative network (BCI-CAN) was formed, under a unified goal to cooperatively drive forward pediatric BCI innovation and impact. This article reflects on the topics and discussions raised in the foundational BCI-CAN meeting held in Toronto, ON, Canada in November 2019 and suggests the next steps required to see BCI impact the lives of children with severe neurological disease and their families.
Collapse
Affiliation(s)
- Eli Kinney-Lang
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Dion Kelly
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Erica D Floreani
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Zeanna Jadavji
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Danette Rowley
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Ephrem Takele Zewdie
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Javad R Anaraki
- Department of Rehabilitation Science, Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada.,PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Hosein Bahari
- I CAN Centre, Glenrose Rehabilitation Hospital, Alberta Health Services, Edmonton, AB, Canada
| | - Kim Beckers
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Karen Castelane
- PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Lindsey Crawford
- PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Sarah House
- PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Chelsea A Rauh
- PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Amber Michaud
- I CAN Centre, Glenrose Rehabilitation Hospital, Alberta Health Services, Edmonton, AB, Canada
| | - Matheus Mussi
- Assistive Technology Laboratory, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada
| | - Jessica Silver
- PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Corinne Tuck
- I CAN Centre, Glenrose Rehabilitation Hospital, Alberta Health Services, Edmonton, AB, Canada
| | - Kim Adams
- Assistive Technology Laboratory, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada
| | - John Andersen
- I CAN Centre, Glenrose Rehabilitation Hospital, Alberta Health Services, Edmonton, AB, Canada.,Assistive Technology Laboratory, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada
| | - Tom Chau
- Department of Rehabilitation Science, Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada.,PRISM Laboratory, Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada
| | - Adam Kirton
- Department of Pediatrics and Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, AB, Canada.,Cumming School of Medicine, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| |
Collapse
|
30
|
Mouli S, Palaniappan R. DIY hybrid SSVEP-P300 LED stimuli for BCI platform using EMOTIV EEG headset. HARDWAREX 2020; 8:e00113. [PMID: 35498243 PMCID: PMC9041272 DOI: 10.1016/j.ohx.2020.e00113] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/26/2019] [Revised: 05/19/2020] [Accepted: 05/20/2020] [Indexed: 06/14/2023]
Abstract
A fully customisable chip-on board (COB) LED design to evoke two brain responses simultaneously (steady state visual evoked potential (SSVEP) and transient evoked potential, P300) is discussed in this paper. Considering different possible modalities in brain-computer interfacing (BCI), SSVEP is widely accepted as it requires a lesser number of electroencephalogram (EEG) electrodes and minimal training time. The aim of this work was to produce a hybrid BCI hardware platform to evoke SSVEP and P300 precisely with reduced fatigue and improved classification performance. The system comprises of four independent radial green visual stimuli controlled individually by a 32-bit microcontroller platform to evoke SSVEP and four red LEDs flashing at random intervals to generate P300 events. The system can also record the P300 event timestamps that can be used in classification, to improve the accuracy and reliability. The hybrid stimulus was tested for real-time classification accuracy by controlling a LEGO robot to move in four directions.
Collapse
Affiliation(s)
- Surej Mouli
- Data Science Research Group, School of Computing, University of Kent
| | | |
Collapse
|
31
|
Brain-to-Brain Neural Synchrony During Social Interactions: A Systematic Review on Hyperscanning Studies. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10196669] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
The aim of this study was to conduct a comprehensive review on hyperscanning research (measuring brain activity simultaneously from more than two people interacting) using an explicit systematic method, the preferred reporting items for systematic reviews and meta-analyses (PRISMA). Data were searched from IEEE Xplore, PubMed, Engineering Village, Web of Science and Scopus databases. Inclusion criteria were journal articles written in English from 2000 to 19 June 2019. A total of 126 empirical studies were screened out to address three specific questions regarding the neuroimaging method, the application domain, and the experiment paradigm. Results showed that the most used neuroimaging method with hyperscanning was magnetoencephalography/electroencephalography (MEG/EEG; 47%), and the least used neuroimaging method was hyper-transcranial Alternating Current Stimulation (tACS) (1%). Applications in cognition accounted for almost half the studies (48%), while educational applications accounted for less than 5% of the studies. Applications in decision-making tasks were the second most common (26%), shortly followed by applications in motor synchronization (23%). The findings from this systematic review that were based on documented, transparent and reproducible searches should help build cumulative knowledge and guide future research regarding inter-brain neural synchrony during social interactions, that is, hyperscanning research.
Collapse
|
32
|
The performance impact of data augmentation in CSP-based motor-imagery systems for BCI applications. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.102152] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register]
|
33
|
Abiri R, Borhani S, Kilmarx J, Esterwood C, Jiang Y, Zhao X. A Usability Study of Low-cost Wireless Brain-Computer Interface for Cursor Control Using Online Linear Model. IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS 2020; 50:287-297. [PMID: 33777542 PMCID: PMC7990128 DOI: 10.1109/thms.2020.2983848] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Computer cursor control using electroencephalogram (EEG) signals is a common and well-studied brain-computer interface (BCI). The emphasis of the literature has been primarily on evaluation of the objective measures of assistive BCIs such as accuracy of the neural decoder whereas the subjective measures such as user's satisfaction play an essential role for the overall success of a BCI. As far as we know, the BCI literature lacks a comprehensive evaluation of the usability of the mind-controlled computer cursor in terms of decoder efficiency (accuracy), user experience, and relevant confounding variables concerning the platform for the public use. To fill this gap, we conducted a two-dimensional EEG-based cursor control experiment among 28 healthy participants. The computer cursor velocity was controlled by the imagery of hand movement using a paradigm presented in the literature named imagined body kinematics (IBK) with a low-cost wireless EEG headset. We evaluated the usability of the platform for different objective and subjective measures while we investigated the extent to which the training phase may influence the ultimate BCI outcome. We conducted pre- and post- BCI experiment interview questionnaires to evaluate the usability. Analyzing the questionnaires and the testing phase outcome shows a positive correlation between the individuals' ability of visualization and their level of mental controllability of the cursor. Despite individual differences, analyzing training data shows the significance of electrooculogram (EOG) on the predictability of the linear model. The results of this work may provide useful insights towards designing a personalized user-centered assistive BCI.
Collapse
Affiliation(s)
- Reza Abiri
- Dept. of Neurology at University of California, San Francisco/Berkeley and Dept. of Mechanical, Aerospace, and Biomedical Engineering at the University of Tennessee, Knoxville
| | - Soheil Borhani
- Department of Mechanical, Aerospace, and Biomedical Engineering, The University of Tennessee, Knoxville, TN 37996 USA
| | - Justin Kilmarx
- Department of Mechanical, Aerospace, and Biomedical Engineering, The University of Tennessee, Knoxville, TN 37996 USA
| | - Connor Esterwood
- College Communication and Information at the University of Tennessee, Knoxville, TN, USA
| | - Yang Jiang
- Department of Behavioral Science, College of Medicine, at University of Kentucky, Lexington KY, USA
| | - Xiaopeng Zhao
- Department of Mechanical, Aerospace, and Biomedical Engineering, University of Tennessee, Knoxville, TN 37996 USA
| |
Collapse
|
34
|
Ziebell P, Stümpfig J, Eidel M, Kleih SC, Kübler A, Latoschik ME, Halder S. Stimulus modality influences session-to-session transfer of training effects in auditory and tactile streaming-based P300 brain-computer interfaces. Sci Rep 2020; 10:11873. [PMID: 32681134 PMCID: PMC7368044 DOI: 10.1038/s41598-020-67887-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Accepted: 05/22/2020] [Indexed: 12/02/2022] Open
Abstract
Despite recent successes, patients suffering from locked-in syndrome (LIS) still struggle to communicate using vision-independent brain–computer interfaces (BCIs). In this study, we compared auditory and tactile BCIs, regarding training effects and cross-stimulus-modality transfer effects, when switching between stimulus modalities. We utilized a streaming-based P300 BCI, which was developed as a low workload approach to prevent potential BCI-inefficiency. We randomly assigned 20 healthy participants to two groups. The participants received three sessions of training either using an auditory BCI or using a tactile BCI. In an additional fourth session, BCI versions were switched to explore possible cross-stimulus-modality transfer effects. Both BCI versions could be operated successfully in the first session by the majority of the participants, with the tactile BCI being experienced as more intuitive. Significant training effects were found mostly in the auditory BCI group and strong evidence for a cross-stimulus-modality transfer occurred for the auditory training group that switched to the tactile version but not vice versa. All participants were able to control at least one BCI version, suggesting that the investigated paradigms are generally feasible and merit further research into their applicability with LIS end-users. Individual preferences regarding stimulus modality should be considered.
Collapse
Affiliation(s)
- P Ziebell
- Institute of Psychology, University of Würzburg, Würzburg, Germany.
| | - J Stümpfig
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - M Eidel
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - S C Kleih
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - A Kübler
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - M E Latoschik
- Institute of Computer Science, University of Würzburg, Würzburg, Germany
| | - S Halder
- School of Computer Science and Electronic Engineering (CSEE), University of Essex, Colchester, UK
| |
Collapse
|
35
|
Idowu OP, Huang J, Zhao Y, Samuel OW, Yu M, Fang P, Li G. A stacked sparse auto-encoder and back propagation network model for sensory event detection via a flexible ECoG. Cogn Neurodyn 2020; 14:591-607. [PMID: 33014175 DOI: 10.1007/s11571-020-09603-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Revised: 04/22/2020] [Accepted: 05/22/2020] [Indexed: 01/22/2023] Open
Abstract
Current prostheses are limited in their ability to provide direct sensory feedback to users with missing limb. Several efforts have been made to restore tactile sensation to amputees but the somatotopic tactile feedback often results in unnatural sensations, and it is yet unclear how and what information the somatosensory system receives during voluntary movement. The present study proposes an efficient model of stacked sparse autoencoder and back propagation neural network for detecting sensory events from a highly flexible electrocorticography (ECoG) electrode. During the mechanical stimulation with Von Frey (VF) filament on the plantar surface of rats' foot, simultaneous recordings of tactile afferent signals were obtained from primary somatosensory cortex (S1) in the brain. In order to achieve a model with optimal performance, Particle Swarm Optimization and Adaptive Moment Estimation (Adam) were adopted to select the appropriate number of neurons, hidden layers and learning rate of each sparse auto-encoder. We evaluated the stimulus-evoked sensation by using an automated up-down (UD) method otherwise called UDReader. The assessment of tactile thresholds with VF shows that the right side of the hind-paw was significantly more sensitive at the tibia-(p = 6.50 × 10-4), followed by the saphenous-(p = 7.84 × 10-4), and sural-(p = 8.24 × 10-4). We then validated our proposed model by comparing with the state-of-the-art methods, and recorded accuracy of 98.8%, sensitivity of 96.8%, and specificity of 99.1%. Hence, we demonstrated the effectiveness of our algorithms in detecting sensory events through flexible ECoG recordings which could be a viable option in restoring somatosensory feedback.
Collapse
Affiliation(s)
- Oluwagbenga Paul Idowu
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| | - Jianping Huang
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| | - Yang Zhao
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| | - Oluwarotimi William Samuel
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| | - Mei Yu
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| | - Peng Fang
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| | - Guanglin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, 518055 China.,Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, 518055 China
| |
Collapse
|
36
|
Carmona L, Diez PF, Laciar E, Mut V. Multisensory Stimulation and EEG Recording Below the Hair-Line: A New Paradigm on Brain Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng 2020; 28:825-831. [PMID: 32149649 DOI: 10.1109/tnsre.2020.2979684] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
To test the feasibility of implementing multisensory (auditory and visual) stimulation in combination with electrodes placed on non-hair positions to design more efficient and comfortable Brain-computer interfaces (BCI). Fifteen volunteers participated in the experiments. They were stimulated by visual, auditory and multisensory stimuli set at 37, 38, 39 and 40Hz and at different phases (0°, 90°, 180° and 270°). The electroencephalogram (EEG) was measured from Oz, T7, T8, Tp9 and Tp10 positions. To evaluate the amplitude of the visual and auditory evoked potentials, the signal-to-noise ratio (SNR) was used and the accuracy of detection was calculated using canonical correlation analysis. Additionally, the volunteers were asked about the discomfort of each kind of stimulus. The multisensory stimulation allows for attaining higher SNR on every electrode. Non-hair (Tp9 and Tp10) positions attained SNR and accuracy similar to the ones obtained from occipital positions on visual stimulation. No significant difference was found on the discomfort produced by each kind of stimulation. The results demonstrated that multisensory stimulation can help in obtaining high amplitude steady-state evoked responses with a similar discomfort level. Then, it is possible to design a more efficient and comfortable hybrid-BCI based on multisensory stimulation and electrodes on non-hair positions. The current article proposes a new paradigm for hybrid-BCI based on steady-state evoked potentials measured from the area behind-the-ears and elicited by multisensory stimulation, thus, allowing subjects to achieve similar performance to the one achieved by visual-occipital BCI, but measuring the EEG on a more comfortable electrode location.
Collapse
|
37
|
Mehdizavareh MH, Hemati S, Soltanian-Zadeh H. Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs. PLoS One 2020; 15:e0226048. [PMID: 31935220 PMCID: PMC6959579 DOI: 10.1371/journal.pone.0226048] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Accepted: 11/17/2019] [Indexed: 11/18/2022] Open
Abstract
Recently, brain-computer interface (BCI) systems developed based on steady-state visual evoked potential (SSVEP) have attracted much attention due to their high information transfer rate (ITR) and increasing number of targets. However, SSVEP-based methods can be improved in terms of their accuracy and target detection time. We propose a new method based on canonical correlation analysis (CCA) to integrate subject-specific models and subject-independent information and enhance BCI performance. We propose to use training data of other subjects to optimize hyperparameters for CCA-based model of a specific subject. An ensemble version of the proposed method is also developed for a fair comparison with ensemble task-related component analysis (TRCA). The proposed method is compared with TRCA and extended CCA methods. A publicly available, 35-subject SSVEP benchmark dataset is used for comparison studies and performance is quantified by classification accuracy and ITR. The ITR of the proposed method is higher than those of TRCA and extended CCA. The proposed method outperforms extended CCA in all conditions and TRCA for time windows greater than 0.3 s. The proposed method also outperforms TRCA when there are limited training blocks and electrodes. This study illustrates that adding subject-independent information to subject-specific models can improve performance of SSVEP-based BCIs.
Collapse
Affiliation(s)
- Mohammad Hadi Mehdizavareh
- CIPCE, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran
| | - Sobhan Hemati
- CIPCE, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran
| | - Hamid Soltanian-Zadeh
- CIPCE, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran
- Medical Image Analysis Laboratory, Departments of Radiology and Research Administration, Henry Ford Health System, Detroit, MI, United States of America
- * E-mail: ,
| |
Collapse
|
38
|
López S, Cervantes JA, Cervantes S, Molina J, Cervantes F. The plausibility of using unmanned aerial vehicles as a serious game for dealing with attention deficit-hyperactivity disorder. COGN SYST RES 2020. [DOI: 10.1016/j.cogsys.2019.09.013] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/17/2023]
|
39
|
Badesa FJ, Diez JA, Catalan JM, Trigili E, Cordella F, Nann M, Crea S, Soekadar SR, Zollo L, Vitiello N, Garcia-Aracil N. Physiological Responses During Hybrid BNCI Control of an Upper-Limb Exoskeleton. SENSORS (BASEL, SWITZERLAND) 2019; 19:E4931. [PMID: 31726745 PMCID: PMC6891352 DOI: 10.3390/s19224931] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2019] [Revised: 10/30/2019] [Accepted: 11/05/2019] [Indexed: 11/20/2022]
Abstract
When combined with assistive robotic devices, such as wearable robotics, brain/neural-computer interfaces (BNCI) have the potential to restore the capabilities of handicapped people to carry out activities of daily living. To improve applicability of such systems, workload and stress should be reduced to a minimal level. Here, we investigated the user's physiological reactions during the exhaustive use of the interfaces of a hybrid control interface. Eleven BNCI-naive healthy volunteers participated in the experiments. All participants sat in a comfortable chair in front of a desk and wore a whole-arm exoskeleton as well as wearable devices for monitoring physiological, electroencephalographic (EEG) and electrooculographic (EoG) signals. The experimental protocol consisted of three phases: (i) Set-up, calibration and BNCI training; (ii) Familiarization phase; and (iii) Experimental phase during which each subject had to perform EEG and EoG tasks. After completing each task, the NASA-TLX questionnaire and self-assessment manikin (SAM) were completed by the user. We found significant differences (p-value < 0.05) in heart rate variability (HRV) and skin conductance level (SCL) between participants during the use of the two different biosignal modalities (EEG, EoG) of the BNCI. This indicates that EEG control is associated with a higher level of stress (associated with a decrease in HRV) and mental work load (associated with a higher level of SCL) when compared to EoG control. In addition, HRV and SCL modulations correlated with the subject's workload perception and emotional responses assessed through NASA-TLX questionnaires and SAM.
Collapse
Affiliation(s)
- Francisco J. Badesa
- Miguel Hernández University of Elche, Av. Universidad w/n, Ed. Innova, 03202 Alicante, Spain; (J.M.C.); (N.G.-A.)
- Universidad de Cádiz, Av. de la Universidad n10, 11519 Puerto Real, Spain
- New technologies for Neurorehabilitation Lab., Av. de la Hospitalidad, s/n, 28054 Madrid, Spain
| | - Jorge A. Diez
- Miguel Hernández University of Elche, Av. Universidad w/n, Ed. Innova, 03202 Alicante, Spain; (J.M.C.); (N.G.-A.)
- New technologies for Neurorehabilitation Lab., Av. de la Hospitalidad, s/n, 28054 Madrid, Spain
| | - Jose Maria Catalan
- Miguel Hernández University of Elche, Av. Universidad w/n, Ed. Innova, 03202 Alicante, Spain; (J.M.C.); (N.G.-A.)
- New technologies for Neurorehabilitation Lab., Av. de la Hospitalidad, s/n, 28054 Madrid, Spain
| | - Emilio Trigili
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Viale Rinaldo Piaggio 34, 56025 Pontedera, Pisa, Italy; (E.T.); (S.C.); (N.V.)
| | - Francesca Cordella
- Unit of Advanced Robotics and Human-centred Technologies, Campus Bio-Medico University of Rome, 00128 Rome, Italy; (F.C.); (L.Z.)
| | - Marius Nann
- Applied Neurotechnology Laboratory, Department of Psychiatry and Psychotherapy, University Hopsital of Tübingen, Calwerstr. 14, 72076 Tübingen, Germany;
| | - Simona Crea
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Viale Rinaldo Piaggio 34, 56025 Pontedera, Pisa, Italy; (E.T.); (S.C.); (N.V.)
- IRCCS Fondazione Don Carlo Gnocchi, Via Alfonso Capecelatro 66, 20148 Milan, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, 56025 Pontedera, Pisa, Italy
| | - Surjo R. Soekadar
- Clinical Neurotechnology Laboratory, Department of Psychiatry and Psychotherapy (CCM), Charité-Universitätsmedizin Berlin, Charitéplatz 1, 10117 Berlin, Germany;
| | - Loredana Zollo
- Unit of Advanced Robotics and Human-centred Technologies, Campus Bio-Medico University of Rome, 00128 Rome, Italy; (F.C.); (L.Z.)
| | - Nicola Vitiello
- The BioRobotics Institute, Scuola Superiore Sant’Anna, Viale Rinaldo Piaggio 34, 56025 Pontedera, Pisa, Italy; (E.T.); (S.C.); (N.V.)
- IRCCS Fondazione Don Carlo Gnocchi, Via Alfonso Capecelatro 66, 20148 Milan, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, 56025 Pontedera, Pisa, Italy
| | - Nicolas Garcia-Aracil
- Miguel Hernández University of Elche, Av. Universidad w/n, Ed. Innova, 03202 Alicante, Spain; (J.M.C.); (N.G.-A.)
- New technologies for Neurorehabilitation Lab., Av. de la Hospitalidad, s/n, 28054 Madrid, Spain
| |
Collapse
|
40
|
Gabriel PG, Chen KJ, Alasfour A, Pailla T, Doyle WK, Devinsky O, Friedman D, Dugan P, Melloni L, Thesen T, Gonda D, Sattar S, Wang SG, Gilja V. Neural correlates of unstructured motor behaviors. J Neural Eng 2019; 16:066026. [DOI: 10.1088/1741-2552/ab355c] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
41
|
de Neeling M, Van Hulle MM. Single-paradigm and hybrid brain computing interfaces and their use by disabled patients. J Neural Eng 2019; 16:061001. [DOI: 10.1088/1741-2552/ab2706] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
42
|
Salazar-Ramirez A, Martin JI, Martinez R, Arruti A, Muguerza J, Sierra B. A hierarchical architecture for recognising intentionality in mental tasks on a brain-computer interface. PLoS One 2019; 14:e0218181. [PMID: 31211812 PMCID: PMC6581259 DOI: 10.1371/journal.pone.0218181] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2019] [Accepted: 05/28/2019] [Indexed: 11/19/2022] Open
Abstract
A brain-computer interface (BCI), based on motor imagery EEG, uses information extracted from the electroencephalography signals generated by a person who intends to perform any action. One of the most important issues of current research is how to detect automatically whether the user intends to send some message to a certain device. This study presents a proposal, based on a hierarchical structured system, for recognising intentional and non-intentional mental tasks on a BCI system by applying machine learning techniques to the EEG signals. First-level clustering is performed to distinguish between intentional control (IC) and non-intentional control (NC) state patterns. Then, the patterns recognised as IC are passed on to a second stage where supervised learning techniques are used to classify them. In BCI applications, it is critical to correctly classify NC states with a low false positive rate (FPR) to avoid undesirable effects. According to the literature, we selected a maximum FPR of 10%. Under these conditions, our proposal achieved an average test accuracy of 66.6%, with an 8.2% FPR, for the BCI competition IIIa dataset. The main contribution of this paper is the hierarchical approach, based on machine learning paradigms, which performs intentional and non-intentional discrimination and, depending on the case, classifies the intended command selected by the user.
Collapse
Affiliation(s)
- Asier Salazar-Ramirez
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| | - Jose I. Martin
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
- * E-mail:
| | - Raquel Martinez
- Department of System Engineering and Automation, University of the Basque Country (UPV/EHU), Bilbao, Spain
| | - Andoni Arruti
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| | - Javier Muguerza
- Department of Computer Architecture and Technology, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| | - Basilio Sierra
- Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain
| |
Collapse
|
43
|
Khalaf A, Sejdic E, Akcakaya M. EEG-fTCD hybrid brain-computer interface using template matching and wavelet decomposition. J Neural Eng 2019; 16:036014. [PMID: 30818297 DOI: 10.1088/1741-2552/ab0b7f] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
OBJECTIVE We aim at developing a hybrid brain-computer interface that utilizes electroencephalography (EEG) and functional transcranial Doppler (fTCD). In this hybrid BCI, EEG and fTCD are used simultaneously to measure electrical brain activity and cerebral blood velocity respectively in response to flickering mental rotation (MR) and word generation (WG) tasks. In this paper, we improve both the accuracy and information transfer rate (ITR) of this novel hybrid brain computer interface (BCI) we designed in our previous work. APPROACH To achieve such aim, we extended our feature extraction approach through using template matching and multi-scale analysis to extract EEG and fTCD features, respectively. In particular, template matching was used to analyze EEG data whereas 5-level wavelet decomposition was applied to fTCD data. Significant EEG and fTCD features were selected using Wilcoxon signed rank test. Support vector machines classifier (SVM) was used to project EEG and fTCD selected features of each trial into scalar SVM scores. Moreover, instead of concatenating EEG and fTCD feature vectors corresponding to each trial, we proposed a Bayesian fusion approach of EEG and fTCD evidences. MAIN RESULTS Average accuracy and average ITR of 98.11% and 21.29 bits min-1 were achieved for WG versus MR classification while MR versus baseline yielded 86.27% average accuracy and 8.95 bit min-1 average ITR. In addition, average accuracy of 85.29% and average ITR of 8.34 bits min-1 were obtained for WG versus baseline. SIGNIFICANCE The proposed analysis techniques significantly improved the hybrid BCI performance. Specifically, for MR/WG versus baseline problems, we achieved twice of the ITRs obtained in our previous study. Moreover, the ITR of WG versus MR problem is 4-times the ITR we obtained before for the same problem. The current analysis methods boosted the performance of our EEG-fTCD BCI such that it outperformed the existing EEG-fNIRS BCIs in comparison.
Collapse
|
44
|
Abiri R, Borhani S, Sellers EW, Jiang Y, Zhao X. A comprehensive review of EEG-based brain–computer interface paradigms. J Neural Eng 2019; 16:011001. [DOI: 10.1088/1741-2552/aaf12e] [Citation(s) in RCA: 270] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
45
|
Silva GA. A New Frontier: The Convergence of Nanotechnology, Brain Machine Interfaces, and Artificial Intelligence. Front Neurosci 2018; 12:843. [PMID: 30505265 PMCID: PMC6250836 DOI: 10.3389/fnins.2018.00843] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2018] [Accepted: 10/29/2018] [Indexed: 12/17/2022] Open
Abstract
A confluence of technological capabilities is creating an opportunity for machine learning and artificial intelligence (AI) to enable "smart" nanoengineered brain machine interfaces (BMI). This new generation of technologies will be able to communicate with the brain in ways that support contextual learning and adaptation to changing functional requirements. This applies to both invasive technologies aimed at restoring neurological function, as in the case of neural prosthesis, as well as non-invasive technologies enabled by signals such as electroencephalograph (EEG). Advances in computation, hardware, and algorithms that learn and adapt in a contextually dependent way will be able to leverage the capabilities that nanoengineering offers the design and functionality of BMI. We explore the enabling capabilities that these devices may exhibit, why they matter, and the state of the technologies necessary to build them. We also discuss a number of open technical challenges and problems that will need to be solved in order to achieve this.
Collapse
Affiliation(s)
- Gabriel A. Silva
- Departments of Bioengineering and Neurosciences, Center for Engineered Natural Intelligence, University of California San Diego, La Jolla, CA, United States
| |
Collapse
|
46
|
Khalaf A, Sejdic E, Akcakaya M. Towards optimal visual presentation design for hybrid EEG-fTCD brain-computer interfaces. J Neural Eng 2018; 15:056019. [PMID: 30021931 DOI: 10.1088/1741-2552/aad46f] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
OBJECTIVE In this paper, we introduce a novel hybrid brain-computer interface (BCI) system that measures electrical brain activity as well as cerebral blood velocity using electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD) respectively in response to flickering mental rotation (MR) and flickering word generation (WG) cognitive tasks as well as a fixation cross that represents the baseline. This work extends our previous approach, in which we showed that motor imagery induces simultaneous changes in EEG and fTCD to enable task discrimination; and hence, provides a design approach for a hybrid BCI. Here, we show that instead of using motor imagery, the proposed visual stimulation technique enables the design of an EEG-fTCD based BCI with higher accuracy. APPROACH Features based on the power spectrum of EEG and fTCD signals were calculated. Mutual information and support vector machines were used for feature selection and classification purposes. MAIN RESULTS EEG-fTCD combination outperformed EEG by 4.05% accuracy for MR versus baseline problem and by 5.81% accuracy for WG versus baseline problem. An average accuracy of 92.38% was achieved for MR versus WG problem using the hybrid combination. Average transmission rates of 4.39, 3.92, and 5.60 bits min-1 were obtained for MR versus baseline, WG versus baseline, and MR versus WG problems respectively. SIGNIFICANCE In terms of accuracy, the current visual presentation outperforms the motor imagery visual presentation we designed before for the EEG-fTCD system by 10% accuracy for task versus task problem. Moreover, the proposed system outperforms the state of the art hybrid EEG-fNIRS BCIs in terms of accuracy and/or information transfer rate. Even though there are still limitations of the proposed system, such promising results show that the proposed hybrid system is a feasible candidate for real-time BCIs.
Collapse
|
47
|
Zhang X, Yong X, Menon C. Evaluating the versatility of EEG models generated from motor imagery tasks: An exploratory investigation on upper-limb elbow-centered motor imagery tasks. PLoS One 2017; 12:e0188293. [PMID: 29186170 PMCID: PMC5706687 DOI: 10.1371/journal.pone.0188293] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2017] [Accepted: 11/03/2017] [Indexed: 11/19/2022] Open
Abstract
Electroencephalography (EEG) has recently been considered for use in rehabilitation of people with motor deficits. EEG data from the motor imagery of different body movements have been used, for instance, as an EEG-based control method to send commands to rehabilitation devices that assist people to perform a variety of different motor tasks. However, it is both time and effort consuming to go through data collection and model training for every rehabilitation task. In this paper, we investigate the possibility of using an EEG model from one type of motor imagery (e.g.: elbow extension and flexion) to classify EEG from other types of motor imagery activities (e.g.: open a drawer). In order to study the problem, we focused on the elbow joint. Specifically, nine kinesthetic motor imagery tasks involving the elbow were investigated in twelve healthy individuals who participated in the study. While results reported that models from goal-oriented motor imagery tasks had higher accuracy than models from the simple joint tasks in intra-task testing (e.g., model from elbow extension and flexion task was tested on EEG data collected from elbow extension and flexion task), models from simple joint tasks had higher accuracies than the others in inter-task testing (e.g., model from elbow extension and flexion task tested on EEG data collected from drawer opening task). Simple single joint motor imagery tasks could, therefore, be considered for training models to potentially reduce the number of repetitive data acquisitions and model training in rehabilitation applications.
Collapse
Affiliation(s)
- Xin Zhang
- Menrva Research Group, Schools of Mechatronic Systems Engineering and Engineering Science, Simon Fraser University, Metro Vancouver, British Columbia, Canada
| | - Xinyi Yong
- Menrva Research Group, Schools of Mechatronic Systems Engineering and Engineering Science, Simon Fraser University, Metro Vancouver, British Columbia, Canada
| | - Carlo Menon
- Menrva Research Group, Schools of Mechatronic Systems Engineering and Engineering Science, Simon Fraser University, Metro Vancouver, British Columbia, Canada
| |
Collapse
|