1
|
Petit J, Rouillard J, Cabestaing F. EEG-based brain-computer interfaces exploiting steady-state somatosensory-evoked potentials: a literature review. J Neural Eng 2021; 18. [PMID: 34725311 DOI: 10.1088/1741-2552/ac2fc4] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 10/14/2021] [Indexed: 11/11/2022]
Abstract
A brain-computer interface (BCI) aims to derive commands from the user's brain activity in order to relay them to an external device. To do so, it can either detect a spontaneous change in the mental state, in the so-called 'active' BCIs, or a transient or sustained change in the brain response to an external stimulation, in 'reactive' BCIs. In the latter, external stimuli are perceived by the user through a sensory channel, usually sight or hearing. When the stimulation is sustained and periodical, the brain response reaches an oscillatory steady-state that can be detected rather easily. We focus our attention on electroencephalography-based BCIs (EEG-based BCI) in which a periodical signal, either mechanical or electrical, stimulates the user skin. This type of stimulus elicits a steady-state response of the somatosensory system that can be detected in the recorded EEG. The oscillatory and phase-locked voltage component characterising this response is called a steady-state somatosensory-evoked potential (SSSEP). It has been shown that the amplitude of the SSSEP is modulated by specific mental tasks, for instance when the user focuses their attention or not to the somatosensory stimulation, allowing the translation of this variation into a command. Actually, SSSEP-based BCIs may benefit from straightforward analysis techniques of EEG signals, like reactive BCIs, while allowing self-paced interaction, like active BCIs. In this paper, we present a survey of scientific literature related to EEG-based BCI exploiting SSSEP. Firstly, we endeavour to describe the main characteristics of SSSEPs and the calibration techniques that allow the tuning of stimulation in order to maximise their amplitude. Secondly, we present the signal processing and data classification algorithms implemented by authors in order to elaborate commands in their SSSEP-based BCIs, as well as the classification performance that they evaluated on user experiments.
Collapse
Affiliation(s)
- Jimmy Petit
- University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| | - José Rouillard
- University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| | - François Cabestaing
- University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| |
Collapse
|
2
|
Eidel M, Kübler A. Wheelchair Control in a Virtual Environment by Healthy Participants Using a P300-BCI Based on Tactile Stimulation: Training Effects and Usability. Front Hum Neurosci 2020; 14:265. [PMID: 32754019 PMCID: PMC7366506 DOI: 10.3389/fnhum.2020.00265] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2020] [Accepted: 06/15/2020] [Indexed: 11/13/2022] Open
Abstract
Tactile stimulation is less frequently used than visual for brain-computer interface (BCI) control, partly because of limitations in speed and accuracy. Non-visual BCI paradigms, however, may be required for patients who struggle with vision dependent BCIs because of a loss of gaze control. With the present study, we attempted to replicate earlier results by Herweg et al. (2016), with several minor adjustments and a focus on training effects and usability. We invited 16 healthy participants and trained them with a 4-class tactile P300-based BCI in five sessions. Their main task was to navigate a virtual wheelchair through a 3D apartment using the BCI. We found significant training effects on information transfer rate (ITR), which increased from a mean of 3.10–9.50 bits/min. Further, both online and offline accuracies significantly increased with training from 65% to 86% and 70% to 95%, respectively. We found only a descriptive increase of P300 amplitudes at Fz and Cz with training. Furthermore, we report subjective data from questionnaires, which indicated a relatively high workload and moderate to high satisfaction. Although our participants have not achieved the same high performance as in the Herweg et al. (2016) study, we provide evidence for training effects on performance with a tactile BCI and confirm the feasibility of the paradigm.
Collapse
Affiliation(s)
- Matthias Eidel
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - Andrea Kübler
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| |
Collapse
|
3
|
Fleury M, Lioi G, Barillot C, Lécuyer A. A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback. Front Neurosci 2020; 14:528. [PMID: 32655347 PMCID: PMC7325479 DOI: 10.3389/fnins.2020.00528] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2020] [Accepted: 04/28/2020] [Indexed: 11/23/2022] Open
Abstract
Neurofeedback (NF) and brain-computer interface (BCI) applications rely on the registration and real-time feedback of individual patterns of brain activity with the aim of achieving self-regulation of specific neural substrates or control of external devices. These approaches have historically employed visual stimuli. However, in some cases vision is unsuitable or inadequately engaging. Other sensory modalities, such as auditory or haptic feedback have been explored, and multisensory stimulation is expected to improve the quality of the interaction loop. Moreover, for motor imagery tasks, closing the sensorimotor loop through haptic feedback may be relevant for motor rehabilitation applications, as it can promote plasticity mechanisms. This survey reviews the various haptic technologies and describes their application to BCIs and NF. We identify major trends in the use of haptic interfaces for BCI and NF systems and discuss crucial aspects that could motivate further studies.
Collapse
Affiliation(s)
- Mathis Fleury
- University of Rennes 1, INRIA, EMPENN & HYBRID, Rennes, France
| | | | | | | |
Collapse
|
4
|
Sadeghi S, Maleki A. Recent Advances in Hybrid Brain-Computer Interface Systems: A Technological and Quantitative Review. Basic Clin Neurosci 2019; 9:373-388. [PMID: 30719252 PMCID: PMC6360492 DOI: 10.32598/bcn.9.5.373] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2017] [Revised: 07/10/2017] [Accepted: 05/29/2018] [Indexed: 12/03/2022] Open
Abstract
Brain-Computer Interface (BCI) is a system that enables users to transmit commands to the computer using their brain activity recorded by electroencephalography. In a Hybrid Brain-Computer Interface (HBCI), a BCI control signal combines with one or more BCI control signals or with Human-Machine Interface (HMI) biosignals to increase classification accuracy, boost system speed, and improve user’s satisfaction. HBCI systems are categorized according to the type of combined signals and the combination technique (simultaneous or sequential). They have been used in several applications such as cursor control, target selection, and spellers. Increasing the number of articles published in this field indicates the significance of these systems. In this paper, different HBCI combinations, their important features, and potential applications are discussed. In most cases, the combination of a BCI control signal with a HMI biosignal yields higher information transfer rate than two BCI control signals.
Collapse
Affiliation(s)
- Sahar Sadeghi
- Department of Biomedical Engineering, Faculty of New Sciences and Technologies, Semnan University, Semnan, Iran
| | - Ali Maleki
- Department of Biomedical Engineering, Faculty of New Sciences and Technologies, Semnan University, Semnan, Iran
| |
Collapse
|
5
|
Hori J, Akagawa R. Improvement in Classification of Tactile Event-Related Potentials using Random-Interval Tasks ∗. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2018:1968-1971. [PMID: 30440784 DOI: 10.1109/embc.2018.8512670] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
The classification accuracy of an event-related potential (ERP) induced by a tactile stimulus is investigated to create a brain-computer interface (BCI). Mechanical tactile stimuli were applied to the left and right index fingers using two piezo actuator probes that were used as a Braille display for the visually impaired. In an experiment, two-class classification was investigated using three kinds of tactile stimulus pattern. The subjects were instructed to pay attention to unusual target stimuli while avoiding other frequent nontarget stimuli. The extracted features were classified using stepwise linear discriminant analysis. As a result, high accuracy was obtained by the task of random intervals compared with the task of constant intervals. It was suggested that the accuracy of the BCI using tactile stimuli is influenced by the concentration on the task.
Collapse
|
6
|
Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2017; 2017:2107451. [PMID: 29056963 PMCID: PMC5625811 DOI: 10.1155/2017/2107451] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2017] [Revised: 07/24/2017] [Accepted: 08/14/2017] [Indexed: 11/24/2022]
Abstract
This paper proposes two multimodal fusion methods between brain and peripheral signals for emotion recognition. The input signals are electroencephalogram and facial expression. The stimuli are based on a subset of movie clips that correspond to four specific areas of valance-arousal emotional space (happiness, neutral, sadness, and fear). For facial expression detection, four basic emotion states (happiness, neutral, sadness, and fear) are detected by a neural network classifier. For EEG detection, four basic emotion states and three emotion intensity levels (strong, ordinary, and weak) are detected by two support vector machines (SVM) classifiers, respectively. Emotion recognition is based on two decision-level fusion methods of both EEG and facial expression detections by using a sum rule or a production rule. Twenty healthy subjects attended two experiments. The results show that the accuracies of two multimodal fusion detections are 81.25% and 82.75%, respectively, which are both higher than that of facial expression (74.38%) or EEG detection (66.88%). The combination of facial expressions and EEG information for emotion recognition compensates for their defects as single information sources.
Collapse
|
7
|
Trends in BCI Research I: Brain-Computer Interfaces for Assessment of Patients with Locked-in Syndrome or Disorders of Consciousness. ACTA ACUST UNITED AC 2017. [DOI: 10.1007/978-3-319-64373-1_11] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register]
|
8
|
Choi I, Rhiu I, Lee Y, Yun MH, Nam CS. A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives. PLoS One 2017; 12:e0176674. [PMID: 28453547 PMCID: PMC5409179 DOI: 10.1371/journal.pone.0176674] [Citation(s) in RCA: 46] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
A new Brain-Computer Interface (BCI) technique, which is called a hybrid BCI, has recently been proposed to address the limitations of conventional single BCI system. Although some hybrid BCI studies have shown promising results, the field of hybrid BCI is still in its infancy and there is much to be done. Especially, since the hybrid BCI systems are so complicated and complex, it is difficult to understand the constituent and role of a hybrid BCI system at a glance. Also, the complicated and complex systems make it difficult to evaluate the usability of the systems. We systematically reviewed and analyzed the current state-of-the-art hybrid BCI studies, and proposed a systematic taxonomy for classifying the types of hybrid BCIs with multiple taxonomic criteria. After reviewing 74 journal articles, hybrid BCIs could be categorized with respect to 1) the source of brain signals, 2) the characteristics of the brain signal, and 3) the characteristics of operation in each system. In addition, we exhaustively reviewed recent literature on usability of BCIs. To identify the key evaluation dimensions of usability, we focused on task and measurement characteristics of BCI usability. We classified and summarized 31 BCI usability journal articles according to task characteristics (type and description of task) and measurement characteristics (subjective and objective measures). Afterwards, we proposed usability dimensions for BCI and hybrid BCI systems according to three core-constructs: Satisfaction, effectiveness, and efficiency with recommendations for further research. This paper can help BCI researchers, even those who are new to the field, can easily understand the complex structure of the hybrid systems at a glance. Recommendations for future research can also be helpful in establishing research directions and gaining insight in how to solve ergonomics and HCI design issues surrounding BCI and hybrid BCI systems by usability evaluation.
Collapse
Affiliation(s)
- Inchul Choi
- Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, North Carolina, United States of America
| | - Ilsun Rhiu
- Division of Global Management Engineering, Hoseo University, Asan, Korea
| | - Yushin Lee
- Department of Industrial Engineering, Seoul National University, Seoul, Korea
| | - Myung Hwan Yun
- Department of Industrial Engineering, Seoul National University, Seoul, Korea
| | - Chang S. Nam
- Edward P. Fitts Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, North Carolina, United States of America
- * E-mail:
| |
Collapse
|
9
|
Vlaar MP, Solis-Escalante T, Dewald JPA, van Wegen EEH, Schouten AC, Kwakkel G, van der Helm FCT. Quantification of task-dependent cortical activation evoked by robotic continuous wrist joint manipulation in chronic hemiparetic stroke. J Neuroeng Rehabil 2017; 14:30. [PMID: 28412953 PMCID: PMC5393035 DOI: 10.1186/s12984-017-0240-3] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2016] [Accepted: 03/30/2017] [Indexed: 01/05/2023] Open
Abstract
Background Cortical damage after stroke can drastically impair sensory and motor function of the upper limb, affecting the execution of activities of daily living and quality of life. Motor impairment after stroke has been thoroughly studied, however sensory impairment and its relation to movement control has received less attention. Integrity of the somatosensory system is essential for feedback control of human movement, and compromised integrity due to stroke has been linked to sensory impairment. Methods The goal of this study is to assess the integrity of the somatosensory system in individuals with chronic hemiparetic stroke with different levels of sensory impairment, through a combination of robotic joint manipulation and high-density electroencephalogram (EEG). A robotic wrist manipulator applied continuous periodic disturbances to the affected limb, providing somatosensory (proprioceptive and tactile) stimulation while challenging task execution. The integrity of the somatosensory system was evaluated during passive and active tasks, defined as ‘relaxed wrist’ and ‘maintaining 20% maximum wrist flexion’, respectively. The evoked cortical responses in the EEG were quantified using the power in the averaged responses and their signal-to-noise ratio. Results Thirty individuals with chronic hemiparetic stroke and ten unimpaired individuals without stroke participated in this study. Participants with stroke were classified as having severe, mild, or no sensory impairment, based on the Erasmus modification of the Nottingham Sensory Assessment. Under passive conditions, wrist manipulation resulted in contralateral cortical responses in unimpaired and chronic stroke participants with mild and no sensory impairment. In participants with severe sensory impairment the cortical responses were strongly reduced in amplitude, which related to anatomical damage. Under active conditions, participants with mild sensory impairment showed reduced responses compared to the passive condition, whereas unimpaired and chronic stroke participants without sensory impairment did not show this reduction. Conclusions Robotic continuous joint manipulation allows studying somatosensory cortical evoked responses during the execution of meaningful upper limb control tasks. Using such an approach it is possible to quantitatively assess the integrity of sensory pathways; in the context of movement control this provides additional information required to develop more effective neurorehabilitation therapies.
Collapse
Affiliation(s)
- Martijn P Vlaar
- BioMechanical Engineering Department, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands.
| | - Teodoro Solis-Escalante
- BioMechanical Engineering Department, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands
| | - Julius P A Dewald
- BioMechanical Engineering Department, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands.,Department of Physical Therapy and Human Movement Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.,Department of Biomedical Engineering, McCormick School of School of Engineering, Northwestern University, Evanston, IL, USA.,MIRA Institute for Biomedical Technology and Technical Medicine, Laboratory of BioMechanical Engineering, University of Twente, Enschede, The Netherlands
| | - Erwin E H van Wegen
- VU University Medical Centre, Amsterdam Neurosciences, Amsterdam, The Netherlands.,MOVE Research Institute Amsterdam, Amsterdam, The Netherlands
| | - Alfred C Schouten
- BioMechanical Engineering Department, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands.,Department of Physical Therapy and Human Movement Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.,MIRA Institute for Biomedical Technology and Technical Medicine, Laboratory of BioMechanical Engineering, University of Twente, Enschede, The Netherlands
| | - Gert Kwakkel
- VU University Medical Centre, Amsterdam Neurosciences, Amsterdam, The Netherlands.,MOVE Research Institute Amsterdam, Amsterdam, The Netherlands
| | - Frans C T van der Helm
- BioMechanical Engineering Department, Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands.,Department of Physical Therapy and Human Movement Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | | |
Collapse
|
10
|
Hori J, Okada N. Classification of tactile event-related potential elicited by Braille display for brain–computer interface. Biocybern Biomed Eng 2017. [DOI: 10.1016/j.bbe.2016.10.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
11
|
Wheelchair control by elderly participants in a virtual environment with a brain-computer interface (BCI) and tactile stimulation. Biol Psychol 2016; 121:117-124. [DOI: 10.1016/j.biopsycho.2016.10.006] [Citation(s) in RCA: 51] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2016] [Revised: 10/03/2016] [Accepted: 10/10/2016] [Indexed: 11/19/2022]
|
12
|
Breitwieser C, Pokorny C, Müller-Putz GR. A hybrid three-class brain-computer interface system utilizing SSSEPs and transient ERPs. J Neural Eng 2016; 13:066015. [PMID: 27788124 DOI: 10.1088/1741-2560/13/6/066015] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE This paper investigates the fusion of steady-state somatosensory evoked potentials (SSSEPs) and transient event-related potentials (tERPs), evoked through tactile simulation on the left and right-hand fingertips, in a three-class EEG based hybrid brain-computer interface. It was hypothesized, that fusing the input signals leads to higher classification rates than classifying tERP and SSSEP individually. APPROACH Fourteen subjects participated in the studies, consisting of a screening paradigm to determine person dependent resonance-like frequencies and a subsequent online paradigm. The whole setup of the BCI system was based on open interfaces, following suggestions for a common implementation platform. During the online experiment, subjects were instructed to focus their attention on the stimulated fingertips as indicated by a visual cue. The recorded data were classified during runtime using a multi-class shrinkage LDA classifier and the outputs were fused together applying a posterior probability based fusion. Data were further analyzed offline, involving a combined classification of SSSEP and tERP features as a second fusion principle. The final results were tested for statistical significance applying a repeated measures ANOVA. MAIN RESULTS A significant classification increase was achieved when fusing the results with a combined classification compared to performing an individual classification. Furthermore, the SSSEP classifier was significantly better in detecting a non-control state, whereas the tERP classifier was significantly better in detecting control states. Subjects who had a higher relative band power increase during the screening session also achieved significantly higher classification results than subjects with lower relative band power increase. SIGNIFICANCE It could be shown that utilizing SSSEP and tERP for hBCIs increases the classification accuracy and also that tERP and SSSEP are not classifying control- and non-control states with the same level of accuracy.
Collapse
|
13
|
Kim KT, Suk HI, Lee SW. Commanding a Brain-Controlled Wheelchair Using Steady-State Somatosensory Evoked Potentials. IEEE Trans Neural Syst Rehabil Eng 2016; 26:654-665. [PMID: 27514060 DOI: 10.1109/tnsre.2016.2597854] [Citation(s) in RCA: 42] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
In this work, we propose a novel brain-controlled wheelchair, one of the major applications of brain-machine interfaces (BMIs), that allows an individual with mobility impairments to perform daily living activities independently. Specifically, we propose to use a steady-state somatosensory evoked potential (SSSEP) paradigm, which elicits brain responses to tactile stimulation of specific frequencies, for a user's intention to control a wheelchair. In our system, a user had three possible commands by concentrating on one of three vibration stimuli, which were attached to the left-hand, right-hand, and right-foot, to selectively control the wheelchair. The three stimuli were associated with three wheelchair commands: turn-left, turn-right, and move-forward. From a machine learning perspective, we also devise a novel feature representation by combining spatial and spectral characteristics of brain signals. In order to validate the effectiveness of the proposed SSSEP-based system, we considered two different tasks: 1) a simple obstacle-avoidance task within a limited time and; 2) a driving task along the predefined trajectory of about 40 m length, where there were a narrow pathway, a door, and obstacles. In both experiments, we recruited 12 subjects and compared the average time of motor imagery (MI) and SSSEP-based controls to complete the task. With the SSSEP-based control, all subjects successfully completed the task without making any collision while four subjects failed it with MI-based control. It is also noteworthy that in terms of the average time to complete the task, the SSSEP-based control outperformed the MI-based control. In the other more challenging task, all subjects successfully reached the target location.
Collapse
|
14
|
Pokorny C, Breitwieser C, Müller-Putz GR. The Role of Transient Target Stimuli in a Steady-State Somatosensory Evoked Potential-Based Brain-Computer Interface Setup. Front Neurosci 2016; 10:152. [PMID: 27092051 PMCID: PMC4823306 DOI: 10.3389/fnins.2016.00152] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2015] [Accepted: 03/21/2016] [Indexed: 11/30/2022] Open
Abstract
In earlier literature, so-called twitches were used to support a user in a steady-state somatosensory evoked potential (SSSEP) based brain–computer interface (BCI) to focus attention on the requested targets. Within this work, we investigate the impact of these transient target stimuli on SSSEPs in a real-life BCI setup. A hybrid BCI was designed which combines SSSEPs and P300 potentials evoked by twitches randomly embedded into the streams of tactile stimuli. The EEG of fourteen healthy subjects was recorded, while their left and right index fingers were simultaneously stimulated using frequencies selected in a screening procedure. The subjects were randomly instructed by a cue on a screen to focus attention on one or none of the fingers. Feature for SSSEPs and P300 potentials were extracted and classified using separately trained multi-class shrinkage LDA classifiers. Three-class classification accuracies significantly better than random could be reached by nine subjects using SSSEP features and by 12 subjects using P300 features respectively. The average classification accuracies were 48.6% using SSSEP and 50.7% using P300 features. By means of a Monte Carlo permutation test it could be shown that twitches have an attenuation effect on the SSSEP. Significant SSSEP blocking effects time-locked to twitch positions were found in seven subjects. Our findings suggest that the attempt to combine different types of stimulation signals like repetitive signals and twitches has a mutual influence on each other, which may be the main reason for the rather moderate BCI performance. This influence is originated at the level of stimulus generation but becomes apparent as physiological effect in the SSSEP. When designing a hybrid BCI based on SSSEPs and P300 potentials, one has to find an optimal tradeoff depending on the overall design goals or individual subjects' performance. Our results give therefore some new insights that may be useful for the successful design of hybrid BCIs.
Collapse
Affiliation(s)
- Christoph Pokorny
- Laboratory of Brain-Computer Interfaces, Institute of Neural Engineering, Graz University of TechnologyGraz, Austria; Institute for Theoretical Computer Science, Graz University of TechnologyGraz, Austria
| | - Christian Breitwieser
- Laboratory of Brain-Computer Interfaces, Institute of Neural Engineering, Graz University of Technology Graz, Austria
| | - Gernot R Müller-Putz
- Laboratory of Brain-Computer Interfaces, Institute of Neural Engineering, Graz University of Technology Graz, Austria
| |
Collapse
|
15
|
Connections between intraparietal sulcus and a sensorimotor network underpin sustained tactile attention. J Neurosci 2015; 35:7938-49. [PMID: 25995478 DOI: 10.1523/jneurosci.3421-14.2015] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
Previous studies on sustained tactile attention draw conclusions about underlying cortical networks by averaging over experimental conditions without considering attentional variance in single trials. This may have formed an imprecise picture of brain processes underpinning sustained tactile attention. In the present study, we simultaneously recorded EEG-fMRI and used modulations of steady-state somatosensory evoked potentials (SSSEPs) as a measure of attentional trial-by-trial variability. Therefore, frequency-tagged streams of vibrotactile stimulations were simultaneously presented to both index fingers. Human participants were cued to sustain attention to either the left or right finger stimulation and to press a button whenever they perceived a target pulse embedded in the to-be-attended stream. In-line with previous studies, a classical general linear model (GLM) analysis based on cued attention conditions revealed increased activity mainly in somatosensory and cerebellar regions. Yet, parametric modeling of the BOLD response using simultaneously recorded SSSEPs as a marker of attentional trial-by-trial variability quarried the intraparietal sulcus (IPS). The IPS in turn showed enhanced functional connectivity to a modality-unspecific attention network. However, this was only revealed on the basis of cued attention conditions in the classical GLM. By considering attentional variability as captured by SSSEPs, the IPS showed increased connectivity to a sensorimotor network, underpinning attentional selection processes between competing tactile stimuli and action choices (press a button or not). Thus, the current findings highlight the potential value by considering attentional variations in single trials and extend previous knowledge on the role of the IPS in tactile attention.
Collapse
|
16
|
Rutkowski TM, Mori H. Tactile and bone-conduction auditory brain computer interface for vision and hearing impaired users. J Neurosci Methods 2015; 244:45-51. [PMID: 24768575 DOI: 10.1016/j.jneumeth.2014.04.010] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2014] [Revised: 04/07/2014] [Accepted: 04/10/2014] [Indexed: 11/15/2022]
Affiliation(s)
- Tomasz M Rutkowski
- Life Science Center of TARA, University of Tsukuba, Japan; RIKEN Brain Science Institute, Japan.
| | - Hiromu Mori
- Life Science Center of TARA, University of Tsukuba, Japan.
| |
Collapse
|
17
|
Severens M, Van der Waal M, Farquhar J, Desain P. Comparing tactile and visual gaze-independent brain–computer interfaces in patients with amyotrophic lateral sclerosis and healthy users. Clin Neurophysiol 2014; 125:2297-2304. [DOI: 10.1016/j.clinph.2014.03.005] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2013] [Revised: 02/24/2014] [Accepted: 03/04/2014] [Indexed: 11/29/2022]
|
18
|
Hogervorst MA, Brouwer AM, van Erp JBF. Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Front Neurosci 2014; 8:322. [PMID: 25352774 PMCID: PMC4196537 DOI: 10.3389/fnins.2014.00322] [Citation(s) in RCA: 98] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2014] [Accepted: 09/25/2014] [Indexed: 11/30/2022] Open
Abstract
While studies exist that compare different physiological variables with respect to their association with mental workload, it is still largely unclear which variables supply the best information about momentary workload of an individual and what is the benefit of combining them. We investigated workload using the n-back task, controlling for body movements and visual input. We recorded EEG, skin conductance, respiration, ECG, pupil size and eye blinks of 14 subjects. Various variables were extracted from these recordings and used as features in individually tuned classification models. Online classification was simulated by using the first part of the data as training set and the last part of the data for testing the models. The results indicate that EEG performs best, followed by eye related measures and peripheral physiology. Combining variables from different sensors did not significantly improve workload assessment over the best performing sensor alone. Best classification accuracy, a little over 90%, was reached for distinguishing between high and low workload on the basis of 2 min segments of EEG and eye related variables. A similar and not significantly different performance of 86% was reached using only EEG from single electrode location Pz.
Collapse
Affiliation(s)
- Maarten A Hogervorst
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research Soesterberg, Netherlands
| | - Anne-Marie Brouwer
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research Soesterberg, Netherlands
| | - Jan B F van Erp
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research Soesterberg, Netherlands
| |
Collapse
|
19
|
Z. Allison B, Jin J, Zhang Y, Wang X. A four-choice hybrid P300/SSVEP BCI for improved accuracy. BRAIN-COMPUTER INTERFACES 2014. [DOI: 10.1080/2326263x.2013.869003] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|