Clinical feasibility of brain-computer interface based on steady-state visual evoked potential in patients with locked-in syndrome: Case studies

EEG Dataset for the Recognition of Different Emotions Induced in Voice-User Interaction

Electroencephalography (EEG)-based open-access datasets are available for emotion recognition studies, where external auditory/visual stimuli are used to artificially evoke pre-defined emotions. In this study, we provide a novel EEG dataset containing the emotional information induced during a realistic human-computer interaction (HCI) using a voice user interface system that mimics natural human-to-human communication. To validate our dataset via neurophysiological investigation and binary emotion classification, we applied a series of signal processing and machine learning methods to the EEG data. The maximum classification accuracy ranged from 43.3% to 90.8% over 38 subjects and classification features could be interpreted neurophysiologically. Our EEG data could be used to develop a reliable HCI system because they were acquired in a natural HCI environment. In addition, auxiliary physiological data measured simultaneously with the EEG data also showed plausible results, i.e., electrocardiogram, photoplethysmogram, galvanic skin response, and facial images, which could be utilized for automatic emotion discrimination independently from, as well as together with the EEG data via the fusion of multi-modal physiological datasets.

A review on the performance of brain-computer interface systems used for patients with locked-in and completely locked-in syndrome

Patients with locked-in syndrome (LIS) and complete locked-in syndrome (CLIS) own a fully functional brain restricted within a non-functional body. In order to help LIS patients stay connected with their surroundings, brain-computer interfaces (BCIs) and related technologies have emerged. BCIs translate brain activity into actions that can be performed by external devices enabling LIS patients to communicate, leading to an increase in their quality of life. The past decade has seen the rapid development of BCIs that have the potential to be used for patients with locked-in syndrome, from which a great deal is tested only on healthy subjects and not on actual patients. This study aims to (1) provide the readers with a comprehensive study that contributes to this growing area of research by exploring the performance of BCIs tested specifically on LIS and CLIS patients, (2) give an overview of different modalities and paradigms used in different stages of the locked-in syndrome, and (3) discuss the contributions and limitations of BCIs introduced for the LIS and CLIS patients in the state-of-the-art and lay a groundwork for researchers interested in this field.

Locked-in syndrome revisited

The locked-in syndrome (LiS) is characterized by quadriplegia with preserved vertical eye and eyelid movements and retained cognitive abilities. Subcategorization, aetiologies and the anatomical foundation of LiS are discussed. The damage of different structures in the pons, mesencephalon and thalamus are attributed to symptoms of classical, complete and incomplete LiS and the locked-in plus syndrome, which is characterized by additional impairments of consciousness, making the clinical distinction to other chronic disorders of consciousness at times difficult. Other differential diagnoses are cognitive motor dissociation (CMD) and akinetic mutism. Treatment options are reviewed and an early, interdisciplinary and aggressive approach, including the provision of psychological support and coping strategies is favoured. The establishment of communication is a main goal of rehabilitation. Finally, the quality of life of LiS patients and ethical implications are considered. While patients with LiS report a high quality of life and well-being, medical professionals and caregivers have largely pessimistic perceptions. The negative view on life with LiS must be overthought and the autonomy and dignity of LiS patients prioritized. Knowledge has to be disseminated, diagnostics accelerated and technical support system development promoted. More well-designed research but also more awareness of the needs of LiS patients and their perception as individual persons is needed to enable a life with LiS that is worth living.

eldBETA: A Large Eldercare-oriented Benchmark Database of SSVEP-BCI for the Aging Population

Global population aging poses an unprecedented challenge and calls for a rising effort in eldercare and healthcare. Steady-state visual evoked potential based brain-computer interface (SSVEP-BCI) boasts its high transfer rate and shows great promise in real-world applications to support aging. Public database is critically important for designing the SSVEP-BCI systems. However, the SSVEP-BCI database tailored for the elder is scarce in existing studies. Therefore, in this study, we present a large eldercare-oriented BEnchmark database of SSVEP-BCI for The Aging population (eldBETA). The eldBETA database consisted of the 64-channel electroencephalogram (EEG) from 100 elder participants, each of whom performed seven blocks of 9-target SSVEP-BCI task. The quality and characteristics of the eldBETA database were validated by a series of analyses followed by a classification analysis of thirteen frequency recognition methods. We expect that the eldBETA database would provide a substrate for the design and optimization of the BCI systems intended for the elders. The eldBETA database is open-access for research and can be downloaded from the website 10.6084/m9.figshare.18032669.

Measurement(s)	Steady-state visual evoked potential (SSVEP)
Technology Type(s)	Electroencephalography (EEG)
Factor Type(s)	Elder population
Sample Characteristic - Organism	Homo sapiens
Sample Characteristic - Environment	Electromagnetic shielding room

Influence of the Number of Channels and Classification Algorithm on the Performance Robustness to Electrode Shift in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces

There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.

Literature review on assistive devices available for quadriplegic people: Indian context

PURPOSE

This literature review aims to find the current state of the art in self-help devices (SHD) available for people with quadriplegia.

MATERIALS AND METHODS

We searched original articles, technical and case studies, conference articles, and literature reviews published between 2014 to 2019 with the keywords ("Self-help devices" OR "Assistive Devices" OR "Assistive Product" OR "Assistive Technology") AND "Quadriplegia" in Science Direct, Pubmed, IEEE Xplore digital library and Web of Science.

RESULTS

Total 222 articles were found. After removing duplicates and screening these articles based on their title and abstracts 80 articles remained. After this, we reviewed the full text, and articles unrelated to SHD development or about the patients who require mechanical ventilation or where the upper limb is functional (C2 or above and T2 or below injuries) were discarded. After the exclusion of articles using the above-mentioned criterion 75 articles were used for further review.

CONCLUSION

The abandonment rate of SHD currently available in the literature is very high. The major requirement of the people was independence and improved quality of life. The situation in India is very bad as compared to the developed countries. The people with spinal cord injury in India are uneducated and very poor, with an average income of 3000 ₹ (41$). They require SHDs and training specially designed for them, keeping their needs in mind.Implications for rehabilitationPeople with quadriplegia are totally dependent on caregivers. Assistive devices not only help these people to do day-to-day tasks but also provides them self-confidence.Even though there are a lot of self-help devices currently available, still they are not able to fulfil the requirements of people with quadriplegia, hence there is a very high abandonment rate of such devices.This study provides an evidence that developing devices after understanding the functional and non-functional requirements of these subjects will decrease the abandonment rate and increase the effectiveness of the device.The results of this study can be used for planning and developing assistive devices which are more focussed on fulfilling the requirements of people with quadriplegia.

High-gamma oscillations precede visual steady-state responses: A human electrocorticography study

The robust steady-state cortical activation elicited by flickering visual stimulation has been exploited by a wide range of scientific studies. As the fundamental neural response inherits the spectral properties of the gazed flickering, the paradigm has been used to chart cortical characteristics and their relation to pathologies. However, despite its widespread adoption, the underlying neural mechanisms are not well understood. Here, we show that the fundamental response is preceded by high-gamma (55-125 Hz) oscillations which are also synchronised to the gazed frequency. Using a subdural recording of the primary and associative visual cortices of one human subject, we demonstrate that the latencies of the high-gamma and fundamental components are highly correlated on a single-trial basis albeit that the latter is consistently delayed by approximately 55 ms. These results corroborate previous reports that top-down feedback projections are involved in the generation of the fundamental response, but, in addition, we show that trial-to-trial variability in fundamental latency is paralleled by a highly similar variability in high-gamma latency. Pathology- or paradigm-induced alterations in steady-state responses could thus originate either from deviating visual gamma responses or from aberrations in the neural feedback mechanism. Experiments designed to tease apart the two processes are expected to provide deeper insights into the studied paradigm.

SSVEP BCI and Eye Tracking Use by Individuals With Late-Stage ALS and Visual Impairments

Access to communication is critical for individuals with late-stage amyotrophic lateral sclerosis (ALS) and minimal volitional movement, but they sometimes present with concomitant visual or ocular motility impairments that affect their performance with eye tracking or visual brain-computer interface (BCI) systems. In this study, we explored the use of modified eye tracking and steady state visual evoked potential (SSVEP) BCI, in combination with the Shuffle Speller typing interface, for this population. Two participants with late-stage ALS, visual impairments, and minimal volitional movement completed a single-case experimental research design comparing copy-spelling performance with three different typing systems: (1) commercially available eye tracking communication software, (2) Shuffle Speller with modified eye tracking, and (3) Shuffle Speller with SSVEP BCI. Participant 1 was unable to type any correct characters with the commercial system, but achieved accuracies of up to 50% with Shuffle Speller eye tracking and 89% with Shuffle Speller BCI. Participant 2 also had higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with eye tracking and 100% accuracy with BCI. However, participants' typing accuracy for both Shuffle Speller conditions was highly variable, particularly in the BCI condition. Both the Shuffle Speller interface and SSVEP BCI input show promise for improving typing performance for people with late-stage ALS. Further development of innovative BCI systems for this population is needed.

Single stimulus location for two inputs: A combined brain-computer interface based on Steady-State Visual Evoked Potential (SSVEP)

Brain-computer interfaces (BCI) help severely paralyzed people communicate with the outside world. One type of BCI depends on eye movements and has high information transfer (ITR) but is tiring for users and not applicable to people with eye dyskinesia. Conversely, independent BCIs enable attention shifts across visual stimuli without eye movement, but at the cost of a lower ITR. Steady-state visual evoked potential (SSVEP) is an oscillatory brain response and typically used as BCI signal sources because of high signal-to-noise ratio (SNR). Considering the effect of attentional modulation on the SSVEP, we proposed the novel concept of one-to-two BCI to optimize existing problems, wherein the target and other stimuli shared the same location. Specifically, two spatially overlapping stimuli were displayed in the center-of-view field, as in the independent BCI, and participants were required to divide their attention between the right and left visual fields, as in the dependent BCI. Using three different design schemes in two experiments, we aimed to provide a new framework for BCI design by exploring the feasibility of a combined BCI that can realize a single stimulus location for two inputs. The results strongly demonstrated that, even when the targets and distractors overlapped spatially, the former evoked stronger SSVEP responses. Notably, the BCI scheme based on the object-based attention could achieve a recognition rate as high as 83.2% and an ITR of 12.5 bits per minute. The feasibility of a one-to-two BCI design, which simplified the keyboard layout, reduced the attention shift, and relieved user fatigue, was established.

Development of a Brain-Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography

Asynchronous brain-computer interfaces (BCIs) based on electroencephalography (EEG) generally suffer from poor performance in terms of classification accuracy and false-positive rate (FPR). Thus, BCI toggle switches based on electrooculogram (EOG) signals were developed to toggle on/off synchronous BCI systems. The conventional BCI toggle switches exhibit fast responses with high accuracy; however, they have a high FPR or cannot be applied to patients with oculomotor impairments. To circumvent these issues, we developed a novel BCI toggle switch that users can employ to toggle on or off synchronous BCIs by holding their breath for a few seconds. Two states-normal breath and breath holding-were classified using a linear discriminant analysis with features extracted from the respiration-modulated photoplethysmography (PPG) signals. A real-time BCI toggle switch was implemented with calibration data trained with only 1-min PPG data. We evaluated the performance of our PPG switch by combining it with a steady-state visual evoked potential-based BCI system that was designed to control four external devices, with regard to the true-positive rate and FPR. The parameters of the PPG switch were optimized through an offline experiment with five subjects, and the performance of the switch system was evaluated in an online experiment with seven subjects. All the participants successfully turned on the BCI by holding their breath for approximately 10 s (100% accuracy), and the switch system exhibited a very low FPR of 0.02 false operations per minute, which is the lowest FPR reported thus far. All participants could successfully control external devices in the synchronous BCI mode. Our results demonstrated that the proposed PPG-based BCI toggle switch can be used to implement practical BCIs.

Brain-computer interfaces for communication

Locked-in syndrome (LIS) is characterized by an inability to move or speak in the presence of intact cognition and can be caused by brainstem trauma or neuromuscular disease. Quality of life (QoL) in LIS is strongly impaired by the inability to communicate, which cannot always be remedied by traditional augmentative and alternative communication (AAC) solutions if residual muscle activity is insufficient to control the AAC device. Brain-computer interfaces (BCIs) may offer a solution by employing the person's neural signals instead of relying on muscle activity. Here, we review the latest communication BCI research using noninvasive signal acquisition approaches (electroencephalography, functional magnetic resonance imaging, functional near-infrared spectroscopy) and subdural and intracortical implanted electrodes, and we discuss current efforts to translate research knowledge into usable BCI-enabled communication solutions that aim to improve the QoL of individuals with LIS.

Steady-state visual evoked potential (SSVEP)-based brain–computer interface (BCI) of Chinese speller for a patient with amyotrophic lateral sclerosis: A case report

This study applied a steady-state visual evoked potential (SSVEP) based brain–computer interface (BCI) to a patient in lock-in state with amyotrophic lateral sclerosis (ALS) and validated its feasibility for communication. The developed calibration-free and asynchronous spelling system provided a natural and efficient communication experience for the patient, achieving a maximum free-spelling accuracy above 90% and an information transfer rate of over 22.203 bits/min. A set of standard frequency scanning and task spelling data were also acquired to evaluate the patient’s SSVEP response and to facilitate further personalized BCI design. The results demonstrated that the proposed SSVEP-based BCI system was practical and efficient enough to provide daily life communication for ALS patients.

Behind the Scenes of Noninvasive Brain-Computer Interfaces: A Review of Electroencephalography Signals, How They Are Recorded, and Why They Matter

Purpose

Brain-computer interface (BCI) techniques may provide computer access for individuals with severe physical impairments. However, the relatively hidden nature of BCI control obscures how BCI systems work behind the scenes, making it difficult to understand how electroencephalography (EEG) records the BCI related brain signals, what brain signals are recorded by EEG, and why these signals are targeted for BCI control. Furthermore, in the field of speech-language-hearing, signals targeted for BCI application have been of primary interest to clinicians and researchers in the area of augmentative and alternative communication (AAC). However, signals utilized for BCI control reflect sensory, cognitive and motor processes, which are of interest to a range of related disciplines including speech science.

Method

This tutorial was developed by a multidisciplinary team emphasizing primary and secondary BCI-AAC related signals of interest to speech-language-hearing.

Results

An overview of BCI-AAC related signals are provided discussing 1) how BCI signals are recorded via EEG, 2) what signals are targeted for non-invasive BCI control, including the P300, sensorimotor rhythms, steady state evoked potentials, contingent negative variation, and the N400, and 3) why these signals are targeted. During tutorial creation, attention was given to help support EEG and BCI understanding for those without an engineering background.

Conclusion

Tutorials highlighting how BCI-AAC signals are elicited and recorded can help increase interest and familiarity with EEG and BCI techniques and provide a framework for understanding key principles behind BCI-AAC design and implementation.

Long-term use of a neural prosthesis in progressive paralysis

Brain–computer interfaces (BCIs) enable communication with others and allow machines or computers to be controlled in the absence of motor activity. Clinical studies evaluating neural prostheses in amyotrophic lateral sclerosis (ALS) patients have been performed; however, to date, no study has reported that ALS patients who progressed from locked-in syndrome (LIS), which has very limited voluntary movement, to a completely locked-in state (CLIS), characterized by complete loss of voluntary movements, were able to continue controlling neural prostheses. To clarify this, we used a BCI system to evaluate three late-stage ALS patients over 27 months. We employed steady-state visual evoked brain potentials elicited by flickering green and blue light-emitting diodes to control the BCI system. All participants reliably controlled the system throughout the entire period (median accuracy: 83.3%). One patient who progressed to CLIS was able to continue operating the system with high accuracy. Furthermore, this patient successfully used the system to respond to yes/no questions. Thus, this CLIS patient was able to operate a neuroprosthetic device, suggesting that the BCI system confers advantages for patients with severe paralysis, including those exhibiting complete loss of muscle movement.

Effects of simulated visual acuity and ocular motility impairments on SSVEP brain-computer interface performance: An experiment with Shuffle Speller

Individuals with severe speech and physical impairments may have concomitant visual acuity impairments (VAI) or ocular motility impairments (OMI) impacting visual BCI use. We report on the use of the Shuffle Speller typing interface for an SSVEP BCI copy-spelling task under three conditions: simulated VAI, simulated OMI, and unimpaired vision. To mitigate the effect of visual impairments, we introduce a method that adaptively selects a user-specific trial length to maximize expected information transfer rate (ITR); expected ITR is shown to closely approximate the rate of correct letter selections. All participants could type under the unimpaired and simulated VAI conditions, with no significant differences in typing accuracy or speed. Most participants (31 of 37) could not type under the simulated OMI condition; some achieved high accuracy but with slower typing speeds. Reported workload and discomfort were low, and satisfaction high, under the unimpaired and simulated VAI conditions. Implications and future directions to examine effect of visual impairment on BCI use is discussed.

Eyes-closed hybrid brain-computer interface employing frontal brain activation

Brain-computer interfaces (BCIs) have been studied extensively in order to establish a non-muscular communication channel mainly for patients with impaired motor functions. However, many limitations remain for BCIs in clinical use. In this study, we propose a hybrid BCI that is based on only frontal brain areas and can be operated in an eyes-closed state for end users with impaired motor and declining visual functions. In our experiment, electroencephalography (EEG) and near-infrared spectroscopy (NIRS) were simultaneously measured while 12 participants performed mental arithmetic (MA) and remained relaxed (baseline state: BL). To evaluate the feasibility of the hybrid BCI, we classified MA- from BL-related brain activation. We then compared classification accuracies using two unimodal BCIs (EEG and NIRS) and the hybrid BCI in an offline mode. The classification accuracy of the hybrid BCI (83.9 ± 10.3%) was shown to be significantly higher than those of unimodal EEG-based (77.3 ± 15.9%) and NIRS-based BCI (75.9 ± 6.3%). The analytical results confirmed performance improvement with the hybrid BCI, particularly for only frontal brain areas. Our study shows that an eyes-closed hybrid BCI approach based on frontal areas could be applied to neurodegenerative patients who lost their motor functions, including oculomotor functions.

An emergency call system for patients in locked-in state using an SSVEP-based brain switch

Patients in a locked-in state (LIS) due to severe neurological disorders such as amyotrophic lateral sclerosis (ALS) require seamless emergency care by their caregivers or guardians. However, it is a difficult job for the guardians to continuously monitor the patients' state, especially when direct communication is not possible. In the present study, we developed an emergency call system for such patients using a steady-state visual evoked potential (SSVEP)-based brain switch. Although there have been previous studies to implement SSVEP-based brain switch system, they have not been applied to patients in LIS, and thus their clinical value has not been validated. In this study, we verified whether the SSVEP-based brain switch system can be practically used as an emergency call system for patients in LIS. The brain switch used for our system adopted a chromatic visual stimulus, which proved to be visually less stimulating than conventional checkerboard-type stimuli but could generate SSVEP responses strong enough to be used for brain-computer interface (BCI) applications. To verify the feasibility of our emergency call system, 14 healthy participants and 3 patients with severe ALS took part in online experiments. All three ALS patients successfully called their guardians to their bedsides in about 6.56 seconds. Furthermore, additional experiments with one of these patients demonstrated that our emergency call system maintains fairly good performance even up to 4 weeks after the first experiment without renewing initial calibration data. Our results suggest that our SSVEP-based emergency call system might be successfully used in practical scenarios.

A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives

A new Brain-Computer Interface (BCI) technique, which is called a hybrid BCI, has recently been proposed to address the limitations of conventional single BCI system. Although some hybrid BCI studies have shown promising results, the field of hybrid BCI is still in its infancy and there is much to be done. Especially, since the hybrid BCI systems are so complicated and complex, it is difficult to understand the constituent and role of a hybrid BCI system at a glance. Also, the complicated and complex systems make it difficult to evaluate the usability of the systems. We systematically reviewed and analyzed the current state-of-the-art hybrid BCI studies, and proposed a systematic taxonomy for classifying the types of hybrid BCIs with multiple taxonomic criteria. After reviewing 74 journal articles, hybrid BCIs could be categorized with respect to 1) the source of brain signals, 2) the characteristics of the brain signal, and 3) the characteristics of operation in each system. In addition, we exhaustively reviewed recent literature on usability of BCIs. To identify the key evaluation dimensions of usability, we focused on task and measurement characteristics of BCI usability. We classified and summarized 31 BCI usability journal articles according to task characteristics (type and description of task) and measurement characteristics (subjective and objective measures). Afterwards, we proposed usability dimensions for BCI and hybrid BCI systems according to three core-constructs: Satisfaction, effectiveness, and efficiency with recommendations for further research. This paper can help BCI researchers, even those who are new to the field, can easily understand the complex structure of the hybrid systems at a glance. Recommendations for future research can also be helpful in establishing research directions and gaining insight in how to solve ergonomics and HCI design issues surrounding BCI and hybrid BCI systems by usability evaluation.