Evaluating the Effect of Stimuli Color and Frequency on SSVEP

A Hybrid BCI Integrating EEG and Eye-Tracking for Assisting Clinical Communication in Patients With Disorders of Consciousness

Assessing communication abilities in patients with disorders of consciousness (DOCs) is challenging due to limitations in the behavioral scale. Electroencephalogram-based brain-computer interfaces (BCIs) and eye-tracking for detecting ocular changes can capture mental activities without requiring physical behaviors and thus may be a solution. This study proposes a hybrid BCI that integrates EEG and eye tracking to facilitate communication in patients with DOC. Specifically, the BCI presented a question and two randomly flashing answers (yes/no). The subjects were instructed to focus on an answer. A multimodal target recognition network (MTRN) is proposed to detect P300 potentials and eye-tracking responses (i.e., pupil constriction and gaze) and identify the target in real time. In the MTRN, the dual-stream feature extraction module with two independent multiscale convolutional neural networks extracts multiscale features from multimodal data. Then, the multimodal attention strategy adaptively extracts the most relevant information about the target from multimodal data. Finally, a prototype network is designed as a classifier to facilitate small-sample data classification. Ten healthy individuals, nine DOC patients and one LIS patient were included in this study. All healthy subjects achieved 100% accuracy. Five patients could communicate with our BCI, with 76.1±7.9% accuracy. Among them, two patients who were noncommunicative on the behavioral scale exhibited communication ability via our BCI. Additionally, we assessed the performance of unimodal BCIs and compared MTRNs with other methods. All the results suggested that our BCI can yield more sensitive outcomes than the CRS-R and can serve as a valuable communication tool.

The mechanism of human color vision and potential implanted devices for artificial color vision

Vision plays a major role in perceiving external stimuli and information in our daily lives. The neural mechanism of color vision is complicated, involving the co-ordinated functions of a variety of cells, such as retinal cells and lateral geniculate nucleus cells, as well as multiple levels of the visual cortex. In this work, we reviewed the history of experimental and theoretical studies on this issue, from the fundamental functions of the individual cells of the visual system to the coding in the transmission of neural signals and sophisticated brain processes at different levels. We discuss various hypotheses, models, and theories related to the color vision mechanism and present some suggestions for developing novel implanted devices that may help restore color vision in visually impaired people or introduce artificial color vision to those who need it.

Advancements in brain-computer interfaces for the rehabilitation of unilateral spatial neglect: a concise review

This short review examines recent advancements in neurotechnologies within the context of managing unilateral spatial neglect (USN), a common condition following stroke. Despite the success of brain-computer interfaces (BCIs) in restoring motor function, there is a notable absence of effective BCI devices for treating cerebral visual impairments, a prevalent consequence of brain lesions that significantly hinders rehabilitation. This review analyzes current non-invasive BCIs and technological solutions dedicated to cognitive rehabilitation, with a focus on visuo-attentional disorders. We emphasize the need for further research into the use of BCIs for managing cognitive impairments and propose a new potential solution for USN rehabilitation, by combining the clinical subtleties of this syndrome with the technological advancements made in the field of neurotechnologies.

A novel brain-controlled prosthetic hand method integrating AR-SSVEP augmentation, asynchronous control, and machine vision assistance

Background and objective

The brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) is expected to help disabled patients achieve alternative prosthetic hand assistance. However, the existing study still has some shortcomings in interaction aspects such as stimulus paradigm and control logic. The purpose of this study is to innovate the visual stimulus paradigm and asynchronous decoding/control strategy by integrating augmented reality technology, and propose an asynchronous pattern recognition algorithm, thereby improving the interaction logic and practical application capabilities of the prosthetic hand with the BCI system.

Methods

An asynchronous visual stimulus paradigm based on an augmented reality (AR) interface was proposed in this paper, in which there were 8 control modes, including Grasp, Put down, Pinch, Point, Fist, Palm push, Hold pen, and Initial. According to the attentional orienting characteristics of the paradigm, a novel asynchronous pattern recognition algorithm that combines center extended canonical correlation analysis and support vector machine (Center-ECCA-SVM) was proposed. Then, this study proposed an intelligent BCI system switch based on a deep learning object detection algorithm (YOLOv4) to improve the level of user interaction. Finally, two experiments were designed to test the performance of the brain-controlled prosthetic hand system and its practical performance in real scenarios.

Results

Under the AR paradigm of this study, compared with the liquid crystal display (LCD) paradigm, the average SSVEP spectrum amplitude of multiple subjects increased by 17.41%, and the signal-noise ratio (SNR) increased by 3.52%. The average stimulus pattern recognition accuracy was 96.71 ± 3.91%, which was 2.62% higher than the LCD paradigm. Under the data analysis time of 2s, the Center-ECCA-SVM classifier obtained 94.66 ± 3.87% and 97.40 ± 2.78% asynchronous pattern recognition accuracy under the Normal metric and the Tolerant metric, respectively. And the YOLOv4-tiny model achieves a speed of 25.29fps and a 96.4% confidence in the prosthetic hand in real-time detection. Finally, the brain-controlled prosthetic hand helped the subjects to complete 4 kinds of daily life tasks in the real scene, and the time-consuming were all within an acceptable range, which verified the effectiveness and practicability of the system.

Conclusion

This research is based on improving the user interaction level of the prosthetic hand with the BCI system, and has made improvements in the SSVEP paradigm, asynchronous pattern recognition, interaction, and control logic. Furthermore, it also provides support for BCI areas for alternative prosthetic control, and movement disorder rehabilitation programs.

Effects of motion type on motion-onset and steady-state visual evoked potentials: rotation vs. flicker

The underlying mechanisms of information processing for two basic motion types, rotation and flicker, are not fully understood. Rotational and flickering animations at four speeds - 7 frames per second (fps), 8 fps, 11 fps, and 12 fps, respectively - are presented as visual stimuli. The motion-onset visual evoked potentials (VEPs) and steady-state VEPs (SSVEP) elicited by these motion stimuli were compared between the rotation and flicker motion types at time windows of 0-500 ms and 1000-5000 ms post-stimulus, respectively. The standardized low-resolution electromagnetic tomography (sLORETA) source localization was investigated as well. Four motion speeds had no effect on the whole VEP waveform in either the rotation or the flicker groups. Significant differences in motion-onset VEPs and sLORETA source localization were found between the rotation and the flicker motion types at time windows of 200-500 ms post-stimulus. For the time windows of 1000-5000 ms post-stimulus, both the rotation and flicker groups all demonstrated the characteristics of SSVEP, with the peak spectral topographies showing at the four different frequencies, which correspond to the four motion speeds. Additionally, a higher power of spectral topography at each of the four motion speeds was found in the rotation relative to the flicker stimulation. The perceptual and cognitive processes are distinct for two types of motion: rotation and flicker. In terms of motion-onset VEPs and the characteristics of SSVEP, rotating visual stimulation is superior to flicker stimulation and may be more appropriate for clinical and engineering applications.

Brain stimulation with 40 Hz heterochromatic flicker extended beyond red, green, and blue

Alzheimer's disease (AD) is associated with electrophysiological changes in the brain. Pre-clinical and early clinical trials have shown promising results for the possible therapy of AD with 40 Hz neurostimulation. The most notable findings used stroboscopic flicker, but this technique poses an inherent barrier for human applications due to its visible flickering and resulting high level of perceived discomfort. Therefore, alternative options should be investigated for entraining 40 Hz brain activity with light sources that appear less flickering. Previously, chromatic flicker based on red, green, and blue (RGB) have been studied in the context of brain-computer interfaces, but this is an incomplete representation of the colours in the visual spectrum. This study introduces a new kind of heterochromatic flicker based on spectral combinations of blue, cyan, green, lime, amber, and red (BCGLAR). These combinations are investigated by the steady-state visually evoked potential (SSVEP) response from the flicker with an aim of optimising the choice of 40 Hz light stimulation with spectrally similar colour combinations in BCGLAR space. Thirty healthy young volunteers were stimulated with heterochromatic flicker in an electroencephalography experiment with randomised complete block design. Responses were quantified as the 40 Hz signal-to-noise ratio and analysed using mixed linear models. The size of the SSVEP response to heterochromatic flicker is dependent on colour combinations and influenced by both visual and non-visual effects. The amber-red flicker combination evoked the highest SSVEP, and combinations that included blue and/or red consistently evoked higher SSVEP than combinations only with mid-spectrum colours. Including a colour from either extreme of the visual spectrum (blue and/or red) in at least one of the dyadic phases appears to be more important than choosing pairs of colours that are far from each other on the visual spectrum. Spectrally adjacent colour pairs appear less flickering to the perceiver, and thus the results motivate investigations into the limits for how alike the two phases can be and still evoke a 40 Hz response. Specifically, combining a colour on either extreme of the visual spectrum with another proximal colour might provide the best trade-off between flickering sensation and SSVEP magnitude.

Rhythmic visual stimulation as a window into early brain development: A systematic review

Rhythmic visual stimulation (RVS), the periodic presentation of visual stimuli to elicit a rhythmic brain response, is increasingly applied to reveal insights into early neurocognitive development. Our systematic review identified 69 studies applying RVS in 0- to 6-year-olds. RVS has long been used to study the development of the visual system and applications have more recently been expanded to uncover higher cognitive functions in the developing brain, including overt and covert attention, face and object perception, numeral cognition, and predictive processing. These insights are owed to the unique benefits of RVS, such as the targeted frequency and stimulus-specific neural responses, as well as a remarkable signal-to-noise ratio. Yet, neural mechanisms underlying the RVS response are still poorly understood. We discuss critical challenges and avenues for future research, and the unique potentials the method holds. With this review, we provide a resource for researchers interested in the breadth of developmental RVS research and hope to inspire the future use of this cutting-edge method in developmental cognitive neuroscience.

A Calibration-Free Hybrid BCI Speller System Based on High-Frequency SSVEP and sEMG

Hybrid brain-computer interface (hBCI) systems that combine steady-state visual evoked potential (SSVEP) and surface electromyography (sEMG) signals have attracted attention of researchers due to the advantage of exhibiting significantly improved system performance. However, almost all existing studies adopt low-frequency SSVEP to build hBCI. It produces much more visual fatigue than high-frequency SSVEP. Therefore, the current study attempts to build a hBCI based on high-frequency SSVEP and sEMG. With these two signals, this study designed and realized a 32-target hBCI speller system. Thirty-two targets were separated from the middle into two groups. Each side contained 16 sets of targets with different high-frequency visual stimuli (i.e., 31-34.75 Hz with an interval of 0.25 Hz). sEMG was utilized to choose the group and SSVEP was adopted to identify intra-group targets. The filter bank canonical correlation analysis (FBCCA) and the root mean square value (RMS) methods were used to identify signals. Therefore, the proposed system allowed users to operate it without system calibration. A total of 12 healthy subjects participated in online experiment, with an average accuracy of 93.52 ± 1.66% and the average information transfer rate (ITR) reached 93.50 ± 3.10 bits/min. Furthermore, 12 participants perfectly completed the free-spelling tasks. These results of the experiments indicated feasibility and practicality of the proposed hybrid BCI speller system.

Brain-Computer Interface Based on Steady-State Visual Evoked Potential Using Quick-Response Code Pattern for Wheelchair Control

Brain-computer interfaces (BCIs) are widely utilized in control applications for people with severe physical disabilities. Several researchers have aimed to develop practical brain-controlled wheelchairs. An existing electroencephalogram (EEG)-based BCI based on steady-state visually evoked potential (SSVEP) was developed for device control. This study utilized a quick-response (QR) code visual stimulus pattern for a robust existing system. Four commands were generated using the proposed visual stimulation pattern with four flickering frequencies. Moreover, we employed a relative power spectrum density (PSD) method for the SSVEP feature extraction and compared it with an absolute PSD method. We designed experiments to verify the efficiency of the proposed system. The results revealed that the proposed SSVEP method and algorithm yielded an average classification accuracy of approximately 92% in real-time processing. For the wheelchair simulated via independent-based control, the proposed BCI control required approximately five-fold more time than the keyboard control for real-time control. The proposed SSVEP method using a QR code pattern can be used for BCI-based wheelchair control. However, it suffers from visual fatigue owing to long-time continuous control. We will verify and enhance the proposed system for wheelchair control in people with severe physical disabilities.

Cross-Platform Implementation of an SSVEP-Based BCI for the Control of a 6-DOF Robotic Arm

Robotics has been successfully applied in the design of collaborative robots for assistance to people with motor disabilities. However, man-machine interaction is difficult for those who suffer severe motor disabilities. The aim of this study was to test the feasibility of a low-cost robotic arm control system with an EEG-based brain-computer interface (BCI). The BCI system relays on the Steady State Visually Evoked Potentials (SSVEP) paradigm. A cross-platform application was obtained in C++. This C++ platform, together with the open-source software Openvibe was used to control a Stäubli robot arm model TX60. Communication between Openvibe and the robot was carried out through the Virtual Reality Peripheral Network (VRPN) protocol. EEG signals were acquired with the 8-channel Enobio amplifier from Neuroelectrics. For the processing of the EEG signals, Common Spatial Pattern (CSP) filters and a Linear Discriminant Analysis classifier (LDA) were used. Five healthy subjects tried the BCI. This work allowed the communication and integration of a well-known BCI development platform such as Openvibe with the specific control software of a robot arm such as Stäubli TX60 using the VRPN protocol. It can be concluded from this study that it is possible to control the robotic arm with an SSVEP-based BCI with a reduced number of dry electrodes to facilitate the use of the system.

Development of an Ultra Low-Cost SSVEP-based BCI Device for Real-Time On-Device Decoding

This study details the development of a novel, approx. £20 electroencephalogram (EEG)-based brain-computer interface (BCI) intended to offer a financially and operationally accessible device that can be deployed on a mass scale to facilitate education and public engagement in the domain of EEG sensing and neurotechnologies. Real-time decoding of steady-state visual evoked potentials (SSVEPs) is achieved using variations of the widely-used canonical correlation analysis (CCA) algorithm: multi-set CCA and generalised CCA. All BCI functionality is executed on board an inexpensive ESP32 microcontroller. SSVEP decoding accuracy of 95.56 ± 3.74% with an ITR of 102 bits/min was achieved with modest calibration.

Steady-State Visual Evoked Potential-Based Brain–Computer Interface Using a Novel Visual Stimulus with Quick Response (QR) Code Pattern

Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems suffer from low SSVEP response intensity and visual fatigue, resulting in lower accuracy when operating the system for continuous commands, such as an electric wheelchair control. This study proposes two SSVEP improvements to create a practical BCI for communication and control in disabled people. The first is flicker pattern modification for increasing SSVEP response through mixing (1) fundamental and first harmonic frequencies, and (2) two fundamental frequencies for an additional number of commands. The second method utilizes a quick response (QR) code for visual stimulus patterns to increase the SSVEP response and reduce visual fatigue. Eight different stimulus patterns from three flickering frequencies (7, 13, and 17 Hz) were presented to twelve participants for the test and score levels of visual fatigue. Two popular SSVEP methods, i.e., power spectral density (PSD) with Welch periodogram and canonical correlation analysis (CCA) with overlapping sliding window, are used to detect SSVEP intensity and response, compared to the checkerboard pattern. The results suggest that the QR code patterns can yield higher accuracy than checkerboard patterns for both PSD and CCA methods. Moreover, a QR code pattern with low frequency can reduce visual fatigue; however, visual fatigue can be easily affected by high flickering frequency. The findings can be used in the future to implement a real-time, SSVEP-based BCI for verifying user and system performance in actual environments.

A hybrid environment control system combining EMG and SSVEP signal based on brain-computer interface technology

The patients who are impaired with neurodegenerative disorders cannot command their muscles through the neural pathways. These patients are given an alternative from their neural path through Brain-Computer Interface (BCI) systems, which are the explicit use of brain impulses without any need for a computer's vocal muscle. Nowadays, the steady-state visual evoked potential (SSVEP) modality offers a robust communication pathway to introduce a non-invasive BCI. There are some crucial constituents, including window length of SSVEP response, the number of electrodes in the acquisition device and system accuracy, which are the critical performance components in any BCI system based on SSVEP signal. In this study, a real-time hybrid BCI system consists of SSVEP and EMG has been proposed for the environmental control system. The feature in terms of the common spatial pattern (CSP) has been extracted from four classes of SSVEP response, and extracted feature has been classified using K-nearest neighbors (k-NN) based classification algorithm. The obtained classification accuracy of eight participants was 97.41%. Finally, a control mechanism that aims to apply for the environmental control system has also been developed. The proposed system can identify 18 commands (i.e., 16 control commands using SSVEP and two commands using EMG). This result represents very encouraging performance to handle real-time SSVEP based BCI system consists of a small number of electrodes. The proposed framework can offer a convenient user interface and a reliable control method for realistic BCI technology.