On the role of generative artificial intelligence in the development of brain-computer interfaces

Since their inception more than 50 years ago, Brain-Computer Interfaces (BCIs) have held promise to compensate for functions lost by people with disabilities through allowing direct communication between the brain and external devices. While research throughout the past decades has demonstrated the feasibility of BCI to act as a successful assistive technology, the widespread use of BCI outside the lab is still beyond reach. This can be attributed to a number of challenges that need to be addressed for BCI to be of practical use including limited data availability, limited temporal and spatial resolutions of brain signals recorded non-invasively and inter-subject variability. In addition, for a very long time, BCI development has been mainly confined to specific simple brain patterns, while developing other BCI applications relying on complex brain patterns has been proven infeasible. Generative Artificial Intelligence (GAI) has recently emerged as an artificial intelligence domain in which trained models can be used to generate new data with properties resembling that of available data. Given the enhancements observed in other domains that possess similar challenges to BCI development, GAI has been recently employed in a multitude of BCI development applications to generate synthetic brain activity; thereby, augmenting the recorded brain activity. Here, a brief review of the recent adoption of GAI techniques to overcome the aforementioned BCI challenges is provided demonstrating the enhancements achieved using GAI techniques in augmenting limited EEG data, enhancing the spatiotemporal resolution of recorded EEG data, enhancing cross-subject performance of BCI systems and implementing end-to-end BCI applications. GAI could represent the means by which BCI would be transformed into a prevalent assistive technology, thereby improving the quality of life of people with disabilities, and helping in adopting BCI as an emerging human-computer interaction technology for general use.

Post-stimulus encoding of decision confidence in EEG: toward a brain-computer interface for decision making

Objective.When making decisions, humans can evaluate how likely they are to be correct. If this subjective confidence could be reliably decoded from brain activity, it would be possible to build a brain-computer interface (BCI) that improves decision performance by automatically providing more information to the user if needed based on their confidence. But this possibility depends on whether confidence can be decoded right after stimulus presentation and before the response so that a corrective action can be taken in time. Although prior work has shown that decision confidence is represented in brain signals, it is unclear if the representation is stimulus-locked or response-locked, and whether stimulus-locked pre-response decoding is sufficiently accurate for enabling such a BCI.Approach.We investigate the neural correlates of confidence by collecting high-density electroencephalography (EEG) during a perceptual decision task with realistic stimuli. Importantly, we design our task to include a post-stimulus gap that prevents the confounding of stimulus-locked activity by response-locked activity and vice versa, and then compare with a task without this gap.Main results.We perform event-related potential and source-localization analyses. Our analyses suggest that the neural correlates of confidence are stimulus-locked, and that an absence of a post-stimulus gap could cause these correlates to incorrectly appear as response-locked. By preventing response-locked activity from confounding stimulus-locked activity, we then show that confidence can be reliably decoded from single-trial stimulus-locked pre-response EEG alone. We also identify a high-performance classification algorithm by comparing a battery of algorithms. Lastly, we design a simulated BCI framework to show that the EEG classification is accurate enough to build a BCI and that the decoded confidence could be used to improve decision making performance particularly when the task difficulty and cost of errors are high.Significance.Our results show feasibility of non-invasive EEG-based BCIs to improve human decision making.

Eyes on the road: brain computer interfaces and cognitive distraction in traffic

Novel wearable neurotechnology is able to provide insight into its wearer's cognitive processes and offers ways to change or enhance their capacities. Moreover, it offers the promise of hands-free device control. These brain-computer interfaces are likely to become an everyday technology in the near future, due to their increasing accessibility and affordability. We, therefore, must anticipate their impact, not only on society and individuals broadly but also more specifically on sectors such as traffic and transport. In an economy where attention is increasingly becoming a scarce good, these innovations may present both opportunities and challenges for daily activities that require focus, such as driving and cycling. Here, we argue that their development carries a dual risk. Firstly, BCI-based devices may match or further increase the intensity of cognitive human-technology interaction over the current hands-free communication devices which, despite being widely accepted, are well-known for introducing a significant amount of cognitive load and distraction. Secondly, BCI-based devices will be typically harder than hands-free devices to both visually detect (e.g., how can law enforcement check when these extremely small and well-integrated devices are used?) and restrain in their use (e.g., how do we prevent users from using such neurotechnologies without breaching personal integrity and privacy?). Their use in traffic should be anticipated by researchers, engineers, and policymakers, in order to ensure the safety of all road users.

Single trial detection of error-related potentials in brain-machine interfaces: a survey and comparison of methods

Objective.Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain-computer interfaces (BCIs), increase the naturalness of human-machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms).Approach.With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification.Main results.From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination).Significance.This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.

EEG-EMG coupling as a hybrid method for steering detection in car driving settings

Understanding mental processes in complex human behavior is a key issue in driving, representing a milestone for developing user-centered assistive driving devices. Here, we propose a hybrid method based on electroencephalographic (EEG) and electromyographic (EMG) signatures to distinguish left and right steering in driving scenarios. Twenty-four participants took part in the experiment consisting of recordings of 128-channel EEG and EMG activity from deltoids and forearm extensors in non-ecological and ecological steering tasks. Specifically, we identified the EEG mu rhythm modulation correlates with motor preparation of self-paced steering actions in the non-ecological task, while the concurrent EMG activity of the left (right) deltoids correlates with right (left) steering. Consequently, we exploited the mu rhythm de-synchronization resulting from the non-ecological task to detect the steering side using cross-correlation analysis with the ecological EMG signals. Results returned significant cross-correlation values showing the coupling between the non-ecological EEG feature and the muscular activity collected in ecological driving conditions. Moreover, such cross-correlation patterns discriminate the steering side earlier relative to the single EMG signal. This hybrid system overcomes the limitation of the EEG signals collected in ecological settings such as low reliability, accuracy, and adaptability, thus adding to the EMG the characteristic predictive power of the cerebral data. These results prove how it is possible to complement different physiological signals to control the level of assistance needed by the driver.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-021-09776-w.

The Application of Electroencephalogram in Driving Safety: Current Status and Future Prospects

The driver is one of the most important factors in the safety of the transportation system. The driver's perceptual characteristics are closely related to driving behavior, while electroencephalogram (EEG) as the gold standard for evaluating human perception is non-deceptive. It is essential to study driving characteristics by analyzing the driver's brain activity pattern, effectively acquiring driver perceptual characteristics, creating a direct connection between the driver's brain and external devices, and realizing information interchange. This paper first introduces the theories related to EEG, then reviews the applications of EEG in scenarios such as fatigue driving, distracted driving, and emotional driving. The limitations of existing research have been identified and the prospect of EEG application in future brain-computer interface automotive assisted driving systems have been proposed. This review provides guidance for researchers to use EEG to improve driving safety. It also offers valuable suggestions for future research.

Binary Controller Based on the Electrical Activity Related to Head Yaw Rotation

A human machine interface (HMI) is presented to switch on/off lights according to the head left/right yaw rotation. The HMI consists of a cap, which can acquire the brain’s electrical activity (i.e., an electroencephalogram, EEG) sampled at 500 Hz on 8 channels with electrodes that are positioned according to the standard 10–20 system. In addition, the HMI includes a controller based on an input–output function that can compute the head position (defined as left, right, and forward position with respect to yaw angle) considering short intervals (10 samples) of the signals coming from three electrodes positioned in O1, O2, and Cz. An artificial neural network (ANN) training based on a Levenberg–Marquardt backpropagation algorithm was used to identify the input–output function. The HMI controller was tested on 22 participants. The proposed classifier achieved an average accuracy of 88% with the best value of 96.85%. After calibration for each specific subject, the HMI was used as a binary controller to verify its ability to switch on/off lamps according to head turning movement. The correct prediction of the head movements was greater than 75% in 90% of the participants when performing the test with open eyes. If the subjects carried out the experiments with closed eyes, the prediction accuracy reached 75% of correctness in 11 participants out of 22. One participant controlled the light system in both experiments, open and closed eyes, with 100% success. The control results achieved in this work can be considered as an important milestone towards humanoid neck systems.

Hybrid Systems to Boost EEG-Based Real-Time Action Decoding in Car Driving Scenarios

The complexity of concurrent cerebral processes underlying driving makes such human behavior one of the most studied real-world activities in neuroergonomics. Several attempts have been made to decode, both offline and online, cerebral activity during car driving with the ultimate goal to develop brain-based systems for assistive devices. Electroencephalography (EEG) is the cornerstone of these studies providing the highest temporal resolution to track those cerebral processes underlying overt behavior. Particularly when investigating real-world scenarios as driving, EEG is constrained by factors such as robustness, comfortability, and high data variability affecting the decoding performance. Hence, additional peripheral signals can be combined with EEG for increasing replicability and the overall performance of the brain-based action decoder. In this regard, hybrid systems have been proposed for the detection of braking and steering actions in driving scenarios to improve the predictive power of the single neurophysiological measurement. These recent results represent a proof of concept of the level of technological maturity. They may pave the way for increasing the predictive power of peripheral signals, such as electroculogram (EOG) and electromyography (EMG), collected in real-world scenarios when informed by EEG measurements, even if collected only offline in standard laboratory settings. The promising usability of such hybrid systems should be further investigated in other domains of neuroergonomics.

Coordinating With a Robot Partner Affects Neural Processing Related to Action Monitoring

Robots start to play a role in our social landscape, and they are progressively becoming responsive, both physically and socially. It begs the question of how humans react to and interact with robots in a coordinated manner and what the neural underpinnings of such behavior are. This exploratory study aims to understand the differences in human-human and human-robot interactions at a behavioral level and from a neurophysiological perspective. For this purpose, we adapted a collaborative dynamical paradigm from the literature. We asked 12 participants to hold two corners of a tablet while collaboratively guiding a ball around a circular track either with another participant or a robot. In irregular intervals, the ball was perturbed outward creating an artificial error in the behavior, which required corrective measures to return to the circular track again. Concurrently, we recorded electroencephalography (EEG). In the behavioral data, we found an increased velocity and positional error of the ball from the track in the human-human condition vs. human-robot condition. For the EEG data, we computed event-related potentials. We found a significant difference between human and robot partners driven by significant clusters at fronto-central electrodes. The amplitudes were stronger with a robot partner, suggesting a different neural processing. All in all, our exploratory study suggests that coordinating with robots affects action monitoring related processing. In the investigated paradigm, human participants treat errors during human-robot interaction differently from those made during interactions with other humans. These results can improve communication between humans and robot with the use of neural activity in real-time.

Cross-Subject Zero Calibration Driver's Drowsiness Detection: Exploring Spatiotemporal Image Encoding of EEG Signals for Convolutional Neural Network Classification

This paper explores two methodologies for drowsiness detection using EEG signals in a sustained-attention driving task considering pre-event time windows, and focusing on cross-subject zero calibration. Driving accidents are a major cause of injuries and deaths on the road. A considerable portion of those are due to fatigue and drowsiness. Advanced driver assistance systems that could detect mental states which are associated with hazardous situations, such as drowsiness, are of critical importance. EEG signals are used widely for brain-computer interfaces, as well as mental state recognition. However, these systems are still difficult to design due to very low signal-to-noise ratios and cross-subject disparities, requiring individual calibration cycles. To tackle this research domain, here, we explore drowsiness detection based on EEG signals' spatiotemporal image encoding representations in the form of either recurrence plots or gramian angular fields for deep convolutional neural network (CNN) classification. Results comparing both techniques using a public dataset of 27 subjects show a superior balanced accuracy of up to 75.87% for leave-one-out cross-validation, using both techniques, against works in the literature, demonstrating the possibility to pursue cross-subject zero calibration design.

Closed-loop EEG study on visual recognition during driving

OBJECTIVE

In contrast to the classical visual BCI paradigms, which adhere to a rigid trial structure and restricted user behavior, EEG-based visual recognition decoding during our daily activities remains challenging. The objective of this study is to explore the feasibility of decoding the EEG signature of visual recognition in experimental conditions promoting our natural ocular behavior when interacting with our dynamic environment.

APPROACH

In our experiment, subjects visually search for a target object among suddenly appearing objects in the environment while driving a car-simulator. Given that subjects exhibit an unconstrained overt visual behavior, we based our study on eye fixation-related potentials (EFRP). We report on gaze behavior and single-trial EFRP decoding performance (fixations on visually similar target vs. non-target objects). In addition, we demonstrate the application of our approach in a closed-loop BCI setup.

MAIN RESULTS

To identify the target out of four symbol types along a road segment, the BCI system integrated decoding probabilities of multiple EFRP and achieved the average online accuracy of 0.37 ± 0.06 (12 subjects), statistically significantly above the chance level. Using the acquired data, we performed a comparative study of classification algorithms (discriminating target vs. non-target) and feature spaces in a simulated online scenario. The EEG approaches yielded similar moderate performances of at most 0.6 AUC, yet statistically significantly above the chance level. In addition, the gaze duration (dwell time) appears to be an additional informative feature in this context.

SIGNIFICANCE

These results show that visual recognition of sudden events can be decoded during active driving. Therefore, this study lays a foundation for assistive and recommender systems based on the driver's brain signals.

Optimized artificial neural network based performance analysis of wheelchair movement for ALS patients

Individuals with neurodegenerative attacks loose the entire motor neuron movements. These conditions affect the individual actions like walking, speaking impairment and totally make the person in to locked in state (LIS). To overcome the miserable condition the person need rehabilitation devices through a Brain Computer Interfaces (BCI) to satisfy their needs. BMI using Electroencephalogram (EEG) receives the mental thoughts from brain and converts into control signals to activate the exterior communication appliances in the absence of biological channels. To design the BCI, we conduct our study with three normal male subjects, three normal female subjects and three ALS affected individuals from the age of 20-60 with three electrode systems for four tasks. One Dimensional Local Binary Patterns (LBP) technique was applied to reduce the digitally sampled features collected from nine subjects was treated with Grey wolf optimization Neural Network (GWONN) to classify the mentally composed words. Using these techniques, we compared the three types of subjects to identify the performances. The study proves that subjects from normal male categories performance was maximum compared with the other subjects. To assess the individual performance of the subject, we conducted the recognition accuracy test in offline mode. From the accuracy test also, we obtained the best performance from the normal male subjects compared with female and ALS subjects with an accuracy of 98.33 %, 95.00 % and 88.33 %. Finally our study concludes that patients with ALS attack need more training than that of the other subjects.

Self-health monitoring and wearable neurotechnologies

Brain-computer interfaces and wearable neurotechnologies are now used to measure real-time neural and physiologic signals from the human body and hold immense potential for advancements in medical diagnostics, prevention, and intervention. Given the future role that wearable neurotechnologies will likely serve in the health sector, a critical state-of-the-art assessment is necessary to gain a better understanding of their current strengths and limitations. In this chapter we present wearable electroencephalography systems that reflect groundbreaking innovations and improvements in real-time data collection and health monitoring. We focus on specifications reflecting technical advantages and disadvantages, discuss their use in fundamental and clinical research, their current applications, limitations, and future directions. While many methodological and ethical challenges remain, these systems host the potential to facilitate large-scale data collection far beyond the reach of traditional research laboratory settings.