Brain-computer interfaces for communication and rehabilitation

Correcting for ERP latency jitter improves gaze-independent BCI decoding

Objective.Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze.Approach.ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment.Main results.WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session.Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.

A Cascade xDAWN EEGNet Structure for Unified Visual-Evoked Related Potential Detection

Visual-based brain-computer interface (BCI) enables people to communicate with others by spelling words from the brain and helps professionals recognize targets in large numbers of images. P300 signals evoked by different types of stimuli, such as words or images, may vary significantly in terms of both amplitude and latency. A unified approach is required to detect variable P300 signals, which facilitates BCI applications, as well as deepens the understanding of the P300 generation mechanism. In this study, our proposed approach involves a cascade network structure that combines xDAWN and classical EEGNet techniques. This network is designed to classify target and non-target stimuli in both P300 speller and rapid serial visual presentation (RSVP) paradigms. The proposed approach is capable of recognizing more symbols with fewer repetitions (up to 5 rounds) compared to other models while possessing a better information transfer rate (ITR) as demonstrated on Dataset II (17.22 bits/min in the second repetition round) of BCI Competition III. Additionally, our approach has the highest unweighted average recall (UAR) performance for both 5 Hz ( 0.8134±0.0259 ) and 20 Hz ( 0.6527±0.0321 ) RSVP. The results show that the cascade network structure has better performance between both the P300 Speller and RSVP paradigms, manifesting that such a cascade structure is robust enough for dealing with P300-related signals (source code is available at https://github.com/embneural/Cascade-xDAWN-EEGNet-for-ERP-Detection).

An EEG & eye-tracking dataset of ALS patients & healthy people during eye-tracking-based spelling system usage

This research presents a dataset consisting of electroencephalogram and eye tracking recordings obtained from six patients with amyotrophic lateral sclerosis (ALS) in a locked-in state and one hundred seventy healthy individuals. The ALS patients exhibited varying degrees of disease progression, ranging from partial mobility and weakened speech to complete paralysis and loss of speech. Despite these physical impairments, the ALS patients retained good eye function, which allowed them to use a virtual keyboard for communication. Data from ALS patients was recorded multiple times at their homes, while data from healthy individuals was recorded once in a laboratory setting. For each data recording, the experimental design involved nine recording sessions per participant, each corresponding to a common human action or demand. This dataset can serve as a valuable benchmark for several applications, such as improving spelling systems with brain-computer interfaces, investigating motor imagination, exploring motor cortex function, monitoring motor impairment progress in patients undergoing rehabilitation, and studying the effects of ALS on cognitive and motor processes.

An Ion-Mediated Spiking Chemical Neuron based on Mott Memristor

Artificial spiking neurons capable of interpreting ionic information into electrical spikes are critical to mimic biological signaling systems. Mott memristors are attractive for constructing artificial spiking neurons due to their simple structure, low energy consumption, and rich neural dynamics. However, challenges remain in achieving ion-mediated spiking and biohybrid-interfacing in Mott neurons. Here, a biomimetic spiking chemical neuron (SCN) utilizing an NbOx Mott memristor and oxide field-effect transistor-type chemical sensor is introduced. The SCN exhibits both excitation and inhibition spiking behaviors toward ionic concentrations akin to biological neural systems. It demonstrates spiking responses across physiological and pathological Na+ concentrations (1-200 × 10-3 m). The Na+-mediated SCN enables both frequency encoding and time-to-first-spike coding schemes, illustrating the rich neural dynamics of Mott neuron. In addition, the SCN interfaced with L929 cells facilitates real-time modulation of ion-mediated spiking under both normal and salty cellular microenvironments.

"Your Thoughts are (were) Free!": Brain-Computer-Interfaces, Neurofeedback, Detection of Deception, and the Future of Mind-Reading

This review describes the historical developement and rationale of clinically relevant research on neurophysiological "mind reading" paradims: Brain- Computer-Interfaces, detection of deception, brain stimulation and neurofeedback and the clinical applications in drug resistant epilepsy, chronic stroke, and communication with paralyzed locked-in persons. The emphasis lies on completely locked-in patients with amyotrophic lateral sclerosis using non-invasive and invasive brain computer interfaces and neurofeedback to restore verbal communication with the social environment. In the second part of the article we argue that success and failure of neurophysiological "mind reading" paradigms may be explained with a motor theory of thinking and emotion in combination with learning theory. The ethical implications of brain computer interface and neurofeedback approaches, particularly for severe chronic paralysis and loss of communication diseases and decisions on hastened death and euthanasia are discussed.

Multi-source deep domain adaptation ensemble framework for cross-dataset motor imagery EEG transfer learning

Objective. Electroencephalography (EEG) is an important kind of bioelectric signal for measuring physiological activities of the brain, and motor imagery (MI) EEG has significant clinical application prospects. Convolutional neural network has become a mainstream algorithm for MI EEG classification, however lack of subject-specific data considerably restricts its decoding accuracy and generalization performance. To address this challenge, a novel transfer learning (TL) framework using auxiliary dataset to improve the MI EEG classification performance of target subject is proposed in this paper.Approach. We developed a multi-source deep domain adaptation ensemble framework (MSDDAEF) for cross-dataset MI EEG decoding. The proposed MSDDAEF comprises three main components: model pre-training, deep domain adaptation, and multi-source ensemble. Moreover, for each component, different designs were examined to verify the robustness of MSDDAEF.Main results. Bidirectional validation experiments were performed on two large public MI EEG datasets (openBMI and GIST). The highest average classification accuracy of MSDDAEF reaches 74.28% when openBMI serves as target dataset and GIST serves as source dataset. While the highest average classification accuracy of MSDDAEF is 69.85% when GIST serves as target dataset and openBMI serves as source dataset. In addition, the classification performance of MSDDAEF surpasses several well-established studies and state-of-the-art algorithms.Significance. The results of this study show that cross-dataset TL is feasible for left/right-hand MI EEG decoding, and further indicate that MSDDAEF is a promising solution for addressing MI EEG cross-dataset variability.

Peripheral nerve transfers for dysfunctions in central nervous system injuries: a systematic review

BACKGROUND

The review highlights recent advancements and innovative uses of nerve transfer surgery in treating dysfunctions caused by central nervous system (CNS) injuries, with a particular focus on spinal cord injury (SCI), stroke, traumatic brain injury, and cerebral palsy.

METHODS

A comprehensive literature search was conducted regarding nerve transfer for restoring sensorimotor functions and bladder control following injuries of spinal cord and brain, across PubMed and Web of Science from January 1920 to May 2023. Two independent reviewers undertook article selection, data extraction, and risk of bias assessment with several appraisal tools, including the Cochrane Risk of Bias Tool, the JBI Critical Appraisal Checklist, and SYRCLE's ROB tool. The study protocol has been registered and reported following PRISMA and AMSTAR guidelines.

RESULTS

Nine hundred six articles were retrieved, of which 35 studies were included (20 on SCI and 15 on brain injury), with 371 participants included in the surgery group and 192 in the control group. These articles were mostly low-risk, with methodological concerns in study types, highlighting the complexity and diversity. For SCI, the strength of target muscle increased by 3.13 of Medical Research Council grade, and the residual urine volume reduced by more than 100 ml in 15 of 20 patients. For unilateral brain injury, the Fugl-Myer motor assessment (FMA) improved 15.14-26 score in upper extremity compared to 2.35-26 in the control group. The overall reduction in Modified Ashworth score was 0.76-2 compared to 0-1 in the control group. Range of motion (ROM) increased 18.4-80° in elbow, 20.4-110° in wrist and 18.8-130° in forearm, while ROM changed -4.03°-20° in elbow, -2.08°-10° in wrist, -2.26°-20° in forearm in the control group. The improvement of FMA in lower extremity was 9 score compared to the presurgery.

CONCLUSION

Nerve transfer generally improves sensorimotor functions in paralyzed limbs and bladder control following CNS injury. The technique effectively creates a 'bypass' for signals and facilitates functional recovery by leveraging neural plasticity. It suggested a future of surgery, neurorehabilitation and robotic-assistants converge to improve outcomes for CNS.

Predicting lever press in a vibrotactile yes/no detection task from S1 cortex of freely behaving rats by µECoG arrays

AIM OF THE STUDY

Brain-computer interfaces (BCIs) may help patients with severe neurological deficits communicate with the external world. Based on microelectrocorticography (µECoG) data recorded from the primary somatosensory cortex (S1) of unrestrained behaving rats, this study attempts to decode lever presses in a psychophysical detection task by using machine learning algorithms.

MATERIALS AND METHODS

16-channel Pt-Ir microelectrode arrays were implanted on the S1 of two rats, and µECoG was recorded during a vibrotactile yes/no detection task. For this task, the rats were trained to press the right lever when they detected the vibrotactile stimulus and the left lever when they did not. The multichannel µECoG data was analysed offline by time-frequency methods and its features were used for binary classification of the lever press at each trial. Several machine learning algorithms were tested as such.

RESULTS

The psychophysical sensitivities (A') were similar and low for both rats (0.58). Rat 2 (B'': -0.11) had higher bias for the right lever than Rat 1 (B'': - 0.01). The lever presses could be predicted with accuracies over 66% with all the tested algorithms, and the highest average accuracy (78%) was with the support vector machine.

CONCLUSION

According to the recent studies, sensory feedback increases the benefit of the BCIs. The current proof-of-concept study shows that lever presses can be decoded from the S1; therefore, this area may be utilised for a bidirectional BCI in the future.

Bridging Minds and Machines: The Recent Advances of Brain-Computer Interfaces in Neurological and Neurosurgical Applications

Brain-computer interfaces (BCIs), a remarkable technological advancement in neurology and neurosurgery, mark a significant leap since the inception of electroencephalography in 1924. These interfaces effectively convert central nervous system signals into commands for external devices, offering revolutionary benefits to patients with severe communication and motor impairments due to a myriad of neurological conditions like stroke, spinal cord injuries, and neurodegenerative disorders. BCIs enable these individuals to communicate and interact with their environment, using their brain signals to operate interfaces for communication and environmental control. This technology is especially crucial for those completely locked in, providing a communication lifeline where other methods fall short. The advantages of BCIs are profound, offering autonomy and an improved quality of life for patients with severe disabilities. They allow for direct interaction with various devices and prostheses, bypassing damaged or nonfunctional neural pathways. However, challenges persist, including the complexity of accurately interpreting brain signals, the need for individual calibration, and ensuring reliable, long-term use. Additionally, ethical considerations arise regarding autonomy, consent, and the potential for dependence on technology. Despite these challenges, BCIs represent a transformative development in neurotechnology, promising enhanced patient outcomes and a deeper understanding of brain-machine interfaces.

Cortical signals analysis to recognize intralimb mobility using modified RNN and various EEG quantities

Electroencephalogram (EEG) signals are critical in interpreting sensorimotor activities for predicting body movements. However, their efficacy in identifying intralimb movements, such as the dorsiflexion and plantar flexion of the foot, remains suboptimal. This study aims to explore whether various EEG signal quantities can effectively recognize intralimb movements to facilitate the development of Brain-Computer Interface (BCI) devices for foot rehabilitation. This research involved twenty-two healthy, right-handed participants. EEG data were collected using 21 electrodes positioned over the motor cortex, while two electromyography (EMG) electrodes recorded the onset of ankle joint movements. The study focused on analyzing slow cortical potential (SCP) and sensorimotor rhythms (SMR) in alpha and beta bands from the EEG. Five key features-fourth-order Autoregressive feature, variance, waveform length, standard deviation, and permutation entropy-were extracted. A modified Recurrent Neural Network (RNN) including Long Short-term Memory (LSTM) and Gated Recurrent Unit (GRU) algorithms was developed for movement recognition. These were compared against conventional machine learning algorithms, including nonlinear Support Vector Machine (SVM) and k Nearest Neighbourhood (kNN) classifiers. The performance of the proposed models was assessed using two data schemes: within-subject and across-subjects. The findings demonstrated that the GRU and LSTM models significantly outperformed traditional machine learning algorithms in recognizing different EEG signal quantities for intralimb movement. The study indicates that deep learning models, particularly GRU and LSTM, hold superior potential over standard machine learning techniques in identifying intralimb movements using EEG signals. Where the accuracies of LSTM for within and across subjects were 98.87 ± 1.80 % and 87.38 ± 0.86 % respectively. Whereas the accuracy of GRU within and across subjects were 99.18 ± 1.28 % and 86.44 ± 0.69 % respectively. This advancement could significantly benefit the development of BCI devices aimed at foot rehabilitation, suggesting a new avenue for enhancing physical therapy outcomes.

Advancements in brain-computer interfaces for the rehabilitation of unilateral spatial neglect: a concise review

This short review examines recent advancements in neurotechnologies within the context of managing unilateral spatial neglect (USN), a common condition following stroke. Despite the success of brain-computer interfaces (BCIs) in restoring motor function, there is a notable absence of effective BCI devices for treating cerebral visual impairments, a prevalent consequence of brain lesions that significantly hinders rehabilitation. This review analyzes current non-invasive BCIs and technological solutions dedicated to cognitive rehabilitation, with a focus on visuo-attentional disorders. We emphasize the need for further research into the use of BCIs for managing cognitive impairments and propose a new potential solution for USN rehabilitation, by combining the clinical subtleties of this syndrome with the technological advancements made in the field of neurotechnologies.

A New Framework Combining Diffusion Models and the Convolution Classifier for Generating Images from EEG Signals

The generation of images from electroencephalography (EEG) signals has become a popular research topic in recent research because it can bridge the gap between brain signals and visual stimuli and has wide application prospects in neuroscience and computer vision. However, due to the high complexity of EEG signals, the reconstruction of visual stimuli through EEG signals continues to pose a challenge. In this work, we propose an EEG-ConDiffusion framework that involves three stages: feature extraction, fine-tuning of the pretrained model, and image generation. In the EEG-ConDiffusion framework, classification features of EEG signals are first obtained through the feature extraction block. Then, the classification features are taken as conditions to fine-tune the stable diffusion model in the image generation block to generate images with corresponding semantics. This framework combines EEG classification and image generation means to enhance the quality of generated images. Our proposed framework was tested on an EEG-based visual classification dataset. The performance of our framework is measured by classification accuracy, 50-way top-k accuracy, and inception score. The results indicate that the proposed EEG-Condiffusion framework can extract effective classification features and generate high-quality images from EEG signals to realize EEG-to-image conversion.

A causal perspective on brainwave modeling for brain-computer interfaces

Objective. Machine learning (ML) models have opened up enormous opportunities in the field of brain-computer Interfaces (BCIs). Despite their great success, they usually face severe limitations when they are employed in real-life applications outside a controlled laboratory setting.Approach. Mixing causal reasoning, identifying causal relationships between variables of interest, with brainwave modeling can change one's viewpoint on some of these major challenges which can be found in various stages in the ML pipeline, ranging from data collection and data pre-processing to training methods and techniques.Main results. In this work, we employ causal reasoning and present a framework aiming to breakdown and analyze important challenges of brainwave modeling for BCIs.Significance. Furthermore, we present how general ML practices as well as brainwave-specific techniques can be utilized and solve some of these identified challenges. And finally, we discuss appropriate evaluation schemes in order to measure these techniques' performance and efficiently compare them with other methods that will be developed in the future.

Evolving Therapeutic Landscape of Intracerebral Hemorrhage: Emerging Cutting-Edge Advancements in Surgical Robots, Regenerative Medicine, and Neurorehabilitation Techniques

Intracerebral hemorrhage (ICH) is the most serious form of stroke and has limited available therapeutic options. As knowledge on ICH rapidly develops, cutting-edge techniques in the fields of surgical robots, regenerative medicine, and neurorehabilitation may revolutionize ICH treatment. However, these new advances still must be translated into clinical practice. In this review, we examined several emerging therapeutic strategies and their major challenges in managing ICH, with a particular focus on innovative therapies involving robot-assisted minimally invasive surgery, stem cell transplantation, in situ neuronal reprogramming, and brain-computer interfaces. Despite the limited expansion of the drug armamentarium for ICH over the past few decades, the judicious selection of more efficacious therapeutic modalities and the exploration of multimodal combination therapies represent opportunities to improve patient prognoses after ICH.

Prediction of cognitive conflict during unexpected robot behavior under different mental workload conditions in a physical human-robot collaboration

Objective. Brain-computer interface (BCI) technology is poised to play a prominent role in modern work environments, especially a collaborative environment where humans and machines work in close proximity, often with physical contact. In a physical human robot collaboration (pHRC), the robot performs complex motion sequences. Any unexpected robot behavior or faulty interaction might raise safety concerns. Error-related potentials, naturally generated by the brain when a human partner perceives an error, have been extensively employed in BCI as implicit human feedback to adapt robot behavior to facilitate a safe and intuitive interaction. However, the integration of BCI technology with error-related potential for robot control demands failure-free integration of highly uncertain electroencephalography (EEG) signals, particularly influenced by the physical and cognitive state of the user. As a higher workload on the user compromises their access to cognitive resources needed for error awareness, it is crucial to study how mental workload variations impact the error awareness as it might raise safety concerns in pHRC. In this study, we aim to study how cognitive workload affects the error awareness of a human user engaged in a pHRC.Approach. We designed a blasting task with an abrasive industrial robot and manipulated the mental workload with a secondary arithmetic task of varying difficulty. EEG data, perceived workload, task and physical performance were recorded from 24 participants moving the robot arm. The error condition was achieved by the unexpected stopping of the robot in 33% of trials.Main results. We observed a diminished amplitude for the prediction error negativity (PEN) and error positivity (Pe), indicating reduced error awareness with increasing mental workload. We further observed an increased frontal theta power and increasing trend in the central alpha and central beta power after the unexpected robot stopping compared to when the robot stopped correctly at the target. We also demonstrate that a popular convolution neural network model, EEGNet, could predict the amplitudes of PEN and Pe from the EEG data prior to the error.Significance. This prediction model could be instrumental in developing an online prediction model that could forewarn the system and operators of the diminished error awareness of the user, alluding to a potential safety breach in error-related potential-based BCI system for pHRC. Therefore, our work paves the way for embracing BCI technology in pHRC to optimally adapt the robot behavior for personalized user experience using real-time brain activity, enriching the quality of the interaction.

A novel theta-controlled vibrotactile brain-computer interface to treat chronic pain: a pilot study

Limitations in chronic pain therapies necessitate novel interventions that are effective, accessible, and safe. Brain-computer interfaces (BCIs) provide a promising modality for targeting neuropathology underlying chronic pain by converting recorded neural activity into perceivable outputs. Recent evidence suggests that increased frontal theta power (4-7 Hz) reflects pain relief from chronic and acute pain. Further studies have suggested that vibrotactile stimulation decreases pain intensity in experimental and clinical models. This longitudinal, non-randomized, open-label pilot study's objective was to reinforce frontal theta activity in six patients with chronic upper extremity pain using a novel vibrotactile neurofeedback BCI system. Patients increased their BCI performance, reflecting thought-driven control of neurofeedback, and showed a significant decrease in pain severity (1.29 ± 0.25 MAD, p = 0.03, q = 0.05) and pain interference (1.79 ± 1.10 MAD p = 0.03, q = 0.05) scores without any adverse events. Pain relief significantly correlated with frontal theta modulation. These findings highlight the potential of BCI-mediated cortico-sensory coupling of frontal theta with vibrotactile stimulation for alleviating chronic pain.

Measuring neuronal avalanches to inform brain-computer interfaces

Large-scale interactions among multiple brain regions manifest as bursts of activations called neuronal avalanches, which reconfigure according to the task at hand and, hence, might constitute natural candidates to design brain-computer interfaces (BCIs). To test this hypothesis, we used source-reconstructed magneto/electroencephalography during resting state and a motor imagery task performed within a BCI protocol. To track the probability that an avalanche would spread across any two regions, we built an avalanche transition matrix (ATM) and demonstrated that the edges whose transition probabilities significantly differed between conditions hinged selectively on premotor regions in all subjects. Furthermore, we showed that the topology of the ATMs allows task-decoding above the current gold standard. Hence, our results suggest that neuronal avalanches might capture interpretable differences between tasks that can be used to inform brain-computer interfaces.

Guidelines for reporting action simulation studies (GRASS): Proposals to improve reporting of research in motor imagery and action observation

Researchers from multiple disciplines have studied the simulation of actions through motor imagery, action observation, or their combination. Procedures used in these studies vary considerably between research groups, and no standardized approach to reporting experimental protocols has been proposed. This has led to under-reporting of critical details, impairing the assessment, replication, synthesis, and potential clinical translation of effects. We provide an overview of issues related to the reporting of information in action simulation studies, and discuss the benefits of standardized reporting. We propose a series of checklists that identify key details of research protocols to include when reporting action simulation studies. Each checklist comprises A) essential methodological details, B) essential details that are relevant to a specific mode of action simulation, and C) further points that may be useful on a case-by-case basis. We anticipate that the use of these guidelines will improve the understanding, reproduction, and synthesis of studies using action simulation, and enhance the translation of research using motor imagery and action observation to applied and clinical settings.

A role for retro-splenial cortex in the task-related P3 network

OBJECTIVE

The P3 is an event-related response observed in relation to task-relevant sensory events. Despite its ubiquitous presence, the neural generators of the P3 are controversial and not well identified.

METHODS

We compared source analysis of combined magneto- and electroencephalography (M/EEG) data with functional magnetic resonance imaging (fMRI) and simulation studies to better understand the sources of the P3 in an auditory oddball paradigm.

RESULTS

Our results suggest that the dominant source of the classical, postero-central P3 lies in the retro-splenial cortex of the ventral cingulate gyrus. A second P3 source in the anterior insular cortex contributes little to the postero-central maximum. Multiple other sources in the auditory, somatosensory, and anterior midcingulate cortex are active in an overlapping time window but can be functionally dissociated based on their activation time courses.

CONCLUSIONS

The retro-splenial cortex is a dominant source of the parietal P3 maximum in EEG.

SIGNIFICANCE

These results provide a new perspective for the interpretation of the extensive research based on the P3 response.

Soft Bioelectronics for Neuroengineering: New Horizons in the Treatment of Brain Tumor and Epilepsy

Soft bioelectronic technologies for neuroengineering have shown remarkable progress, which include novel soft material technologies and device design strategies. Such technological advances that are initiated from fundamental brain science are applied to clinical neuroscience and provided meaningful promises for significant improvement in the diagnosis efficiency and therapeutic efficacy of various brain diseases recently. System-level integration strategies in consideration of specific disease circumstances can enhance treatment effects further. Here, recent advances in soft implantable bioelectronics for neuroengineering, focusing on materials and device designs optimized for the treatment of intracranial disease environments, are reviewed. Various types of soft bioelectronics for neuroengineering are categorized and exemplified first, and then details for the sensing and stimulating device components are explained. Next, application examples of soft implantable bioelectronics to clinical neuroscience, particularly focusing on the treatment of brain tumor and epilepsy are reviewed. Finally, an ideal system of soft intracranial bioelectronics such as closed-loop-type fully-integrated systems is presented, and the remaining challenges for their clinical translation are discussed.

GAHLS: an optimized graph analytics based high level synthesis framework

The urgent need for low latency, high-compute and low power on-board intelligence in autonomous systems, cyber-physical systems, robotics, edge computing, evolvable computing, and complex data science calls for determining the optimal amount and type of specialized hardware together with reconfigurability capabilities. With these goals in mind, we propose a novel comprehensive graph analytics based high level synthesis (GAHLS) framework that efficiently analyzes complex high level programs through a combined compiler-based approach and graph theoretic optimization and synthesizes them into message passing domain-specific accelerators. This GAHLS framework first constructs a compiler-assisted dependency graph (CaDG) from low level virtual machine (LLVM) intermediate representation (IR) of high level programs and converts it into a hardware friendly description representation. Next, the GAHLS framework performs a memory design space exploration while account for the identified computational properties from the CaDG and optimizing the system performance for higher bandwidth. The GAHLS framework also performs a robust optimization to identify the CaDG subgraphs with similar computational structures and aggregate them into intelligent processing clusters in order to optimize the usage of underlying hardware resources. Finally, the GAHLS framework synthesizes this compressed specialized CaDG into processing elements while optimizing the system performance and area metrics. Evaluations of the GAHLS framework on several real-life applications (e.g., deep learning, brain machine interfaces) demonstrate that it provides 14.27× performance improvements compared to state-of-the-art approaches such as LegUp 6.2.

TSPNet: a time-spatial parallel network for classification of EEG-based multiclass upper limb motor imagery BCI

The classification of electroencephalogram (EEG) motor imagery signals has emerged as a prominent research focus within the realm of brain-computer interfaces. Nevertheless, the conventional, limited categories (typically just two or four) offered by brain-computer interfaces fail to provide an extensive array of control modes. To address this challenge, we propose the Time-Spatial Parallel Network (TSPNet) for recognizing six distinct categories of upper limb motor imagery. Within TSPNet, temporal and spatial features are extracted separately, with the time dimension feature extractor and spatial dimension feature extractor performing their respective functions. Following this, the Time-Spatial Parallel Feature Extractor is employed to decouple the connection between temporal and spatial features, thus diminishing feature redundancy. The Time-Spatial Parallel Feature Extractor deploys a gating mechanism to optimize weight distribution and parallelize time-spatial features. Additionally, we introduce a feature visualization algorithm based on signal occlusion frequency to facilitate a qualitative analysis of TSPNet. In a six-category scenario, TSPNet achieved an accuracy of 49.1% ± 0.043 on our dataset and 49.7% ± 0.029 on a public dataset. Experimental results conclusively establish that TSPNet outperforms other deep learning methods in classifying data from these two datasets. Moreover, visualization results vividly illustrate that our proposed framework can generate distinctive classifier patterns for multiple categories of upper limb motor imagery, discerned through signals of varying frequencies. These findings underscore that, in comparison to other deep learning methods, TSPNet excels in intention recognition, which bears immense significance for non-invasive brain-computer interfaces.

Real-Time Deep Neurolinguistic Learning Enhances Noninvasive Neural Language Decoding for Brain-Machine Interaction

Electroencephalogram (EEG)-based brain-machine interface (BMI) has been utilized to help patients regain motor function and has recently been validated for its use in healthy people because of its ability to directly decipher human intentions. In particular, neurolinguistic research using EEGs has been investigated as an intuitive and naturalistic communication tool between humans and machines. In this study, the human mind directly decoded the neural languages based on speech imagery using the proposed deep neurolinguistic learning. Through real-time experiments, we evaluated whether BMI-based cooperative tasks between multiple users could be accomplished using a variety of neural languages. We successfully demonstrated a BMI system that allows a variety of scenarios, such as essential activity, collaborative play, and emotional interaction. This outcome presents a novel BMI frontier that can interact at the level of human-like intelligence in real time and extends the boundaries of the communication paradigm.

Boosting brain-computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome

Individuals with a locked-in state live with severe whole-body paralysis that limits their ability to communicate with family and loved ones. Recent advances in brain-computer interface (BCI) technology have presented a potential alternative for these people to communicate by detecting neural activity associated with attempted hand or speech movements and translating the decoded intended movements to a control signal for a computer. A technique that could potentially enrich the communication capacity of BCIs is functional electrical stimulation (FES) of paralyzed limbs and face to restore body and facial movements of paralyzed individuals, allowing to add body language and facial expression to communication BCI utterances. Here, we review the current state of the art of existing BCI and FES work in people with paralysis of body and face and propose that a combined BCI-FES approach, which has already proved successful in several applications in stroke and spinal cord injury, can provide a novel promising mode of communication for locked-in individuals.

CAMBA framework: Unveiling the brain asymmetry alterations and longitudinal changes after stroke using resting-state EEG

Hemispheric asymmetry or lateralization is a fundamental principle of brain organization. However, it is poorly understood to what extent the brain asymmetries across different levels of functional organizations are evident in health or altered in brain diseases. Here, we propose a framework that integrates three degrees of brain interactions (isolated nodes, node-node, and edge-edge) into a unified analysis pipeline to capture the sliding window-based asymmetry dynamics at both the node and hemisphere levels. We apply this framework to resting-state EEG in healthy and stroke populations and investigate the stroke-induced abnormal alterations in brain asymmetries and longitudinal asymmetry changes during poststroke rehabilitation. We observe that the mean asymmetry in patients was abnormally enhanced across different frequency bands and levels of brain interactions, with these abnormal patterns strongly associated with the side of the stroke lesion. Compared to healthy controls, patients displayed significant alterations in asymmetry fluctuations, disrupting and reconfiguring the balance of inter-hemispheric integration and segregation. Additionally, analyses reveal that specific abnormal asymmetry metrics in patients tend to move towards those observed in healthy controls after short-term brain-computer interface rehabilitation. Furthermore, preliminary evidence suggests that baseline clinical and asymmetry features can predict poststroke improvements in the Fugl-Meyer assessment of the lower extremity (mean absolute error of about 2). Overall, these findings advance our understanding of hemispheric asymmetry. Our framework offers new insights into the mechanisms underlying brain alterations and recovery after a brain lesion, may help identify prognostic biomarkers, and can be easily extended to different functional modalities.

An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth

Recent advancements in machine learning and deep learning (DL) based neural decoders have significantly improved decoding capabilities using scalp electroencephalography (EEG). However, the interpretability of DL models remains an under-explored area. In this study, we compared multiple model explanation methods to identify the most suitable method for EEG and understand when some of these approaches might fail. A simulation framework was developed to evaluate the robustness and sensitivity of twelve back-propagation-based visualization methods by comparing to ground truth features. Multiple methods tested here showed reliability issues after randomizing either model weights or labels: e.g., the saliency approach, which is the most used visualization technique in EEG, was not class or model-specific. We found that DeepLift was consistently accurate as well as robust to detect the three key attributes tested here (temporal, spatial, and spectral precision). Overall, this study provides a review of model explanation methods for DL-based neural decoders and recommendations to understand when some of these methods fail and what they can capture in EEG.

Application and Development of EEG Acquisition and Feedback Technology: A Review

This review focuses on electroencephalogram (EEG) acquisition and feedback technology and its core elements, including the composition and principles of the acquisition devices, a wide range of applications, and commonly used EEG signal classification algorithms. First, we describe the construction of EEG acquisition and feedback devices encompassing EEG electrodes, signal processing, and control and feedback systems, which collaborate to measure faint EEG signals from the scalp, convert them into interpretable data, and accomplish practical applications using control feedback systems. Subsequently, we examine the diverse applications of EEG acquisition and feedback across various domains. In the medical field, EEG signals are employed for epilepsy diagnosis, brain injury monitoring, and sleep disorder research. EEG acquisition has revealed associations between brain functionality, cognition, and emotions, providing essential insights for psychologists and neuroscientists. Brain-computer interface technology utilizes EEG signals for human-computer interaction, driving innovation in the medical, engineering, and rehabilitation domains. Finally, we introduce commonly used EEG signal classification algorithms. These classification tasks can identify different cognitive states, emotional states, brain disorders, and brain-computer interface control and promote further development and application of EEG technology. In conclusion, EEG acquisition technology can deepen the understanding of EEG signals while simultaneously promoting developments across multiple domains, such as medicine, science, and engineering.

A role for retro-splenial cortex in the task-related P3 network

Objective

The P3 is an event-related response observed in relation to task-relevant sensory events. Despite its ubiquitous presence, the neural generators of the P3 are controversial and not well identified.

Methods

We compared source analysis of combined magneto- and electroencephalography (M/EEG) data with functional magnetic resonance imaging (fMRI) and simulation studies to better understand the sources of the P3 in an auditory oddball paradigm.

Results

Our results suggest that the dominant source of the classical, postero-central P3 lies in the retro-splenial cortex of the ventral cingulate gyrus. A second P3 source in the anterior insular cortex contributes little to the postero-central maximum. Multiple other sources in the auditory, somatosensory, and anterior midcingulate cortex are active in an overlapping time window but can be functionally dissociated based on their activation time courses.

Conclusion

The retro-splenial cortex is a dominant source of the parietal P3 maximum in EEG.

Significance

These results provide a new perspective for the interpretation of the extensive research based on the P3 response.

TDLNet: Transfer Data Learning Network for Cross-Subject Classification Based on Multiclass Upper Limb Motor Imagery EEG

The limited number of brain-computer interface based on motor imagery (MI-BCI) instruction sets for different movements of single limbs makes it difficult to meet practical application requirements. Therefore, designing a single-limb, multi-category motor imagery (MI) paradigm and effectively decoding it is one of the important research directions in the future development of MI-BCI. Furthermore, one of the major challenges in MI-BCI is the difficulty of classifying brain activity across different individuals. In this article, the transfer data learning network (TDLNet) is proposed to achieve the cross-subject intention recognition for multiclass upper limb motor imagery. In TDLNet, the Transfer Data Module (TDM) is used to process cross-subject electroencephalogram (EEG) signals in groups and then fuse cross-subject channel features through two one-dimensional convolutions. The Residual Attention Mechanism Module (RAMM) assigns weights to each EEG signal channel and dynamically focuses on the EEG signal channels most relevant to a specific task. Additionally, a feature visualization algorithm based on occlusion signal frequency is proposed to qualitatively analyze the proposed TDLNet. The experimental results show that TDLNet achieves the best classification results on two datasets compared to CNN-based reference methods and transfer learning method. In the 6-class scenario, TDLNet obtained an accuracy of 65%±0.05 on the UML6 dataset and 63%±0.06 on the GRAZ dataset. The visualization results demonstrate that the proposed framework can produce distinct classifier patterns for multiple categories of upper limb motor imagery through signals of different frequencies. The ULM6 dataset is available at https://dx.doi.org/10.21227/8qw6-f578.

Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models

Objective.Development of brain-computer interface (BCI) technology is key for enabling communication in individuals who have lost the faculty of speech due to severe motor paralysis. A BCI control strategy that is gaining attention employs speech decoding from neural data. Recent studies have shown that a combination of direct neural recordings and advanced computational models can provide promising results. Understanding which decoding strategies deliver best and directly applicable results is crucial for advancing the field.Approach.In this paper, we optimized and validated a decoding approach based on speech reconstruction directly from high-density electrocorticography recordings from sensorimotor cortex during a speech production task.Main results.We show that (1) dedicated machine learning optimization of reconstruction models is key for achieving the best reconstruction performance; (2) individual word decoding in reconstructed speech achieves 92%-100% accuracy (chance level is 8%); (3) direct reconstruction from sensorimotor brain activity produces intelligible speech.Significance.These results underline the need for model optimization in achieving best speech decoding results and highlight the potential that reconstruction-based speech decoding from sensorimotor cortex can offer for development of next-generation BCI technology for communication.

Challenges of neural interfaces for stroke motor rehabilitation

More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.

Decoding Depression Severity From Intracranial Neural Activity

BACKGROUND

Disorders of mood and cognition are prevalent, disabling, and notoriously difficult to treat. Fueling this challenge in treatment is a significant gap in our understanding of their neurophysiological basis.

METHODS

We recorded high-density neural activity from intracranial electrodes implanted in depression-relevant prefrontal cortical regions in 3 human subjects with severe depression. Neural recordings were labeled with depression severity scores across a wide dynamic range using an adaptive assessment that allowed sampling with a temporal frequency greater than that possible with typical rating scales. We modeled these data using regularized regression techniques with region selection to decode depression severity from the prefrontal recordings.

RESULTS

Across prefrontal regions, we found that reduced depression severity is associated with decreased low-frequency neural activity and increased high-frequency activity. When constraining our model to decode using a single region, spectral changes in the anterior cingulate cortex best predicted depression severity in all 3 subjects. Relaxing this constraint revealed unique, individual-specific sets of spatiospectral features predictive of symptom severity, reflecting the heterogeneous nature of depression.

CONCLUSIONS

The ability to decode depression severity from neural activity increases our fundamental understanding of how depression manifests in the human brain and provides a target neural signature for personalized neuromodulation therapies.

A shallow mirror transformer for subject-independent motor imagery BCI

OBJECTIVE

Motor imagery BCI plays an increasingly important role in motor disorders rehabilitation. However, the position and duration of the discriminative segment in an EEG trial vary from subject to subject and even trial to trial, and this leads to poor performance of subject-independent motor imagery classification. Thus, determining how to detect and utilize the discriminative signal segments is crucial for improving the performance of subject-independent motor imagery BCI.

APPROACH

In this paper, a shallow mirror transformer is proposed for subject-independent motor imagery EEG classification. Specifically, a multihead self-attention layer with a global receptive field is employed to detect and utilize the discriminative segment from the entire input EEG trial. Furthermore, the mirror EEG signal and the mirror network structure are constructed to improve the classification precision based on ensemble learning. Finally, the subject-independent setup was used to evaluate the shallow mirror transformer on motor imagery EEG signals from subjects existing in the training set and new subjects.

MAIN RESULTS

The experiments results on BCI Competition IV datasets 2a and 2b and the OpenBMI dataset demonstrated the promising effectiveness of the proposed shallow mirror transformer. The shallow mirror transformer obtained average accuracies of 74.48% and 76.1% for new subjects and existing subjects, respectively, which were highest among the compared state-of-the-art methods. In addition, visualization of the attention score showed the ability of discriminative EEG segment detection. This paper demonstrated that multihead self-attention is effective in capturing global EEG signal information in motor imagery classification.

SIGNIFICANCE

This study provides an effective model based on a multihead self-attention layer for subject-independent motor imagery-based BCIs. To the best of our knowledge, this is the shallowest transformer model available, in which a small number of parameters promotes the performance in motor imagery EEG classification for such a small sample problem.

Research hotspots and trends of brain-computer interface technology in stroke: a bibliometric study and visualization analysis

Background

The incidence and mortality rates of stroke are escalating due to the growing aging population, which presents a significant hazard to human health. In the realm of stroke, brain-computer interface (BCI) technology has gained considerable attention as a means to enhance treatment efficacy and improve quality of life. Consequently, a bibliometric visualization analysis was performed to investigate the research hotspots and trends of BCI technology in stroke, with the objective of furnishing reference and guidance for future research.

Methods

This study utilized the Science Citation Index Expanded (SCI-Expanded) within the Web of Science Core Collection (WoSCC) database as the data source, selecting relevant literature published between 2013 and 2022 as research sample. Through the application of VOSviewer 1.6.19 and CiteSpace 6.2.R2 visualization analysis software, as well as the bibliometric online analysis platform, the scientific knowledge maps were constructed and subjected to visualization display, and statistical analysis.

Results

This study encompasses a total of 693 relevant literature, which were published by 2,556 scholars from 975 institutions across 53 countries/regions and have been collected by 185 journals. In the past decade, BCI technology in stroke research has exhibited an upward trend in both annual publications and citations. China and the United States are high productivity countries, while the University of Tubingen stands out as the most contributing institution. Birbaumer N and Pfurtscheller G are the authors with the highest publication and citation frequency in this field, respectively. Frontiers in Neuroscience has published the most literature, while Journal of Neural Engineering has the highest citation frequency. The research hotspots in this field cover keywords such as stroke, BCI, rehabilitation, motor imagery (MI), motor recovery, electroencephalogram (EEG), neurorehabilitation, neural plasticity, task analysis, functional electrical stimulation (FES), motor impairment, feature extraction, and induced movement therapy, which to a certain extent reflect the development trend and frontier research direction of this field.

Conclusion

This study comprehensively and visually presents the extensive and in-depth literature resources of BCI technology in stroke research in the form of knowledge maps, which facilitates scholars to gain a more convenient understanding of the development and prospects in this field, thereby promoting further research work.

Decoding the EEG patterns induced by sequential finger movement for brain-computer interfaces

Objective

In recent years, motor imagery-based brain-computer interfaces (MI-BCIs) have developed rapidly due to their great potential in neurological rehabilitation. However, the controllable instruction set limits its application in daily life. To extend the instruction set, we proposed a novel movement-intention encoding paradigm based on sequential finger movement.

Approach

Ten subjects participated in the offline experiment. During the experiment, they were required to press a key sequentially [i.e., Left→Left (LL), Right→Right (RR), Left→Right (LR), and Right→Left (RL)] using the left or right index finger at about 1 s intervals under an auditory prompt of 1 Hz. The movement-related cortical potential (MRCP) and event-related desynchronization (ERD) features were used to investigate the electroencephalography (EEG) variation induced by the sequential finger movement tasks. Twelve subjects participated in an online experiment to verify the feasibility of the proposed paradigm.

Main results

As a result, both the MRCP and ERD features showed the specific temporal-spatial EEG patterns of different sequential finger movement tasks. For the offline experiment, the average classification accuracy of the four tasks was 71.69%, with the highest accuracy of 79.26%. For the online experiment, the average accuracies were 83.33% and 82.71% for LL-versus-RR and LR-versus-RL, respectively.

Significance

This paper demonstrated the feasibility of the proposed sequential finger movement paradigm through offline and online experiments. This study would be helpful for optimizing the encoding method of motor-related EEG information and providing a promising approach to extending the instruction set of the movement intention-based BCIs.

Digital Alternative Communication for Individuals with Amyotrophic Lateral Sclerosis: What We Have

Amyotrophic Lateral Sclerosis is a disease that compromises the motor system and the functional abilities of the person in an irreversible way, causing the progressive loss of the ability to communicate. Tools based on Augmentative and Alternative Communication are essential for promoting autonomy and improving communication, life quality, and survival. This Systematic Literature Review aimed to provide evidence on eye-image-based Human-Computer Interaction approaches for the Augmentative and Alternative Communication of people with Amyotrophic Lateral Sclerosis. The Systematic Literature Review was conducted and guided following a protocol consisting of search questions, inclusion and exclusion criteria, and quality assessment, to select primary studies published between 2010 and 2021 in six repositories: Science Direct, Web of Science, Springer, IEEE Xplore, ACM Digital Library, and PubMed. After the screening, 25 primary studies were evaluated. These studies showcased four low-cost, non-invasive Human-Computer Interaction strategies employed for Augmentative and Alternative Communication in people with Amyotrophic Lateral Sclerosis. The strategies included Eye-Gaze, which featured in 36% of the studies; Eye-Blink and Eye-Tracking, each accounting for 28% of the approaches; and the Hybrid strategy, employed in 8% of the studies. For these approaches, several computational techniques were identified. For a better understanding, a workflow containing the development phases and the respective methods used by each strategy was generated. The results indicate the possibility and feasibility of developing Human-Computer Interaction resources based on eye images for Augmentative and Alternative Communication in a control group. The absence of experimental testing in people with Amyotrophic Lateral Sclerosis reiterates the challenges related to the scalability, efficiency, and usability of these technologies for people with the disease. Although challenges still exist, the findings represent important advances in the fields of health sciences and technology, promoting a promising future with possibilities for better life quality.

Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network

Introduction

In this study, we classified electroencephalography (EEG) data of imagined speech using signal decomposition and multireceptive convolutional neural network. The imagined speech EEG with five vowels /a/, /e/, /i/, /o/, and /u/, and mute (rest) sounds were obtained from ten study participants.

Materials and methods

First, two different signal decomposition methods were applied for comparison: noise-assisted multivariate empirical mode decomposition and wavelet packet decomposition. Six statistical features were calculated from the decomposed eight sub-frequency bands EEG. Next, all features obtained from each channel of the trial were vectorized and used as the input vector of classifiers. Lastly, EEG was classified using multireceptive field convolutional neural network and several other classifiers for comparison.

Results

We achieved an average classification rate of 73.09 and up to 80.41% in a multiclass (six classes) setup (Chance: 16.67%). In comparison with various other classifiers, significant improvements for other classifiers were achieved (p-value < 0.05). From the frequency sub-band analysis, high-frequency band regions and the lowest-frequency band region contain more information about imagined vowel EEG data. The misclassification and classification rate of each vowel imaginary EEG was analyzed through a confusion matrix.

Discussion

Imagined speech EEG can be classified successfully using the proposed signal decomposition method and a convolutional neural network. The proposed classification method for imagined speech EEG can contribute to developing a practical imagined speech-based brain-computer interfaces system.

Neural Decoding for Intracortical Brain-Computer Interfaces

Brain-computer interfaces have revolutionized the field of neuroscience by providing a solution for paralyzed patients to control external devices and improve the quality of daily life. To accurately and stably control effectors, it is important for decoders to recognize an individual's motor intention from neural activity either by noninvasive or intracortical neural recording. Intracortical recording is an invasive way of measuring neural electrical activity with high temporal and spatial resolution. Herein, we review recent developments in neural signal decoding methods for intracortical brain-computer interfaces. These methods have achieved good performance in analyzing neural activity and controlling robots and prostheses in nonhuman primates and humans. For more complex paradigms in motor rehabilitation or other clinical applications, there remains more space for further improvements of decoders.

Altered brain dynamics index levels of arousal in complete locked-in syndrome

Complete locked-in syndrome (CLIS) resulting from late-stage amyotrophic lateral sclerosis (ALS) is characterised by loss of motor function and eye movements. The absence of behavioural indicators of consciousness makes the search for neuronal correlates as possible biomarkers clinically and ethically urgent. EEG-based measures of brain dynamics such as power-law exponent (PLE) and Lempel-Ziv complexity (LZC) have been shown to have explanatory power for consciousness and may provide such neuronal indices for patients with CLIS. Here, we validated PLE and LZC (calculated in a dynamic way) as benchmarks of a wide range of arousal states across different reference states of consciousness (e.g., awake, sleep stages, ketamine, sevoflurane). We show a tendency toward high PLE and low LZC, with high intra-subject fluctuations and inter-subject variability in a cohort of CLIS patients with values graded along different arousal states as in our reference data sets. In conclusion, changes in brain dynamics indicate altered arousal in CLIS. Specifically, PLE and LZC are potentially relevant biomarkers to identify or diagnose the arousal level in CLIS and to determine the optimal time point for treatment, including communication attempts.

Tensor Decomposition Analysis of Longitudinal EEG Signals Reveals Differential Oscillatory Dynamics in Eyes-Closed and Eyes-Open Motor Imagery BCI: A Case Report

Functional dissociation of brain neural activity induced by opening or closing the eyes has been well established. However, how the temporal dynamics of the underlying neuronal modulations differ between these eye conditions during movement-related behaviours is less known. Using a robotic-assisted motor imagery brain-computer interface (MI BCI), we measured neural activity over the motor regions with electroencephalography (EEG) in a stroke survivor during his longitudinal rehabilitation training. We investigated lateralized oscillatory sensorimotor rhythm modulations while the patient imagined moving his hemiplegic hand with closed and open eyes to control an external robotic splint. In order to precisely identify the main profiles of neural activation affected by MI with eyes-open (MIEO) and eyes-closed (MIEC), a data-driven approach based on parallel factor analysis (PARAFAC) tensor decomposition was employed. Using the proposed framework, a set of narrow-band, subject-specific sensorimotor rhythms was identified; each of them had its own spatial and time signature. When MIEC trials were compared with MIEO trials, three key narrow-band rhythms whose peak frequencies centred at ∼8.0 Hz, ∼11.5 Hz, and ∼15.5 Hz, were identified with differently modulated oscillatory dynamics during movement preparation, initiation, and completion time frames. Furthermore, we observed that lower and higher sensorimotor oscillations represent different functional mechanisms within the MI paradigm, reinforcing the hypothesis that rhythmic activity in the human sensorimotor system is dissociated. Leveraging PARAFAC, this study achieves remarkable precision in estimating latent sensorimotor neural substrates, aiding the investigation of the specific functional mechanisms involved in the MI process.

EEG motor imagery classification using deep learning approaches in naïve BCI users

Motor Imagery (MI)-Brain Computer-Interfaces (BCI) illiteracy defines that not all subjects can achieve a good performance in MI-BCI systems due to different factors related to the fatigue, substance consumption, concentration, and experience in the use. To reduce the effects of lack of experience in the use of BCI systems (naïve users), this paper presents the implementation of three Deep Learning (DL) methods with the hypothesis that the performance of BCI systems could be improved compared with baseline methods in the evaluation of naïve BCI users. The methods proposed here are based on Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM)/Bidirectional Long Short-Term Memory (BiLSTM), and a combination of CNN and LSTM used for upper limb MI signal discrimination on a dataset of 25 naïve BCI users. The results were compared with three widely used baseline methods based on the Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP), and Filter Bank Common Spatial-Spectral Pattern (FBCSSP), in different temporal window configurations. As results, the LSTM-BiLSTM-based approach presented the best performance, according to the evaluation metrics of Accuracy, F-score, Recall, Specificity, Precision, and ITR, with a mean performance of 80% (maximum 95%) and ITR of 10 bits/min using a temporal window of 1.5 s. The DL Methods represent a significant increase of 32% compared with the baseline methods (p< 0.05). Thus, with the outcomes of this study, it is expected to increase the controllability, usability, and reliability of the use of robotic devices in naïve BCI users.

Bridging the BCI illiteracy gap: a subject-to-subject semantic style transfer for EEG-based motor imagery classification

Introduction

Brain-computer interfaces (BCIs) facilitate direct interaction between the human brain and computers, enabling individuals to control external devices through cognitive processes. Despite its potential, the problem of BCI illiteracy remains one of the major challenges due to inter-subject EEG variability, which hinders many users from effectively utilizing BCI systems. In this study, we propose a subject-to-subject semantic style transfer network (SSSTN) at the feature-level to address the BCI illiteracy problem in electroencephalogram (EEG)-based motor imagery (MI) classification tasks.

Methods

Our approach uses the continuous wavelet transform method to convert high-dimensional EEG data into images as input data. The SSSTN 1) trains a classifier for each subject, 2) transfers the distribution of class discrimination styles from the source subject (the best-performing subject for the classifier, i.e., BCI expert) to each subject of the target domain (the remaining subjects except the source subject, specifically BCI illiterates) through the proposed style loss, and applies a modified content loss to preserve the class-relevant semantic information of the target domain, and 3) finally merges the classifier predictions of both source and target subject using an ensemble technique.

Results and discussion

We evaluate the proposed method on the BCI Competition IV-2a and IV-2b datasets and demonstrate improved classification performance over existing methods, especially for BCI illiterate users. The ablation experiments and t-SNE visualizations further highlight the effectiveness of the proposed method in achieving meaningful feature-level semantic style transfer.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

A Methodology for Enhancing SSVEP Features Using Adaptive Filtering Based on the Spatial Distribution of EEG Signals

In this paper, we propose a classification algorithm of EEG signal based on canonical correlation analysis (CCA) and integrated with adaptive filtering. It can enhance the detection of steady-state visual evoked potentials (SSVEPs) in a brain-computer interface (BCI) speller. An adaptive filter is employed in front of the CCA algorithm to improve the signal-to-noise ratio (SNR) of SSVEP signals by removing background electroencephalographic (EEG) activities. The ensemble method is developed to integrate recursive least squares (RLS) adaptive filter corresponding to multiple stimulation frequencies. The method is tested by the SSVEP signal recorded from six targets by actual experiment and the EEG in a public SSVEP dataset of 40 targets from Tsinghua University. The accuracy rates of the CCA method and the CCA-based integrated RLS filter algorithm (RLS-CCA method) are compared. Experiment results show that the proposed RLS-CCA-based method significantly improves the classification accuracy compared with the pure CCA method. Especially when the number of EEG leads is low (three occipital electrodes and five non occipital electrodes), its advantage is more significant, and accuracy reaches 91.23%, which is more suitable for wearable environments where high-density EEG is not easy to collect.

Tailoring brain-machine interface rehabilitation training based on neural reorganization: towards personalized treatment for stroke patients

Electroencephalogram (EEG)-based brain-machine interface (BMI) has the potential to enhance rehabilitation training efficiency, but it still remains elusive regarding how to design BMI training for heterogeneous stroke patients with varied neural reorganization. Here, we hypothesize that tailoring BMI training according to different patterns of neural reorganization can contribute to a personalized rehabilitation trajectory. Thirteen stroke patients were recruited in a 2-week personalized BMI training experiment. Clinical and behavioral measurements, as well as cortical and muscular activities, were assessed before and after training. Following treatment, significant improvements were found in motor function assessment. Three types of brain activation patterns were identified during BMI tasks, namely, bilateral widespread activation, ipsilesional focusing activation, and contralesional recruitment activation. Patients with either ipsilesional dominance or contralesional dominance can achieve recovery through personalized BMI training. Results indicate that personalized BMI training tends to connect the potentially reorganized brain areas with event-contingent proprioceptive feedback. It can also be inferred that personalization plays an important role in establishing the sensorimotor loop in BMI training. With further understanding of neural rehabilitation mechanisms, personalized treatment strategy is a promising way to improve the rehabilitation efficacy and promote the clinical use of rehabilitation robots and other neurotechnologies.

A static paradigm based on illusion-induced VEP for brain-computer interfaces

OBJECTIVE

Visual evoked potentials (VEPs) have been commonly applied in brain-computer interfaces (BCIs) due to their satisfactory classification performance recently. However, most existing methods with flickering or oscillating stimuli will induce visual fatigue under long-term training, thus restricting the implementation of VEP-based BCIs. To address this issue, a novel paradigm adopting static motion illusion based on illusion-induced visual evoked potential (IVEP) is proposed for BCIs to enhance visual experience and practicality.

APPROACH

This study explored the responses to baseline and illusion tasks including the Rotating-Tilted-Lines (RTL) illusion and Rotating-Snakes (RS) illusion. The distinguishable features were examined between different illusions by analyzing the event-related potentials (ERPs) and amplitude modulation of evoked oscillatory responses.

MAIN RESULTS

The illusion stimuli elicited VEPs in an early time window encompassing a negative component (N1) from 110 to 200 ms and a positive component (P2) between 210 and 300 ms. Based on the feature analysis, a filter bank was designed to extract discriminative signals. The task-related component analysis (TRCA) was used to evaluate the binary classification task performance of the proposed method. Then the highest accuracy of 86.67% was achieved with a data length of 0.6 s.

SIGNIFICANCE

The results of this study demonstrate that the static motion illusion paradigm has the feasibility of implementation and is promising for VEP-based BCI applications.

Mental strategies and resting state EEG: Effect on high alpha amplitude modulation by neurofeedback in healthy young adults

Neurofeedback (NFB) is a brain-computer interface which allows individuals to modulate their brain activity. Despite the self-regulatory nature of NFB, the effectiveness of strategies used during NFB training has been little investigated. In a single session of NFB training (6*3 min training blocks) with healthy young participants, we experimentally tested if providing a list of mental strategies (list group, N = 46), compared with a group receiving no strategies (no list group, N = 39), affected participants' neuromodulation ability of high alpha (10-12 Hz) amplitude. We additionally asked participants to verbally report the mental strategies used to enhance high alpha amplitude. The verbatim was then classified in pre-established categories in order to examine the effect of type of mental strategy on high alpha amplitude. First, we found that giving a list to the participants did not promote the ability to neuromodulate high alpha activity. However, our analysis of the specific strategies reported by learners during training blocks revealed that cognitive effort and recalling memories were associated with higher high alpha amplitude. Furthermore, the resting amplitude of trained high alpha frequency predicted an amplitude increase during training, a factor that may optimize inclusion in NFB protocols. The present results also corroborate the interrelation with other frequency bands during NFB training. Although these findings are based on a single NFB session, our study represents a further step towards developing effective protocols for high alpha neuromodulation by NFB.