A functional source separation algorithm to enhance error-related potentials monitoring in noninvasive brain-computer interface

Fractal Dimension as a discriminative feature for high accuracy classification in motor imagery EEG-based brain-computer interface

BACKGROUND AND OBJECTIVE

The brain-computer interface (BCI) technology acquires human brain electrical signals, which can be effectively and successfully used to control external devices, potentially supporting subjects suffering from motor impairments in the interaction with the environment. To this aim, BCI systems must correctly decode and interpret neurophysiological signals reflecting the intention of the subjects to move. Therefore, the accurate classification of single events in motor tasks represents a fundamental challenge in ensuring efficient communication and control between users and BCIs. Movement-associated changes in electroencephalographic (EEG) sensorimotor rhythms, such as event-related desynchronization (ERD), are well-known features of discriminating motor tasks. Fractal dimension (FD) can be used to evaluate the complexity and self-similarity in brain signals, potentially providing complementary information to frequency-based signal features.

METHODS

In the present work, we introduce FD as a novel feature for subject-independent event classification, and we test several machine learning (ML) models in behavioural tasks of motor imagery (MI) and motor execution (ME).

RESULTS

Our results show that FD improves the classification accuracy of ML compared to ERD. Furthermore, unilateral hand movements have higher classification accuracy than bilateral movements in both MI and ME tasks.

CONCLUSIONS

These results provide further insights into subject-independent event classification in BCI systems and demonstrate the potential of FD as a discriminative feature for EEG signals.

Single trial detection of error-related potentials in brain-machine interfaces: a survey and comparison of methods

Objective.Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain-computer interfaces (BCIs), increase the naturalness of human-machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms).Approach.With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification.Main results.From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination).Significance.This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.

Dynamics of the "Cognitive" Brain Wave P3b at Rest for Alzheimer Dementia Prediction in Mild Cognitive Impairment

Alzheimer's disease (AD) is the most common cause of dementia that involves a progressive and irrevocable decline in cognitive abilities and social behavior, thus annihilating the patient's autonomy. The theoretical assumption that disease-modifying drugs are most effective in the early stages hopefully in the prodromal stage called mild cognitive impairment (MCI) urgently pushes toward the identification of robust and individualized markers of cognitive decline to establish an early pharmacological intervention. This requires the combination of well-established neural mechanisms and the development of increasingly sensitive methodologies. Among the neurophysiological markers of attention and cognition, one of the sub-components of the 'cognitive brain wave' P300 recordable in an odd-ball paradigm -namely the P3b- is extensively regarded as a sensitive indicator of cognitive performance. Several studies have reliably shown that changes in the amplitude and latency of the P3b are strongly related to cognitive decline and aging both healthy and pathological. Here, we used a P3b spatial filter to enhance the electroencephalographic (EEG) characteristics underlying 175 subjects divided into 135 MCI subjects, 20 elderly controls (EC), and 20 young volunteers (Y). The Y group served to extract the P3b spatial filter from EEG data, which was later applied to the other groups during resting conditions with eyes open and without being asked to perform any task. The group of 135 MCI subjects could be divided into two subgroups at the end of a month follow-up: 75 with stable MCI (MCI-S, not converted to AD), 60 converted to AD (MCI-C). The P3b spatial filter was built by means of a signal processing method called Functional Source Separation (FSS), which increases signal-to-noise ratio by using a weighted sum of all EEG recording channels rather than relying on a single, or a small sub-set, of channels. A clear difference was observed for the P3b dynamics at rest between groups. Moreover, a machine learning approach showed that P3b at rest could correctly distinguish MCI from EC (80.6% accuracy) and MCI-S from MCI-C (74.1% accuracy), with an accuracy as high as 93.8% in discriminating between MCI-C and EC. Finally, a comparison of the Bayes factor revealed that the group differences among MCI-S and MCI-C were 138 times more likely to be detected using the P3b dynamics compared with the best performing single electrode (Pz) approach. In conclusion, we propose that P3b as measured through spatial filters can be safely regarded as a simple and sensitive marker to predict the conversion from an MCI to AD status eventually combined with other non-neurophysiological biomarkers for a more precise definition of dementia having neuropathological Alzheimer characteristics.

A CNN-transformer hybrid approach for decoding visual neural activity into text

BACKGROUND AND OBJECTIVE

Most studies used neural activities evoked by linguistic stimuli such as phrases or sentences to decode the language structure. However, compared to linguistic stimuli, it is more common for the human brain to perceive the outside world through non-linguistic stimuli such as natural images, so only relying on linguistic stimuli cannot fully understand the information perceived by the human brain. To address this, an end-to-end mapping model between visual neural activities evoked by non-linguistic stimuli and visual contents is demanded.

METHODS

Inspired by the success of the Transformer network in neural machine translation and the convolutional neural network (CNN) in computer vision, here a CNN-Transformer hybrid language decoding model is constructed in an end-to-end fashion to decode functional magnetic resonance imaging (fMRI) signals evoked by natural images into descriptive texts about the visual stimuli. Specifically, this model first encodes a semantic sequence extracted by a two-layer 1D CNN from the multi-time visual neural activity into a multi-level abstract representation, then decodes this representation, step by step, into an English sentence.

RESULTS

Experimental results show that the decoded texts are semantically consistent with the corresponding ground truth annotations. Additionally, by varying the encoding and decoding layers and modifying the original positional encoding of the Transformer, we found that a specific architecture of the Transformer is required in this work.

CONCLUSIONS

The study results indicate that the proposed model can decode the visual neural activities evoked by natural images into descriptive text about the visual stimuli in the form of sentences. Hence, it may be considered as a potential computer-aided tool for neuroscientists to understand the neural mechanism of visual information processing in the human brain in the future.

Comparing between Different Sets of Preprocessing, Classifiers, and Channels Selection Techniques to Optimise Motor Imagery Pattern Classification System from EEG Pattern Recognition

The ability to control external devices through thought is increasingly becoming a reality. Human beings can use the electrical signals of their brain to interact or change the surrounding environment and more. The development of this technology called brain-computer interface (BCI) will increasingly allow people with motor disabilities to communicate or use assistive devices to walk, manipulate objects and communicate. Using data from the PhysioNet database, this study implemented a pattern classification system for use in a BCI on 109 healthy volunteers during real movement activities and motor imagery recorded by 64-channels electroencephalography (EEG) system. Different classifiers such as Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Trees (TREE) were applied on different combinations of EEG channels. Starting from two channels (C3, C4 and CP3 and CP4) positioned on the contralateral and ipsilateral sensorimotor cortex, the Region of Interest (RoI) centred on C3/Cp3 and C4/Cp4 and, finally, a data-driven automatic channels selection was tested to explore the best channel combination able to increase the classification accuracy. The results showed that the proposed automatic channels selection was able to significantly improve the performance of each classifier achieving 98% of accuracy for classification of real and imagined hand movement (sensitivity = 97%, specificity = 99%, AUC = 0.99) by SVM. While the accuracy of the classification between the imagery of hand and foot movements was 91% (sensitivity = 87%, specificity = 86%, AUC = 0.93) also with SVM. In the proposed approach, the data-driven automatic channels selection outperforms classical a priori channel selection models such as C3/C4, Cp3/Cp4, or RoIs around those channels with the utmost accuracy to help remove the boundaries of human communication and improve the quality of life of people with disabilities.

A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface

OBJECTIVE

Brain-computer interface (BCI) aims to establish communication paths between the brain processes and external devices. Different methods have been used to extract human intentions from electroencephalography (EEG) recordings. Those based on motor imagery (MI) seem to have a great potential for future applications. These approaches rely on the extraction of EEG distinctive patterns during imagined movements. Techniques able to extract patterns from raw signals represent an important target for BCI as they do not need labor-intensive data pre-processing.

APPROACH

We propose a new approach based on a 10-layer one-dimensional convolution neural network (1D-CNN) to classify five brain states (four MI classes plus a 'baseline' class) using a data augmentation algorithm and a limited number of EEG channels. In addition, we present a transfer learning method used to extract critical features from the EEG group dataset and then to customize the model to the single individual by training its outer layers with only 12-minute individual-related data.

MAIN RESULTS

The model tested with the 'EEG Motor Movement/Imagery Dataset' outperforms the current state-of-the-art models by achieving a 99.38% accuracy at the group level. In addition, the transfer learning approach we present achieves an average accuracy of 99.46%.

SIGNIFICANCE

The proposed methods could foster future BCI applications relying on few-channel portable recording devices and individual-based training.

Proof-of-concept evidence for trimodal simultaneous investigation of human brain function

The link between spatial (where) and temporal (when) aspects of the neural correlates of most psychological phenomena is not clear. Elucidation of this relation, which is crucial to fully understand human brain function, requires integration across multiple brain imaging modalities and cognitive tasks that reliably modulate the engagement of the brain systems of interest. By overcoming the methodological challenges posed by simultaneous recordings, the present report provides proof‐of‐concept evidence for a novel approach using three complementary imaging modalities: functional magnetic resonance imaging (fMRI), event‐related potentials (ERPs), and event‐related optical signals (EROS). Using the emotional oddball task, a paradigm that taps into both cognitive and affective aspects of processing, we show the feasibility of capturing converging and complementary measures of brain function that are not currently attainable using traditional unimodal or other multimodal approaches. This opens up unprecedented possibilities to clarify spatiotemporal integration of brain function.

Role of the Ipsilateral Primary Motor Cortex in the Visuo-Motor Network During Fine Contractions and Accurate Performance

It is widely recognized that continuous sensory feedback plays a crucial role in accurate motor control in everyday life. Feedback information is used to adapt force output and to correct errors. While primary motor cortex contralateral to the movement (cM1) plays a dominant role in this control, converging evidence supports the idea that ipsilateral primary motor cortex (iM1) also directly contributes to hand and finger movements. Similarly, when visual feedback is available, primary visual cortex (V1) and its interactions with the motor network also become important for accurate motor performance. To elucidate this issue, we performed and integrated behavioral and electroencephalography (EEG) measurements during isometric compression of a compliant rubber bulb, at 10% and 30% of maximum voluntary contraction, both with and without visual feedback. We used a semi-blind approach (functional source separation (FSS)) to identify separate functional sources of mu-frequency (8-13[Formula: see text]Hz) EEG responses in cM1, iM1 and V1. Here for the first time, we have used orthogonal FSS to extract multiple sources, by using the same functional constraint, providing the ability to extract different sources that oscillate in the same frequency range but that have different topographic distributions. We analyzed the single-trial timecourses of mu power event-related desynchronization (ERD) in these sources and linked them with force measurements to understand which aspects are most important for good task performance. Whilst the amplitude of mu power was not related to contraction force in any of the sources, it was able to provide information on performance quality. We observed stronger ERDs in both contralateral and ipsilateral motor sources during trials where contraction force was most consistently maintained. This effect was most prominent in the ipsilateral source, suggesting the importance of iM1 to accurate performance. This ERD effect was sustained throughout the duration of visual feedback trials, but only present at the start of no feedback trials, consistent with more variable performance in the absence of feedback. Overall, we found that the behavior of the ERD in iM1 was the most informative aspect concerning the accuracy of the contraction performance, and the ability to maintain a steady level of contraction. This new approach of using FSS to extract multiple orthogonal sources provides the ability to investigate both contralateral and ipsilateral nodes of the motor network without the need for additional information (e.g. electromyography). The enhanced signal-to-noise ratio provided by FSS opens the possibility of extracting complex EEG features on an individual trial basis, which is crucial for a more nuanced understanding of fine motor performance, as well as for applications in brain-computer interfacing.

Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review

BACKGROUND

Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015.

OBJECTIVES

We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic.

METHODS

We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends.

RESULTS

Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency.

SIGNIFICANCE

To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.

A Computationally Efficient Method for Hybrid EEG-fNIRS BCI Based on the Pearson Correlation

A hybrid brain computer interface (BCI) system considered here is a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). EEG-fNIRS signals are simultaneously recorded to achieve high motor imagery task classification. This integration helps to achieve better system performance, but at the cost of an increase in system complexity and computational time. In hybrid BCI studies, channel selection is recognized as the key element that directly affects the system's performance. In this paper, we propose a novel channel selection approach using the Pearson product-moment correlation coefficient, where only highly correlated channels are selected from each hemisphere. Then, four different statistical features are extracted, and their different combinations are used for the classification through KNN and Tree classifiers. As far as we know, there is no report available that explored the Pearson product-moment correlation coefficient for hybrid EEG-fNIRS BCI channel selection. The results demonstrate that our hybrid system significantly reduces computational burden while achieving a classification accuracy with high reliability comparable to the existing literature.