EEG-inception: an accurate and robust end-to-end neural network for EEG-based motor imagery classification

Classification of EEG-based motor imagery (MI) is a crucial non-invasive application in brain-computer interface (BCI) research. This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based MI classification that outperforms the state-of-the-art methods. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network, which has showed to be highly efficient and accurate for time-series classification. Also, the proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing. Furthermore, this paper proposes a novel data augmentation method for EEG signals to enhance the accuracy, at least by 3%, and reduce overfitting with limited BCI datasets. The proposed model outperforms all state-of-the-art methods by achieving the average accuracy of 88.4% and 88.6% on the 2008 BCI Competition IV 2a (four-classes) and 2b datasets (binary-classes), respectively. Furthermore, it takes less than 0.025 seconds to test a sample suitable for real-time processing. Moreover, the classification standard deviation for nine different subjects achieves the lowest value of 5.5 for the 2b dataset and 7.1 for the 2a dataset, which validates that the proposed method is highly robust. From the experiment results, it can be inferred that the EEG-Inception network exhibits a strong potential as a subject-independent classifier for EEG-based MI tasks.

Dynamic Joint Domain Adaptation Network for Motor Imagery Classification

Electroencephalogram (EEG) has been widely used in brain computer interface (BCI) due to its convenience and reliability. The EEG-based BCI applications are majorly limited by the time-consuming calibration procedure for discriminative feature representation and classification. Existing EEG classification methods either heavily depend on the handcrafted features or require adequate annotated samples at each session for calibration. To address these issues, we propose a novel dynamic joint domain adaptation network based on adversarial learning strategy to learn domain-invariant feature representation, and thus improve EEG classification performance in the target domain by leveraging useful information from the source session. Specifically, we explore the global discriminator to align the marginal distribution across domains, and the local discriminator to reduce the conditional distribution discrepancy between sub-domains via conditioning on deep representation as well as the predicted labels from the classifier. In addition, we further investigate a dynamic adversarial factor to adaptively estimate the relative importance of alignment between the marginal and conditional distributions. To evaluate the efficacy of our method, extensive experiments are conducted on two public EEG datasets, namely, Datasets IIa and IIb of BCI Competition IV. The experimental results demonstrate that the proposed method achieves superior performance compared with the state-of-the-art methods.

Development of a Human-Display Interface with Vibrotactile Feedback for Real-World Assistive Applications

It is important to operate devices with control panels and touch screens assisted by haptic feedback in mobile environments such as driving automobiles and electric power wheelchairs. A lot of consideration is needed to give accurate haptic feedback, especially, presenting clear touch feedback to the elderly and people with reduced sensation is a very critical issue from healthcare and safety perspectives. In this study, we aimed to identify the perceptual characteristics for the frequency and direction of haptic vibration on the touch screen with vehicle-driving vibration and to propose an efficient haptic system based on these characteristics. As a result, we demonstrated that the detection threshold shift decreased at frequencies above 210 Hz due to the contact pressure during active touch, but the detection threshold shift increased at below 210 Hz. We found that the detection thresholds were 0.30–0.45 gpeak with similar sensitivity in the 80–270 Hz range. The haptic system implemented by reflecting the experimental results achieved characteristics suitable for use scenarios in automobiles. Ultimately, it could provide practical guidelines for the development of touch screens to give accurate touch feedback in the real-world environment.

Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review

BACKGROUND

Brain-Computer Interface (BCI) is becoming more reliable, thanks to the advantages of Artificial Intelligence (AI). Recently, hybrid Deep Learning (hDL), which combines different DL algorithms, has gained momentum over the past five years. In this work, we proposed a review on hDL-based BCI starting from the seminal studies in 2015.

OBJECTIVES

We have reviewed 47 papers that apply hDL to the BCI system published between 2015 and 2020 extracting trends and highlighting relevant aspects to the topic.

METHODS

We have queried four scientific search engines (Google Scholar, PubMed, IEEE Xplore and Elsevier Science Direct) and different data items were extracted from each paper such as the database used, kind of application, online/offline training, tasks used for the BCI, pre-processing methodology adopted, type of normalization used, which kind of features were extracted, type of DL architecture used, number of layers implemented and which optimization approach were used as well. All these items were then investigated one by one to uncover trends.

RESULTS

Our investigation reveals that Electroencephalography (EEG) has been the most used technique. Interestingly, despite the lower Signal-to-Noise Ratio (SNR) of the EEG data that makes pre-processing of that data mandatory, we have found that the pre-processing has only been used in 21.28% of the cases by showing that hDL seems to be able to overcome this intrinsic drawback of the EEG data. Temporal-features seem to be the most effective with 93.94% accuracy, while spatial-temporal features are the most used with 33.33% of the cases investigated. The most used architecture has been Convolutional Neural Network-Recurrent Neural Network CNN-RNN with 47% of the cases. Moreover, half of the studies have used a low number of layers to achieve a good compromise between the complexity of the network and computational efficiency.

SIGNIFICANCE

To give useful information to the scientific community, we make our summary table of hDL-based BCI papers available and invite the community to published work to contribute to it directly. We have indicated a list of open challenges, emphasizing the need to use neuroimaging techniques other than EEG, such as functional Near-Infrared Spectroscopy (fNIRS), deeper investigate the advantages and disadvantages of using pre-processing and the relationship with the accuracy obtained. To implement new combinations of architectures, such as RNN-based and Deep Belief Network DBN-based, it is necessary to better explore the frequency and temporal-frequency features of the data at hand.

Neural decoding of continuous upper limb movements: a meta-analysis

OBJECTIVE

EEG-based motion trajectory decoding makes a promising approach for neurotechnology which can be used for neural control of motion reconstruction and neurorehabilitation tools. However, the feasibility and validity of continuous motion decoding by non-invasive brain activity are not clear. The main aim of this study was to perform a meta-analysis across studies that examined the ability of EEG-based continuous motion decoding of upper limb movements.

APPROACH

Pearson's correlation coefficient (CC) was used to evaluate the model performance of the studies and considered as an effect size. To estimate the overall effect size of neural decoding of motion trajectory across studies, characteristics of included studies were addressed and the random effect model was applied to the heterogeneous studies which estimated overall effect size distribution. Furthermore, the significant difference between the two subgroups of imagined and executed movements was analysed.

MAIN RESULTS

The mean of the overall effect size was computed 0.46 across the nonhomogeneous studies. The results showed no significant difference between imagined and executed movements (Chi²=0.28, df = 1, p = 0.60).

SIGNIFICANCE

Meta-analysis results confirm that imagination like execution movements can be used for neural decoding of motion trajectory in neural motor control systems. Also, nonlinear compare with linear model statistically confirmed to be more beneficial for complex movements. Furthermore, a new approach of synergy-based motion decoding can be significantly effective to increase model performance and more research needs to evaluate this method for different levels of complexity of movements.IMPLICATIONS FOR REHABILITATIONNeural decoding methods base on EEG as a non-invasive brain activity, are more user friendly for neurorehabilitation than invasive methods that developing of it makes it more applicable for reconstructing activities of daily living.Neurotechnology for neural control of motion reconstruction, makes the rehabilitation tools to be more synchrony with human intentional movement that can be used to improve the brain neuroplastisity in stroke or other paralysed people.The feasibility and validity of imagined movements equal with executed movements show that amputee people also can benefit EEG-based motion decoding for controling rehabilitation tools just by imagination of their intentional movements.For neurorehabilitation tools, comparing the study outcomes illucidate that the approach of synergy-based motor control in brain activities concluded significantly high performance that highlighted the need it to more investigated in future research.

Decoding of Grasp Motions from EEG Signals Based on a Novel Data Augmentation Strategy

Electroencephalogram (EEG) based braincomputer interface (BCI) systems are useful tools for clinical purposes like neural prostheses. In this study, we collected EEG signals related to grasp motions. Five healthy subjects participated in this experiment. They executed and imagined five sustained-grasp actions. We proposed a novel data augmentation method that increases the amount of training data using labels obtained from electromyogram (EMG) signals analysis. For implementation, we recorded EEG and EMG simultaneously. The data augmentation over the original EEG data concluded higher classification accuracy than other competitors. As a result, we obtained the average classification accuracy of 52.49(±8.74)% for motor execution (ME) and 40.36(±3.39)% for motor imagery (MI). These are 9.30% and 6.19% higher, respectively than the result of the comparable methods. Moreover, the proposed method could minimize the need for the calibration session, which reduces the practicality of most BCIs. This result is encouraging, and the proposed method could potentially be used in future applications such as a BCI-driven robot control for handling various daily use objects.

Multimodal signal dataset for 11 intuitive movement tasks from single upper extremity during multiple recording sessions

BACKGROUND

Non-invasive brain-computer interfaces (BCIs) have been developed for realizing natural bi-directional interaction between users and external robotic systems. However, the communication between users and BCI systems through artificial matching is a critical issue. Recently, BCIs have been developed to adopt intuitive decoding, which is the key to solving several problems such as a small number of classes and manually matching BCI commands with device control. Unfortunately, the advances in this area have been slow owing to the lack of large and uniform datasets. This study provides a large intuitive dataset for 11 different upper extremity movement tasks obtained during multiple recording sessions. The dataset includes 60-channel electroencephalography, 7-channel electromyography, and 4-channel electro-oculography of 25 healthy participants collected over 3-day sessions for a total of 82,500 trials across all the participants.

FINDINGS

We validated our dataset via neurophysiological analysis. We observed clear sensorimotor de-/activation and spatial distribution related to real-movement and motor imagery, respectively. Furthermore, we demonstrated the consistency of the dataset by evaluating the classification performance of each session using a baseline machine learning method.

CONCLUSIONS

The dataset includes the data of multiple recording sessions, various classes within the single upper extremity, and multimodal signals. This work can be used to (i) compare the brain activities associated with real movement and imagination, (ii) improve the decoding performance, and (iii) analyze the differences among recording sessions. Hence, this study, as a Data Note, has focused on collecting data required for further advances in the BCI technology.