Dendrite morphological neural networks for motor task recognition from electroencephalographic signals

Research of intent recognition in rehabilitation robots: a systematic review

PURPOSE

Rehabilitation robots with intent recognition are helping people with dysfunction to enjoy better lives. Many rehabilitation robots with intent recognition have been developed by academic institutions and commercial companies. However, there is no systematic summary about the application of intent recognition in the field of rehabilitation robots. Therefore, the purpose of this paper is to summarize the application of intent recognition in rehabilitation robots, analyze the current status of their research, and provide cutting-edge research directions for colleagues.

MATERIALS AND METHODS

Literature searches were conducted on Web of Science, IEEE Xplore, ScienceDirect, SpringerLink, and Medline. Search terms included "rehabilitation robot", "intent recognition", "exoskeleton", "prosthesis", "surface electromyography (sEMG)" and "electroencephalogram (EEG)". References listed in relevant literature were further screened according to inclusion and exclusion criteria.

RESULTS

In this field, most studies have recognized movement intent by kinematic, sEMG, and EEG signals. However, in practical studies, the development of intent recognition in rehabilitation robots is limited by the hysteresis of kinematic signals and the weak anti-interference ability of sEMG and EEG signals.

CONCLUSIONS

Intent recognition has achieved a lot in the field of rehabilitation robotics but the key factors limiting its development are still timeliness and accuracy. In the future, intent recognition strategy with multi-sensor information fusion may be a good solution.

Learning smooth dendrite morphological neurons by stochastic gradient descent for pattern classification

This article presents a learning algorithm for dendrite morphological neurons (DMN) based on stochastic gradient descent (SGD). In particular, we focus on a DMN topology that comprises spherical dendrites, smooth maximum activation function nodes, and a softmax output layer, whose original learning algorithm is performed in two independent stages: (1) dendrites' centroids are learned by k-means, and (2) softmax layer weights are adjusted by gradient descent. A drawback of this learning method is that both stages are unplugged; once dendrites' centroids are defined, they keep static during weights learning, so no feedback is performed to correct the dendrites' positions to improve classification performance. To overcome this issue, we derive the delta rules for adjusting the dendrites' centroids and the output layer weights by minimizing the cross-entropy loss function under an SGD scheme. This gradient descent-based learning is feasible because the smooth maximum activation function that interfaces the dendrite units with the output layer is differentiable. The proposed DMN is compared against eight morphological neuron models with distinct topologies and learning methods and four well-established classifiers: support vector machine (SVM), multilayer perceptron (MLP), and random forest (RF), and k-nearest neighbors (k-NN). Besides, the classification performance is evaluated on 81 datasets. The experimental results show that the proposed method tends to outperform the DMN methods and is competitive or even better than SVM, MLP, RF, and k-NN. Thus, it is an alternative approach that can effectively be used for pattern classification. Moreover, SGD for DMN learning standardizes this neural model, like current artificial neural networks.

Artificial intelligence based multimodal language decoding from brain activity: A review

Decoding brain activity is conducive to the breakthrough of brain-computer interface (BCI) technology. The development of artificial intelligence (AI) continually promotes the progress of brain language decoding technology. Existent research has mainly focused on a single modality and paid insufficient attention to AI methods. Therefore, our objective is to provide an overview of relevant decoding research from the perspective of different modalities and methodologies. The modalities involve text, speech, image, and video, whereas the core method is using AI-built decoders to translate brain signals induced by multimodal stimuli into text or vocal language. The semantic information of brain activity can be successfully decoded into a language at various levels, ranging from words through sentences to discourses. However, the decoding effect is affected by various factors, such as the decoding model, vector representation model, and brain regions. Challenges and future directions are also discussed. The advances in brain language decoding and BCI technology will potentially assist patients with clinical aphasia in regaining the ability to communicate.

Classification of EEG signals related to real and imagery knee movements using deep learning for brain computer interfaces

BACKGROUND

Non-invasive Brain-Computer Interface (BCI) uses an electroencephalogram (EEG) to obtain information on brain neural activity. Because EEG can be contaminated by various artifacts during the collection process, it has primarily evolved into motor imagery (MI) with a low risk of contamination. However, MI has a disadvantage in that accurate data is difficult to obtain.

OBJECTIVE

The goal of this study was to determine which motor imagery and movement execution (ME) of the knee has the best classification performance.

METHODS

Ten subjects were selected to provide MI and ME data for four different types of knee exercise. The experiment was conducted to keep the left, right, and both knees extend or bend for five seconds, and there was a five seconds break between each movement. Each motion was performed 20 times and the MI was carried out in the same protocol. Motions were classified through a modified model of the Lenet-5 of CNN (Convolution Neural Network).

RESULTS

The deep learning data was classified, and a study discovered that ME (98.91%) could be classified significantly more accurately than MI (98.37%) (p< 0.001).

CONCLUSION

If future studies on other body movements are conducted, we anticipate that BCI can be further developed to be more accurate. And such advancements in BCI can be used to facilitate the patient's communication by analyzing the user's movement intention. These results can also be used for various controls such as robots using a combination of MI and ME.

A State-of-the-Art Review of EEG-Based Imagined Speech Decoding

Currently, the most used method to measure brain activity under a non-invasive procedure is the electroencephalogram (EEG). This is because of its high temporal resolution, ease of use, and safety. These signals can be used under a Brain Computer Interface (BCI) framework, which can be implemented to provide a new communication channel to people that are unable to speak due to motor disabilities or other neurological diseases. Nevertheless, EEG-based BCI systems have presented challenges to be implemented in real life situations for imagined speech recognition due to the difficulty to interpret EEG signals because of their low signal-to-noise ratio (SNR). As consequence, in order to help the researcher make a wise decision when approaching this problem, we offer a review article that sums the main findings of the most relevant studies on this subject since 2009. This review focuses mainly on the pre-processing, feature extraction, and classification techniques used by several authors, as well as the target vocabulary. Furthermore, we propose ideas that may be useful for future work in order to achieve a practical application of EEG-based BCI systems toward imagined speech decoding.

Feature- and classification analysis for detection and classification of tongue movements from single-trial pre-movement EEG

Individuals with severe tetraplegia can benefit from brain-computer interfaces (BCIs). While most movement-related BCI systems focus on right/left hand and/or foot movements, very few studies have considered tongue movements to construct a multiclass BCI. The aim of this study was to decode four movement directions of the tongue (left, right, up, and down) from single-trial pre-movement EEG and provide a feature and classifier investigation. In offline analyses (from ten healthy participants) detection and classification were performed using temporal, spectral, entropy, and template features classified using either a linear discriminative analysis, support vector machine, random forest or multilayer perceptron classifiers. Besides the 4-class classification scenario, all possible 3-, and 2-class scenarios were tested to find the most discriminable movement type. The linear discriminant analysis achieved on average, higher classification accuracies for both movement detection and classification. The right- and down tongue movements provided the highest and lowest detection accuracy (95.3±4.3% and 91.7±4.8%), respectively. The 4-class classification achieved an accuracy of 62.6±7.2%, while the best 3-class classification (using left, right, and up movements) and 2-class classification (using left and right movements) achieved an accuracy of 75.6±8.4% and 87.7±8.0%, respectively. Using only a combination of the temporal and template feature groups provided further classification accuracy improvements. Presumably, this is because these feature groups utilize the movement-related cortical potentials, which are noticeably different on the left- versus right brain hemisphere for the different movements. This study shows that the cortical representation of the tongue is useful for extracting control signals for multi-class movement detection BCIs.

Identification of Visual Imagery by Electroencephalography Based on Empirical Mode Decomposition and an Autoregressive Model

The traditional imagery task for brain–computer interfaces (BCIs) consists of motor imagery (MI) in which subjects are instructed to imagine moving certain parts of their body. This kind of imagery task is difficult for subjects. In this study, we used a less studied yet more easily performed type of mental imagery—visual imagery (VI)—in which subjects are instructed to visualize a picture in their brain to implement a BCI. In this study, 18 subjects were recruited and instructed to observe one of two visual-cued pictures (one was static, while the other was moving) and then imagine the cued picture in each trial. Simultaneously, electroencephalography (EEG) signals were collected. Hilbert–Huang Transform (HHT), autoregressive (AR) models, and a combination of empirical mode decomposition (EMD) and AR were used to extract features, respectively. A support vector machine (SVM) was used to classify the two kinds of VI tasks. The average, highest, and lowest classification accuracies of HHT were 68.14 ± 3.06%, 78.33%, and 53.3%, respectively. The values of the AR model were 56.29 ± 2.73%, 71.67%, and 30%, respectively. The values obtained by the combination of the EMD and the AR model were 78.40 ± 2.07%, 87%, and 48.33%, respectively. The results indicate that multiple VI tasks were separable based on EEG and that the combination of EMD and an AR model used in VI feature extraction was better than an HHT or AR model alone. Our work may provide ideas for the construction of a new online VI-BCI.

Neural Decoding of Chinese Sign Language With Machine Learning for Brain-Computer Interfaces

Limb motion decoding is an important part of brain-computer interface (BCI) research. Among the limb motion, sign language not only contains rich semantic information and abundant maneuverable actions but also provides different executable commands. However, many researchers focus on decoding the gross motor skills, such as the decoding of ordinary motor imagery or simple upper limb movements. Here we explored the neural features and decoding of Chinese sign language from electroencephalograph (EEG) signal with motor imagery and motor execution. Sign language not only contains rich semantic information, but also has abundant maneuverable actions, and provides us with more different executable commands. In this paper, twenty subjects were instructed to perform movement execution and movement imagery based on Chinese sign language. Seven classifiers are employed to classify the selected features of sign language EEG. L1 regularization is used to learn and select features that contain more information from the mean, power spectral density, sample entropy, and brain network connectivity. The best average classification accuracy of the classifier is 89.90% (imagery sign language is 83.40%). These results have shown the feasibility of decoding between different sign languages. The source location reveals that the neural circuits involved in sign language are related to the visual contact area and the pre-movement area. Experimental evaluation shows that the proposed decoding strategy based on sign language can obtain outstanding classification results, which provides a certain reference value for the subsequent research of limb decoding based on sign language.

A facile and flexible motor imagery classification using electroencephalogram signals

BACKGROUND

Mind machine interface (MMI) enables communication with milieu by measuring brain activities. The reliability of MMI systems is highly dependent on the identification of various motor imagery (MI) tasks. Perfect discrimination of brain activities is required to avoid miscommunication. Electroencephalogram (EEG) signals provide a scrupulous solution for the development of MMI. Analysis of multi-channel EEG signals increases the burden of computation drastically. The extraction of hidden information from raw EEG signals is difficult due to its complex nature. A signal is needed to be decomposed and classified for the extraction of hidden information from it. But selecting the uniform decomposition and hyperparameters for decomposition and classification of the signal can lead to information loss and misclassification.

METHOD

This paper presents a novel method for identifying right-hand and right-foot MI tasks. The method employs a single-channel adaptive decomposition and EEG signal classification. The multi-cluster unsupervised learning method is employed for the selection of significant channel. Further, flexible variational mode decomposition (F-VMD) is used for the adaptive decomposition of signals. The values of decomposition parameters are selected adaptively following the nature of EEG signals. The value of decomposition parameters is used to decompose the signals into narrow-band modes. Hjorth, entropy and quartile based features are elicited from the modes of F-VMD. These features are classified by using a flexible extreme learning machine (F-ELM). F-ELM selects the hyperparameters and kernel adaptively by reducing the classification error.

RESULTS

The performance of the proposed method is evaluated by measuring five performance parameters namely accuracy (ACC), sensitivity (SEN), specificity (SPE), Mathew's correlation coefficient (MCC), and F-1 score. An ACC, SEN, SPE, MCC and F-1 score is obtained as 100%, 100%, 100%, 100%, and 1. The performance parameters obtained by the proposed method prove the superiority over other methodologies using the same data-set.

CONCLUSION

The proposed method proved to be promising and efficient with a single channel and two features. This framework can be utilized for the development of a real-time mind-machine interface like robotic arm, wheel chairs, etc.

A Lasso quantile periodogram based feature extraction for EEG-based motor imagery

BACKGROUND

The extraction of relevant and distinct features from the electroencephalogram (EEG) signals is one of the most challenging task when implementing Brain Computer Interface (BCI) based systems. Frequency analysis techniques are recognised as one of the most suitable methods to have distinct information from EEG signals. However, existing studies use mostly classical approaches assuming that the signal is Gaussian, stationary and linear. These properties are not verified in the EEG case considering the complexity of the brain electrical activity.

NEW METHOD

This paper proposes two new spectral estimators that are robust against non-Gaussian, non-linear and non-stationary signals. These two approaches use quantile regression and L₁-norm regularisation to estimate the spectrum of the motor imagery (MI) related EEG.

RESULTS

A dataset collected during a study of BCI motor imagery project conducted at Tshwane University of Technology (TUT), Pretoria, South Africa, is used to validate the proposed estimators. Experimental results demonstrate that the newly proposed approaches help improve the classification performance of MI.

COMPARISON WITH EXISTING METHODS

In order to show the effectiveness of the proposed estimators, a comparative study is conducted, considering classical commonly used techniques such as FFT and Welch periodogram through 5 classification algorithms.

CONCLUSIONS

The proposed Quantile-based spectral estimators are potential methods to improve the classification performance of the EEG-Based motor imagery systems.

Individual-Specific Classification of Mental Workload Levels Via an Ensemble Heterogeneous Extreme Learning Machine for EEG Modeling

In a human–machine cooperation system, assessing the mental workload (MW) of the human operator is quite crucial to maintaining safe operation conditions. Among various MW indicators, electroencephalography (EEG) signals are particularly attractive because of their high temporal resolution and sensitivity to the occupation of working memory. However, the individual difference of the EEG feature distribution may impair the machine-learning based MW classifier. In this paper, we employed a fast-training neural network, extreme learning machine (ELM), as the basis to build an individual-specific classifier ensemble to recognize binary MW. To improve the diversity of the classification committee, heterogeneous member classifiers were adopted by fusing multiple ELMs and Bayesian models. Specifically, a deep network structure was applied in each weak model aiming at finding informative EEG feature representations. The structure of hyper-parameters of the proposed heterogeneous ensemble ELM (HE-ELM) was then identified and then its performance was compared against several competitive MW classifiers. We found that the HE-ELM model was superior for improving the individual-specific accuracy of MW assessments.