Recognition of EEG Signals from Imagined Vowels Using Deep Learning Methods

Decoding imagined speech with delay differential analysis

Speech decoding from non-invasive EEG signals can achieve relatively high accuracy (70-80%) for strictly delimited classification tasks, but for more complex tasks non-invasive speech decoding typically yields a 20-50% classification accuracy. However, decoder generalization, or how well algorithms perform objectively across datasets, is complicated by the small size and heterogeneity of existing EEG datasets. Furthermore, the limited availability of open access code hampers a comparison between methods. This study explores the application of a novel non-linear method for signal processing, delay differential analysis (DDA), to speech decoding. We provide a systematic evaluation of its performance on two public imagined speech decoding datasets relative to all publicly available deep learning methods. The results support DDA as a compelling alternative or complementary approach to deep learning methods for speech decoding. DDA is a fast and efficient time-domain open-source method that fits data using only few strong features and does not require extensive preprocessing.

The role of vowel and consonant onsets in neural tracking of natural speech

Objective. To investigate how the auditory system processes natural speech, models have been created to relate the electroencephalography (EEG) signal of a person listening to speech to various representations of the speech. Mainly the speech envelope has been used, but also phonetic representations. We investigated to which degree of granularity phonetic representations can be related to the EEG signal.Approach. We used recorded EEG signals from 105 subjects while they listened to fairy tale stories. We utilized speech representations, including onset of any phone, vowel-consonant onsets, broad phonetic class (BPC) onsets, and narrow phonetic class onsets, and related them to EEG using forward modeling and match-mismatch tasks. In forward modeling, we used a linear model to predict EEG from speech representations. In the match-mismatch task, we trained a long short term memory based model to determine which of two candidate speech segments matches with a given EEG segment.Main results. Our results show that vowel-consonant onsets outperform onsets of any phone in both tasks, which suggests that neural tracking of the vowel vs. consonant exists in the EEG to some degree. We also observed that vowel (syllable nucleus) onsets exhibit a more consistent representation in EEG compared to syllable onsets.Significance. Finally, our findings suggest that neural tracking previously thought to be associated with BPCs might actually originate from vowel-consonant onsets rather than the differentiation between different phonetic classes.

Advanced Modeling and Signal Processing Methods in Brain-Computer Interfaces Based on a Vector of Cyclic Rhythmically Connected Random Processes

In this study is substantiated the new mathematical model of vector of electroencephalographic signals, registered under the conditions of multiple repetitions of the mental control influences of brain-computer interface operator, in the form of a vector of cyclic rhythmically connected random processes, which, due to taking into account the stochasticity and cyclicity, the variability and commonality of the rhythm of the investigated signals have a number of advantages over the known models. This new model opens the way for the study of multidimensional distribution functions; initial, central, and mixed moment functions of higher order such as for each electroencephalographic signal separately; as well as for their respective compatible probabilistic characteristics, among which the most informative characteristics can be selected. This provides an increase in accuracy in the detection (classification) of mental control influences of the brain-computer interface operators. Based on the developed mathematical model, the statistical processing methods of vector of electroencephalographic signals are substantiated, which consist of statistical evaluation of its probabilistic characteristics and make it possible to conduct an effective joint statistical estimation of the probability characteristics of electroencephalographic signals. This provides the basis for coordinated integration of information from different sensors. The use of moment functions of higher order and their spectral images in the frequency domain, as informative characteristics in brain-computer interface systems, are substantiated. Their significant sensitivity to the mental controlling influence of the brain-computer interface operator is experimentally established. The application of Bessel's inequality to the problems of reducing the dimensions (from 500 to 20 numbers) of the vectors of informative features makes it possible to significantly reduce the computational complexity of the algorithms for the functioning of brain-computer interface systems. Namely, we experimentally established that only the first 20 values of the Fourier transform of the estimation of moment functions of higher-order electroencephalographic signals are sufficient to form the vector of informative features in brain-computer interface systems, because these spectral components make up at least 95% of the total energy of the corresponding statistical estimate of the moment functions of higher-order electroencephalographic signals.

Multiclass Classification of Imagined Speech Vowels and Words of Electroencephalography Signals Using Deep Learning

The paper’s emphasis is on the imagined speech decoding of electroencephalography (EEG) neural signals of individuals in accordance with the expansion of the brain-computer interface to encompass individuals with speech problems encountering communication challenges. Decoding an individual’s imagined speech from nonstationary and nonlinear EEG neural signals is a complex task. Related research work in the field of imagined speech has revealed that imagined speech decoding performance and accuracy require attention to further improve. The evolution of deep learning technology increases the likelihood of decoding imagined speech from EEG signals with enhanced performance. We proposed a novel supervised deep learning model that combined the temporal convolutional networks and the convolutional neural networks with the intent of retrieving information from the EEG signals. The experiment was carried out using an open-access dataset of fifteen subjects’ imagined speech multichannel signals of vowels and words. The raw multichannel EEG signals of multiple subjects were processed using discrete wavelet transformation technique. The model was trained and evaluated using the preprocessed signals, and the model hyperparameters were adjusted to achieve higher accuracy in the classification of imagined speech. The experiment results demonstrated that the multiclass imagined speech classification of the proposed model exhibited a higher overall accuracy of 0.9649 and a classification error rate of 0.0350. The results of the study indicate that individuals with speech difficulties might well be able to leverage a noninvasive EEG-based imagined speech brain-computer interface system as one of the long-term alternative artificial verbal communication mediums.

A Novel Stress State Assessment Method for College Students Based on EEG

Stress is an unavoidable problem for today's college students. Stress can arouse strong personal emotional and behavioral responses. Compared with other groups of the same age, college students have a special way of life and living environment. They have complex interpersonal relationships and relatively weak social support systems. At the same time, they also face fierce competition in both academic and employment. However, they lack the skills to deal with the crisis and are reluctant to ask others for help, which leads to a simultaneous increase in mental stress. The pressure on college students mainly comes from study, family, social, employment, society, and economy. When students face multiple pressures from family, school, society, etc., some students are prone to some psychological problems due to their own personality or external environment and other reasons. Therefore, regular assessment of students' stress status is an important means to prevent college students' psychological problems. Considering that in real life, the number of students whose pressure is within the tolerable range is the majority, while the number of students who are under too much pressure is a minority. Therefore, the actual dataset to be identified belongs to a kind of imbalanced data. In this study, an improved extreme learning machine (IELM) is used to improve the performance of the recognition model as much as possible. IELM takes the idea of label weighting as the starting point, introduces the AdaBoost algorithm, and combines its weight distribution with the label weighted extreme learning machine (ELM). During the weight update process, the advantage of the imbalanced nature of multi-label datasets is taken. IELM was used to classify EEG data to determine the stress level of college students. The experimental results demonstrate that the algorithm used in this study has excellent classification performance and can accurately assess students' stress levels. The accurate assessment of stress has provided a solid foundation for the development of students' mental health and has significant practical implications.