Multiscale Entropy Analysis for Recognition of Visually Elicited Negative Stress From EEG Recordings

Immersive media experience: a survey of existing methods and tools for human influential factors assessment

Virtual reality (VR) applications, especially those where the user is untethered to a computer, are becoming more prevalent as new hardware is developed, computational power and artificial intelligence algorithms are available, and wireless communication networks are becoming more reliable, fast, and providing higher reliability. In fact, recent projections show that by 2022 the number of VR users will double, suggesting the sector was not negatively affected by the worldwide COVID-19 pandemic. The success of any immersive communication system is heavily dependent on the user experience it delivers, thus now more than ever has it become crucial to develop reliable models of immersive media experience (IMEx). In this paper, we survey the literature for existing methods and tools to assess human influential factors (HIFs) related to IMEx. In particular, subjective, behavioural, and psycho-physiological methods are covered. We describe tools available to monitor these HIFs, including the user's sense of presence and immersion, cybersickness, and mental/affective states, as well as their role in overall experience. Special focus is placed on psycho-physiological methods, as it was found that such in-depth evaluation was lacking from the existing literature. We conclude by touching on emerging applications involving multiple-sensorial immersive media and provide suggestions for future research directions to fill existing gaps. It is hoped that this survey will be useful for researchers interested in building new immersive (adaptive) applications that maximize user experience.

Entropy and the Emotional Brain: Overview of a Research Field

During the last years, there has been a notable increase in the number of studies focused on the assessment of brain dynamics for the recognition of emotional states by means of nonlinear methodologies. More precisely, different entropy metrics have been applied for the analysis of electroencephalographic recordings for the detection of emotions. In this sense, regularity-based entropy metrics, symbolic predictability-based entropy indices, and different multiscale and multilag variants of the aforementioned methods have been successfully tested in a series of studies for emotion recognition from the EEG recording. This chapter aims to unify all those contributions to this scientific area, summarizing the main discoverings recently achieved in this research field.

Evaluation of Brain Functional Connectivity from Electroencephalographic Signals Under Different Emotional States

The identification of the emotional states corresponding to the four quadrants of the valence/arousal space has been widely analyzed in the scientific literature by means of multiple techniques. Nevertheless, most of these methods were based on the assessment of each brain region separately, without considering the possible interactions among different areas. In order to study these interconnections, this study computes for the first time the functional connectivity metric called cross-sample entropy for the analysis of the brain synchronization in four groups of emotions from electroencephalographic signals. Outcomes reported a strong synchronization in the interconnections among central, parietal and occipital areas, while the interactions between left frontal and temporal structures with the rest of brain regions presented the lowest coordination. These differences were statistically significant for the four groups of emotions. All emotions were simultaneously classified with a 95.43% of accuracy, overcoming the results reported in previous studies. Moreover, the differences between high and low levels of valence and arousal, taking into account the state of the counterpart dimension, also provided notable findings about the degree of synchronization in the brain within different emotional conditions and the possible implications of these outcomes from a psychophysiological point of view.

Wrist pulse diagnosis of stable coronary heart disease based on acoustics waveforms

BACKGROUND AND OBJECTIVE

As a common pathological pulse, unsmooth pulse has important diagnostic value in traditional Chinese medicine (TCM). In modern pulse diagnosis, unsmooth pulse plays an important role in the diagnosis of disease location and nature, but there are few studies on it. In this paper, a pulse diagnosis approach based on acoustic waveforms was proposed, the wrist pulse was divided into five layers vertically for the first time. Five layers acoustic waves of the radial artery in stable coronary heart disease (CHD) patients and relatively healthy people were compared to explore whether there are abnormal changes in acoustic pulse in stable CHD patients.

METHODS

The acoustic features of unsmooth pulse in patients with stable CHD were analyzed in time domain, frequency domain and empirical mode decomposition, combined with shannon entropy and multi-scale entropy. Sixteen pulse characteristics were discovered, and one-way analysis of variance were performed. The characteristics of the two groups were tested by T test. 13 features were used to identify patients with stable CHD by support vector machine (SVM).

RESULTS

Compared to healthy people, all parameters of the third layer of the stable CHD left Cun pulse were significantly different from those of the healthy people. The identification rates of the fourth and third layer of the left Cun pulse were the first (90.79%) and the second (88.16%), respectively.

CONCLUSION

Abnormal acoustic pulse appeared in the radial artery in patients with stable CHD. According to these changes, patients with stable CHD can be effectively identified from the perspective of pulse.

Conditional Entropy: A Potential Digital Marker for Stress

Recent decades have witnessed a substantial progress in the utilization of brain activity for the identification of stress digital markers. In particular, the success of entropic measures for this purpose is very appealing, considering (1) their suitability for capturing both linear and non-linear characteristics of brain activity recordings and (2) their direct association with the brain signal variability. These findings rely on external stimuli to induce the brain stress response. On the other hand, research suggests that the use of different types of experimentally induced psychological and physical stressors could potentially yield differential impacts on the brain response to stress and therefore should be dissociated from more general patterns. The present study takes a step toward addressing this issue by introducing conditional entropy (CE) as a potential electroencephalography (EEG)-based resting-state digital marker of stress. For this purpose, we use the resting-state multi-channel EEG recordings of 20 individuals whose responses to stress-related questionnaires show significantly higher and lower level of stress. Through the application of representational similarity analysis (RSA) and K-nearest-neighbor (KNN) classification, we verify the potential that the use of CE can offer to the solution concept of finding an effective digital marker for stress.

Detection of Epileptic Seizure Using Pretrained Deep Convolutional Neural Network and Transfer Learning

INTRODUCTION

The diagnosis of epilepsy takes a certain process, depending entirely on the attending physician. However, the human factor may cause erroneous diagnosis in the analysis of the EEG signal. In the past 2 decades, many advanced signal processing and machine learning methods have been developed for the detection of epileptic seizures. However, many of these methods require large data sets and complex operations.

METHODS

In this study, an end-to-end machine learning model is presented for detection of epileptic seizure using the pretrained deep two-dimensional convolutional neural network (CNN) and the concept of transfer learning. The EEG signal is converted directly into visual data with a spectrogram and used directly as input data.

RESULTS

The authors analyzed the results of the training of the proposed pretrained AlexNet CNN model. Both binary and ternary classifications were performed without any extra procedure such as feature extraction. By performing data set creation from short-term spectrogram graphic images, the authors were able to achieve 100% accuracy for binary classification for epileptic seizure detection and 100% for ternary classification.

DISCUSSION/CONCLUSION

The proposed automatic identification and classification model can help in the early diagnosis of epilepsy, thus providing the opportunity for effective early treatment.

A New dispersion entropy and fuzzy logic system methodology for automated classification of dementia stages using electroencephalograms

A new EEG-based methodology is presented for differential diagnosis of the Alzheimer's disease (AD), Mild Cognitive Impairment (MCI), and healthy subjects employing the discrete wavelet transform (DWT), dispersion entropy index (DEI), a recently-proposed nonlinear measurement, and a fuzzy logic-based classification algorithm. The effectiveness and usefulness of the proposed methodology are evaluated by employing a database of measured EEG data acquired from 135 subjects, 45 MCI, 45 AD and 45 healthy subjects. The proposed methodology differentiates MCI and AD patients from HC subjects with an accuracy of 82.6-86.9%, sensitivity of 91 %, and specificity of 87 %.

The Role of Features Types and Personalized Assessment in Detecting Affective State Using Dry Electrode EEG

Assessing the human affective state using electroencephalography (EEG) have shown good potential but failed to demonstrate reliable performance in real-life applications. Especially if one applies a setup that might impact affective processing and relies on generalized models of affect. Additionally, using subjective assessment of ones affect as ground truth has often been disputed. To shed the light on the former challenge we explored the use of a convenient EEG system with 20 participants to capture their reaction to affective movie clips in a naturalistic setting. Employing state-of-the-art machine learning approach demonstrated that the highest performance is reached when combining linear features, namely symmetry features and single-channel features, with nonlinear ones derived by a multiscale entropy approach. Nevertheless, the best performance, reflected in the highest F1-score achieved in a binary classification task for valence was 0.71 and for arousal 0.62. The performance was 10–20% better compared to using ratings provided by 13 independent raters. We argue that affective self-assessment might be underrated and it is crucial to account for personal differences in both perception and physiological response to affective cues.

Development of a Neurodegenerative Disease Gait Classification Algorithm Using Multiscale Sample Entropy and Machine Learning Classifiers

The prevalence of neurodegenerative diseases (NDD) has grown rapidly in recent years and NDD screening receives much attention. NDD could cause gait abnormalities so that to screen NDD using gait signal is feasible. The research aim of this study is to develop an NDD classification algorithm via gait force (GF) using multiscale sample entropy (MSE) and machine learning models. The Physionet NDD gait database is utilized to validate the proposed algorithm. In the preprocessing stage of the proposed algorithm, new signals were generated by taking one and two times of differential on GF and are divided into various time windows (10/20/30/60-sec). In feature extraction, the GF signal is used to calculate statistical and MSE values. Owing to the imbalanced nature of the Physionet NDD gait database, the synthetic minority oversampling technique (SMOTE) was used to rebalance data of each class. Support vector machine (SVM) and k-nearest neighbors (KNN) were used as the classifiers. The best classification accuracies for the healthy controls (HC) vs. Parkinson’s disease (PD), HC vs. Huntington’s disease (HD), HC vs. amyotrophic lateral sclerosis (ALS), PD vs. HD, PD vs. ALS, HD vs. ALS, HC vs. PD vs. HD vs. ALS, were 99.90%, 99.80%, 100%, 99.75%, 99.90%, 99.55%, and 99.68% under 10-sec time window with KNN. This study successfully developed an NDD gait classification based on MSE and machine learning classifiers.

A Novel Method for Sleep-Stage Classification Based on Sonification of Sleep Electroencephalogram Signals Using Wavelet Transform and Recurrent Neural Network

INTRODUCTION

Visual sleep-stage scoring is a time-consuming technique that cannot extract the nonlinear characteristics of electroencephalogram (EEG). This article presents a novel method for sleep-stage differentiation based on sonification of sleep-EEG signals using wavelet transform and recurrent neural network (RNN).

METHODS

Two RNNs were designed and trained separately based on a database of classical guitar pieces and Kurdish tanbur Makams using a long short-term memory model. Moreover, discrete wavelet transform and wavelet packet decomposition were used to determine the association between the EEG signals and musical pitches. Continuous wavelet transform was applied to extract musical beat-based features from the EEG. Then, the pretrained RNN was used to generate music. To test the proposed model, 11 sleep EEGs were mapped onto the guitar and tanbur frequency intervals and presented to the pretrained RNN. Next, the generated music was randomly presented to 2 neurologists.

RESULTS

The proposed model classified the sleep stages with an accuracy of >81% for tanbur and more than 93% for guitar musical pieces. The inter-rater reliability measured by Cohen's kappa coefficient (κ) revealed good reliability for both tanbur (κ = 0.64, p < 0.001) and guitar musical pieces (κ = 0.85, p < 0.001).

CONCLUSION

The present EEG sonification method leads to valid sleep staging by clinicians. The method could be used on various EEG databases for classification, differentiation, diagnosis, and treatment purposes. Real-time EEG sonification can be used as a feedback tool for replanning of neurophysiological functions for the management of many neurological and psychiatric disorders in the future.

Single-trial EEG emotion recognition using Granger Causality/Transfer Entropy analysis

BACKGROUND

Emotion recognition has been studied for decades, but the classification accuracy needs to be improved.

NEW METHOD

In this study, a novel emotional classification approach is proposed by combining the Histogram of Oriented Gradient (HOG) method with the Granger Causality (GC) or Transfer Entropy (TE) methods. HOG extracts local valid information from the GC/TE relationship matrices and then the Support Vector Machine (SVM) is employed to classify the emotional states of stress and calm.

RESULTS

Compared with previous studies, the classification accuracy has been greatly improved. The results of this study show that the classification based on GC or TE with HOG offers an average accuracy 88.93 % and 95.21 %, respectively. The achieved accuracy is about 12 % higher than that achieved without using HOG feature extraction.

COMPARISON WITH EXISTING METHOD(S)

Numerous efforts have been devoted to classify emotional states by extracting EEG characteristics on a single channel basis, the method developed in this study utilizes information interaction between brain channels as a feature to classify emotional states. Furthermore, this study combines HOG and relation matrices for the first time and uses image processing to extract EEG features.

CONCLUSION

Our results demonstrate the feasibility of combining TE with HOG for emotion recognition with improved classification accuracy by taking advantage of both network and gradient features. More specific features can be selected to improve classification accuracy by taking advantage of information exchanges between EEG channels directly or the extracted property of the relationship matrix based on information interactions.

Multiscaled Neural Autoregressive Distributed Lag: A New Empirical Mode Decomposition Model for Nonlinear Time Series Forecasting

Forecasting has always been the cornerstone of machine learning and statistics. Despite the great evolution of the time series theory, forecasters are still in the hunt for better models to make more accurate decisions. The huge advances in neural networks over the last years has led to the emergence of a new generation of effective models replacing classic econometric models. It is in this direction that we propose, in this paper, a new multiscaled Feedforward Neural Network (FNN), with the aim of forecasting multivariate time series. This new model, called Empirical Mode Decomposition (EMD)-based Neural ARDL, is inspired from the well-known Autoregressive Distributed Lag (ARDL) model being our proposal founded upon the concepts of nonlinearity, EMD-multiresolution and neural networks. These features give the model the ability to effectively capture many nonlinear patterns like the ones often present in econophysical time series, such as nonlinear trends, seasonal effects, long-range dependency, etc. The proposed algorithm can be summarized into the following four basic tasks: (i) EMD breaking-down multivariate time series into different resolution levels, (ii) feeding EMD components from the same levels into a number of feedforward neural ARDL models, (iii) from one level to the next, extrapolating the component corresponding to the response variable (scalar output) a number of steps ahead, and finally, (iv) recombining level-by-level forecasts into a single output. An optimal learning scheme is rigorously designed for efficiently training the new proposed architecture. The approach is finally tested and compared to a number of powerful benchmark models, where experiments are conducted on real-world data.

Deep Support Vector Machines for the Identification of Stress Condition from Electrodermal Activity

Early detection of stress condition is beneficial to prevent long-term mental illness like depression and anxiety. This paper introduces an accurate identification of stress/calm condition from electrodermal activity (EDA) signals. The acquisition of EDA signals from a commercial wearable as well as their storage and processing are presented. Several time-domain, frequency-domain and morphological features are extracted over the skin conductance response of the EDA signals. Afterwards, a classification is undergone by using several classical support vector machines (SVMs) and deep support vector machines (D-SVMs). In addition, several binary classifiers are also compared with SVMs in the stress/calm identification task. Moreover, a series of video clips evoking calm and stress conditions have been viewed by 147 volunteers in order to validate the classification results. The highest F1-score obtained for SVMs and D-SVMs are 83% and 92%, respectively. These results demonstrate that not only classical SVMs are appropriate for classification of biomarker signals, but D-SVMs are very competitive in comparison to other classification techniques. In addition, the results have enabled drawing useful considerations for the future use of SVMs and D-SVMs in the specific case of stress/calm identification.

EEG Connectivity Analysis Using Denoising Autoencoders for the Detection of Dyslexia

The Temporal Sampling Framework (TSF) theorizes that the characteristic phonological difficulties of dyslexia are caused by an atypical oscillatory sampling at one or more temporal rates. The LEEDUCA study conducted a series of Electroencephalography (EEG) experiments on children listening to amplitude modulated (AM) noise with slow-rythmic prosodic (0.5-1[Formula: see text]Hz), syllabic (4-8[Formula: see text]Hz) or the phoneme (12-40[Formula: see text]Hz) rates, aimed at detecting differences in perception of oscillatory sampling that could be associated with dyslexia. The purpose of this work is to check whether these differences exist and how they are related to children's performance in different language and cognitive tasks commonly used to detect dyslexia. To this purpose, temporal and spectral inter-channel EEG connectivity was estimated, and a denoising autoencoder (DAE) was trained to learn a low-dimensional representation of the connectivity matrices. This representation was studied via correlation and classification analysis, which revealed ability in detecting dyslexic subjects with an accuracy higher than 0.8, and balanced accuracy around 0.7. Some features of the DAE representation were significantly correlated ([Formula: see text]) with children's performance in language and cognitive tasks of the phonological hypothesis category such as phonological awareness and rapid symbolic naming, as well as reading efficiency and reading comprehension. Finally, a deeper analysis of the adjacency matrix revealed a reduced bilateral connection between electrodes of the temporal lobe (roughly the primary auditory cortex) in DD subjects, as well as an increased connectivity of the F7 electrode, placed roughly on Broca's area. These results pave the way for a complementary assessment of dyslexia using more objective methodologies such as EEG.

A Novel Microwave Treatment for Sleep Disorders and Classification of Sleep Stages Using Multi-Scale Entropy

The aim of this study was to develop an integrated system of non-contact sleep stage detection and sleep disorder treatment for health monitoring. Hence, a method of brain activity detection based on microwave scattering technology instead of scalp electroencephalogram was developed to evaluate the sleep stage. First, microwaves at a specific frequency were used to penetrate the functional sites of the brain in patients with sleep disorders to change the firing frequency of the activated areas of the brain and analyze and evaluate statistically the effects on sleep improvement. Then, a wavelet packet algorithm was used to decompose the microwave transmission signal, the refined composite multiscale sample entropy, the refined composite multiscale fluctuation-based dispersion entropy and multivariate multiscale weighted permutation entropy were obtained as features from the wavelet packet coefficient. Finally, the mutual information-principal component analysis feature selection method was used to optimize the feature set and random forest was used to classify and evaluate the sleep stage. The results show that after four times of microwave modulation treatment, sleep efficiency improved continuously, the overall maintenance was above 80%, and the insomnia rate was reduced gradually. The overall classification accuracy of the four sleep stages was 86.4%. The results indicate that the microwaves with a certain frequency can treat sleep disorders and detect abnormal brain activity. Therefore, the microwave scattering method is of great significance in the development of a new brain disease treatment, diagnosis and clinical application system.

Correlation of EEG spectra, connectivity, and information theoretical biomarkers with psychological states in the epilepsy monitoring unit - A pilot study

At the level of individual experience, the relation between electroencephalographic (EEG) phenomena and subjective ratings of psychological states is poorly examined. This study investigated the correlation of quantitative EEG markers with systematic high-frequency monitoring of psychological states in patients admitted to the epilepsy monitoring unit (EMU). We used a digital questionnaire, including eight standardized items about stress, energy level, mood, ward atmosphere, seizure likelihood, hopefulness/frustration, boredom, and self-efficacy. Self-assessments were collected four times per day, in total 15 times during the stay in the EMU. We extracted brainrate, Hjorth parameters, Hurst exponent, Wackermann parameters, and power spectral density from the EEG. We performed correlation between these quantitative EEG measures and responses to the 8 items and evaluated their significance on single subject and on group level. Twenty-one consecutive patients (12 women/9 men, median age: 29 years, range: 18-74 years) were recruited. On group level, no significant correlations were found whereas on single-subject level, we found significant correlations for 6 out of 21 patients. Most significant correlations were found between Hjorth parameters and items that reflect changes in mood or stress. This study supports the feasibility of correlating quantitative EEG measures with psychological states in routine EMU settings and emphasizes the need for single-subject statistics when assessing aspects with high interindividual variance. Future studies should select samples with high within-subject variability of psychological states and examine a subsample with patients encountering a critical number of seizures needed in order to relate the psychological states to the ultimate question: Are psychological states potential indicators for seizure likelihood?

Multi-Lag Analysis of Symbolic Entropies on EEG Recordings for Distress Recognition

Distress is a critical problem in developed societies given its long-term negative effects on physical and mental health. The interest in studying this emotion has notably increased during last years, being electroencephalography (EEG) signals preferred over other physiological variables in this research field. In addition, the non-stationary nature of brain dynamics has impulsed the use of non-linear metrics, such as symbolic entropies in brain signal analysis. Thus, the influence of time-lag on brain patterns assessment has not been tested. Hence, in the present study two permutation entropies denominated Delayed Permutation Entropy and Permutation Min-Entropy have been computed for the first time at different time-lags to discern between emotional states of calmness and distress from EEG signals. Moreover, a number of curve-related features were also calculated to assess brain dynamics across different temporal intervals. Complementary information among these variables was studied through sequential forward selection and 10-fold cross-validation approaches. According to the results obtained, the multi-lag entropy analysis has been able to reveal new significant insights so far undiscovered, thus notably improving the process of distress recognition from EEG recordings.

Artificial Neural Networks to Assess Emotional States from Brain-Computer Interface

Estimation of human emotions plays an important role in the development of modern brain-computer interface devices like the Emotiv EPOC+ headset. In this paper, we present an experiment to assess the classification accuracy of the emotional states provided by the headset’s application programming interface (API). In this experiment, several sets of images selected from the International Affective Picture System (IAPS) dataset are shown to sixteen participants wearing the headset. Firstly, the participants’ responses in form of a self-assessment manikin questionnaire to the emotions elicited are compared with the validated IAPS predefined valence, arousal and dominance values. After statistically demonstrating that the responses are highly correlated with the IAPS values, several artificial neural networks (ANNs) based on the multilayer perceptron architecture are tested to calculate the classification accuracy of the Emotiv EPOC+ API emotional outcomes. The best result is obtained for an ANN configuration with three hidden layers, and 30, 8 and 3 neurons for layers 1, 2 and 3, respectively. This configuration offers 85% classification accuracy, which means that the emotional estimation provided by the headset can be used with high confidence in real-time applications that are based on users’ emotional states. Thus the emotional states given by the headset’s API may be used with no further processing of the electroencephalogram signals acquired from the scalp, which would add a level of difficulty.