An accurate emotion recognition system using ECG and GSR signals and matching pursuit method

Background

The purpose of the current study was to examine the effectiveness of Matching Pursuit (MP) algorithm in emotion recognition.

Methods

Electrocardiogram (ECG) and galvanic skin responses (GSR) of 11 healthy students were collected while subjects were listening to emotional music clips. Applying three dictionaries, including two wavelet packet dictionaries (Coiflet, and Daubechies) and discrete cosine transform, MP coefficients were extracted from ECG and GSR signals. Next, some statistical indices were calculated from the MP coefficients. Then, three dimensionality reduction methods, including Principal Component Analysis (PCA), Linear Discriminant Analysis, and Kernel PCA were applied. The dimensionality reduced features were fed into the Probabilistic Neural Network in subject-dependent and subject-independent modes. Emotion classes were described by a two-dimensional emotion space, including four quadrants of valence and arousal plane, valence based, and arousal based emotional states.

Results

Using PCA, the highest recognition rate of 100% was achieved for sigma = 0.01 in all classification schemes. In addition, the classification performance of ECG features was evidently better than that of GSR features. Similar results were obtained for subject-dependent emotion classification mode.

Conclusions

An accurate emotion recognition system was proposed using MP algorithm and wavelet dictionaries.

Fusion of heart rate variability and pulse rate variability for emotion recognition using lagged poincare plots

Designing an efficient automatic emotion recognition system based on physiological signals has attracted great interests within the research of human-machine interactions. This study was aimed to classify emotional responses by means of a simple dynamic signal processing technique and fusion frameworks. The electrocardiogram and finger pulse activity of 35 participants were recorded during rest condition and when subjects were listening to music intended to stimulate certain emotions. Four emotion categories, including happiness, sadness, peacefulness, and fear were chosen. Estimating heart rate variability (HRV) and pulse rate variability (PRV), 4 Poincare indices in 10 lags were extracted. The support vector machine classifier was used for emotion classification. Both feature level (FL) and decision level (DL) fusion schemes were examined. Significant differences have been observed between lag 1 Poincare plot indices and the other lagged measures. The mean accuracies of 84.1, 82.9, 79.68, and 76.05% were obtained for PRV, DL, FL, and HRV measures, respectively. However, DL outperformed others in discriminating sadness and peacefulness, using SD₁ and total features, correspondingly. In both cases, the classification rates improved up to 92% (with the sensitivity of 95% and specificity of 83.33%). Totally, DL resulted in better performances compared to FL. In addition, the impact of the fusion rules on the classification performances has been confirmed.

EEG spectral powers and source localization in depressing, sad, and fun music videos focusing on gender differences

Previously, gender-specific affective responses have been shown using neurophysiological signals. The present study intended to compare the differences in electroencephalographic (EEG) power spectra and EEG brain sources between men and women during the exposure of affective music video stimuli. The multi-channel EEG signals of 15 males and 15 females available in the database for emotion analysis using physiological signals were studied, while subjects were watching sad, depressing, and fun music videos. Seven EEG frequency bands were computed using average Fourier cross-spectral matrices. Then, standardized low-resolution electromagnetic tomography (sLORETA) was used to localize regions involved specifically in these emotional responses. To evaluate gender differences, independent sample t test was calculated for the sLORETA source powers. Our results showed that (1) the mean EEG power for all frequency bands in the women's group was significantly higher than that of the men's group; (2) spatial distribution differentiation between men and women was detected in all EEG frequency bands. (3) This difference has been related to the emotional stimuli, which was more evident for negative emotions. Taken together, our results showed that men and women recruited dissimilar brain networks for processing sad, depressing, and fun audio-visual stimuli.

Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination

Interest in human emotion recognition, regarding physiological signals, has recently risen. In this study, an efficient emotion recognition system, based on geometrical analysis of autonomic nervous system signals, is presented. The electrocardiogram recordings of 47 college students were obtained during rest condition and affective visual stimuli. Pictures with four emotional contents, including happiness, peacefulness, sadness, and fear were selected. Then, ten lags of Poincare plot were constructed for heart rate variability (HRV) segments. For each lag, five geometrical indices were extracted. Next, these features were fed into an automatic classification system for the recognition of the four affective states and rest condition. The results showed that the Poincare plots have different shapes for different lags, as well as for different affective states. Considering higher lags, the greatest increment in SD₁ and decrements in SD₂ occurred during the happiness stimuli. In contrast, the minimum changes in the Poincare measures were perceived during the fear inducements. Therefore, the HRV geometrical shapes and dynamics were altered by the positive and negative values of valence-based emotion dimension. Using a probabilistic neural network, a maximum recognition rate of 97.45% was attained. Applying the proposed methodology based on lagged Poincare indices, a valuable tool for discriminating the emotional states was provided.

The potential of photoplethysmogram and galvanic skin response in emotion recognition using nonlinear features

Recently, developing an accurate automatic emotion recognition system using a minimum number of bio-signals has become a challenging issue in "affective computing." This study aimed to propose a reliable system by examining nonlinear dynamics of photoplethysmogram (PPG) and galvanic skin response (GSR). To address this goal, two strategies were adopted. First, the efficiency of each signal in valence/arousal based emotion categorization was examined. Then, the proficiency of a hybrid feature, by combining both GSR and PPG features was studied. Lyapunov exponents, lagged Poincare's measures, and approximate entropy were extracted to characterize the irregularity and chaotic behavior of the phase space. To discriminate two levels of arousal and two levels of the valence, a probabilistic neural network (PNN) with different sigma adjustment parameter was examined. The results showed that the phase space geometry and consequently, the signal dynamics are influenced by the emotional music video. Additionally, distinctive patterns of the phase space behavior were observed under the influence of different lags. For both signals, the most irregularity was observed during the high valence, and the least irregularity was seen during the low valence. Consequently, signals' irregularity is affected by the valence dimension. The results showed that the fusion has more potential for emotion recognition than that of using each signal separately. For sigma = 0.1, the highest recognition rate was 100% in a subject-dependent mode. In a subject-independent mode, the maximum accuracies of 88.57 and 86.8% were obtained for arousal and valence dimensions, respectively.

Schizophrenia diagnosis using innovative EEG feature-level fusion schemes

Electroencephalogram (EEG) has become a practical tool for monitoring and diagnosing pathological/psychological brain states. To date, an increasing number of investigations considered differences between brain dynamic of patients with schizophrenia and healthy controls. However, insufficient studies have been performed to provide an intelligent and accurate system that detects the schizophrenia using EEG signals. This paper concerns this issue by providing new feature-level fusion algorithms. Firstly, we analyze EEG dynamics using three well-known nonlinear measures, including complexity (Cx), Higuchi fractal dimension (HFD), and Lyapunov exponents (Lya). Next, we propose some innovative feature-level fusion strategies to combine the information of these indices. We evaluate the effect of the classifier parameter (σ) adjustment and the cross-validation partitioning criteria on classification accuracy. The performance of EEG classification using combined features was compared with the non-combined attributes. Experimental results showed higher classification accuracy when feature-level features were utilized, compared to when each feature was used individually or all fed to the classifier simultaneously. Using the proposed algorithm, the classification accuracy increased up to 100%. These results establish the suggested framework as a superior scheme compared to the state-of-the-art EEG schizophrenia diagnosis tool.

Comparison of higher order spectra in heart rate signals during two techniques of meditation: Chi and Kundalini meditation

The human heartbeat is one of the important examples of complex physiologic fluctuations. For the first time in this study higher order spectra of heart rate signals during meditation have explored. Specifically, the aim of this study was to analysis and compares the contribution of quadratic phase coupling of human heart rate variability during two forms of meditation: (1) Chinese Chi (or Qigong) meditation and (2) Kundalini Yoga meditation. For this purpose, Bispectrum was estimated by using biased, parametric and the direct (FFT) method. The results show that the mean Bispectrum magnitude of heart rate signals increased during Kundalini Yoga meditation, but it decreased significantly during Chi meditation. However, in both meditation techniques phase-coupled harmonics are shifted to the higher frequencies during meditation. In addition, it has shown that not only there are significant differences between rest and meditation states, but also heart rate patterns appear to be influenced by different types of meditation.

Human identification using a new matching Pursuit-based feature set of ECG

BACKGROUND AND OBJECTIVE

In recent years, many attempts have been made to design reliable systems for identifying individuals using biometrics. Electrocardiogram (ECG) biometric is one of the newest methods that not only offers unique characteristics of individuals for human identification, but also the possibility of counterfeiting it is negligible. In this paper, our objective was to develop an identification system using a non-fiducial one-lead ECG feature set based on a sparse algorithm.

METHODS

The ECG signals of 90 participants were decomposed using a matching pursuit (MP) and several statistical and nonlinear measures were extracted from the MP coefficients. Then, the performance of ECG characteristics delivered by MP analysis in human identification was evaluated by the probabilistic neural network (PNN) and k-nearest neighbor (kNN) with one vs. all strategy. The role of the feature set in classification rates was also tested in different modes, including linear attributes, nonlinear indices, all features, features selected by principal component analysis (PCA), and features selected by linear discriminant analysis (LDA).

RESULTS

Experimental results showed that (1) the highest recognition rate was 99.68%; (2) the performance of the PNN was superior to the kNN; and (3) selecting features with LDA resulted in higher identification rates.

CONCLUSIONS

The results are prominent from the performance perspective because it gives higher recognition rates over the group of 90 participants. The great performance of the proposed identification system advocates that it can be employed confidently in different smart systems.

A NOVEL SIGNAL-BASED FUSION APPROACH FOR ACCURATE MUSIC EMOTION RECOGNITION

The objective of this study is to propose an accurate emotion recognition methodology. To this end, a novel fusion framework based on wavelet transform (WT), and matching pursuit (MP) algorithm was offered. Electrocardiogram (ECG) and galvanic skin response (GSR) of 11 healthy students were collected while subjects listened to emotional music clips. In both fusion techniques, Coiflet wavelet (Coif5 at level 14) was chosen as a wavelet family and MP dictionary, respectively. After employing the proposed fusion framework, some statistical measures were extracted. To describe emotions, three schemes were adopted: two-dimensional model (five classes), valence-(three classes), and arousal-(three classes) based emotion categories. Subsequently, the probabilistic neural network (PNN) was applied to classify affective states. The experiments indicate that the MP-based fusion approach outperform the wavelet-based fusion technique or methods using only ECG or GSR indices. Considering the proposed fusion techniques, the maximum classification rate of 99.64% and 92.31% was reached for the fusion methodology based on the MP algorithm (five classes of emotion) and wavelet-based fusion technique (three classes of valence), respectively.

DYNAMICAL ANALYSIS OF EMOTIONAL STATES FROM ELECTROENCEPHALOGRAM SIGNALS

The current study evaluates the dynamics of the electroencephalogram signals during specific emotional states in order to obtain a detailed understanding of the affective EEG patterns. Employing recurrence analysis, the dynamical states of the emotional brain during visual stimuli is evaluated. Three channels of electroencephalogram time series (Fz, Cz, and Pz) available in eNTERFACE06_EMOBRAIN database are used in this study. Electroencephalogram signals are recorded from 5 subjects in three emotional categories: exciting negative (disgust), neutral and exciting positive (happy). Recurrence quantification analysis (RQA) was applied to study electroencephalogram morphological changes in different emotional states (happy, disgust, neutral). The ANOVA and [Formula: see text]-test are done to detect significant differences in RQA measures of the EEGs. It has shown that the phase space trajectory becomes more periodic during exciting negative. In addition, the results reveal that in comparison with negative emotion and neutral, the behavior of EEGs in positive emotion is highly chaotic. Performing statistical analysis, significant differences were observed in recurrence rate, determinism, average diagonal, line length, Shannon entropy, laminarity, trapping time among three emotional evoked groups. Although these changes occurred in all EEG channels, a better distinction between each emotional state can be observed in Pz location. It seems that recurrence analysis is a promising non-linear approach for detecting instantaneous changes in the emotional EEG induced by visual stimuli. RQA has the potential to discover differences of signal features in response to an emotional stimulus.

GENDER DIFFERENCES IN RESPONSE TO AFFECTIVE AUDIO AND VISUAL INDUCTIONS: EXAMINATION OF NONLINEAR DYNAMICS OF AUTONOMIC SIGNALS

Physiological reflection of emotions can be tracked by autonomic signals. Several studies have been conducted using autonomic signal processing to examine men and women differences during the exposure of affective stimuli. Emotional pictures and music are two commonly used methods to induce affects in an experimental setup. The biological changes have been commonly monitored during a certain emotional inducement protocol, solely. This study was aimed to examine two induction paradigms involved auditory and visual cues using nonlinear dynamical approaches. To this end, various nonlinear parameters of galvanic skin response (GSR) and pulse signals of men and women were examined. The nonlinear analysis was performed using lagged Poincare parameters, detrended fluctuation indices (DFAs), Lyapunov exponents (LEs), some entropy measures, and recurrence quantification analysis (RQA). The Wilcoxon rank sum test was used to show significant differences between the groups. The results indicate that besides the type of affect induction, physiological differences of men and women are notable in negative emotions (sadness and fear). Regardless to the inducements, lagged Poincare parameters of the pulse signals and DFA indices of the GSR have shown significant differences in gender affective responses. However, applying pictorial stimuli, LEs are appropriate indicators for gender discrimination. It is also concluded that GSR dynamics are intensely affected by the kind of stimuli; while this is not validated for the pulse. These findings suggest that different emotional inductions evoked different autonomic responses in men and women, which can be appropriately monitored using nonlinear signal processing approaches.

EVALUATION OF SIGNAL PROCESSING TECHNIQUES IN DISCRIMINATING ECG SIGNALS OF MEN AND WOMEN DURING REST CONDITION AND EMOTIONAL STATES

Great range of electrocardiogram (ECG) signal processing methods can be found in the literature. In addition, the importance of gender differences in physiological activities was also identified in various conditions. This article aims to provide a comprehensive evaluation of linear and nonlinear ECG parameters to indicate suitable signal processing approaches which can show significant differences between men and women. These differences were investigated in two conditions: (i) during rest condition, and (ii) during the affective image inducements. A wide range of parameters from time-, frequency-, wavelet-, and nonlinear-techniques were examined. Applying the Wilcoxon rank sum test, significant differences between two genders were inspected. The analysis was performed on 47 college students at rest condition and while subjects watching four types of affective pictures, including sadness, happiness, fear, and peacefulness. The impact of these emotions on the results was also investigated. The results indicated that 72.95% and 72.61% of all features were significantly different between male and female in rest condition and affective inducements, respectively. In addition, the highest percentage of the significant difference between ECG parameters of men and women was achieved using nonlinear characteristics. Considering all features together, the highest significant difference between two genders was achieved for negative emotions, including sadness and fear. In conclusion, the results of this study emphasized the importance of gender role in cardiac responses during rest condition and different emotional states. Since these gender differences are well manifested by nonlinear signal processing techniques, dynamical gender-specific ECG system may improve the automatic emotion recognition accuracies.

Emotion Recognition Using a Novel Granger Causality Quantifier and Combined Electrodes of EEG

Electroencephalogram (EEG) connectivity patterns can reflect neural correlates of emotion. However, the necessity of evaluating bulky data for multi-channel measurements increases the computational cost of the EEG network. To date, several approaches have been presented to pick the optimal cerebral channels, mainly depending on available data. Consequently, the risk of low data stability and reliability has increased by reducing the number of channels. Alternatively, this study suggests an electrode combination approach in which the brain is divided into six areas. After extracting EEG frequency bands, an innovative Granger causality-based measure was introduced to quantify brain connectivity patterns. The feature was subsequently subjected to a classification module to recognize valence-arousal dimensional emotions. A Database for Emotion Analysis Using Physiological Signals (DEAP) was used as a benchmark database to evaluate the scheme. The experimental results revealed a maximum accuracy of 89.55%. Additionally, EEG-based connectivity in the beta-frequency band was able to effectively classify dimensional emotions. In sum, combined EEG electrodes can efficiently replicate 32-channel EEG information.

Verhulst map measures: new biomarkers for heart rate classification

Recording, monitoring, and analyzing biological signals has received significant attention in medicine. A fundamental phase for understanding a bio-system under various conditions is to process the corresponding bio-signal appropriately. To this effect, different conventional and nonlinear approaches have been proposed. However, since the non-stationary properties of the bio-signals are not revealed by traditional linear methods, nonlinear dynamical techniques play a crucial role in examining the behavior of a bio-system. This work proposes new bio-markers based on the chaotic nature of the biomedical signals. These measures were introduced using the Verhulst map, a simple tool for characterizing the morphology of the reconstructed phase space. For this purpose, we extracted the features from the heart rate (HR) signals of six groups of meditators and non-meditators. For a typical classification problem, the performance of some conventional classifiers, including the k-nearest neighbor, support vector machine, and Naïve Bayes, was appraised separately. In addition, the competence of a hybrid classification strategy was inspected using majority voting. The results indicated a maximum accuracy, F1-score, and sensitivity of 100%. These findings reveal that the proposed framework is eminently capable of analyzing and classifying the HR signals of the groups. In conclusion, the Verhulst diagram-based measures are simple and based on the dynamics of the bio-signals, which can be served for quantifying different signals in medical systems.

Lemniscate of Bernoulli's map quantifiers: innovative measures for EEG emotion recognition

Thanks to the advent of affective computing, designing an automatic human emotion recognition system for clinical and non-clinical applications has attracted the attention of many researchers. Currently, multi-channel electroencephalogram (EEG)-based emotion recognition is a fundamental but challenging issue. This experiment envisioned developing a new scheme for automated EEG affect recognition. An innovative nonlinear feature engineering approach was presented based on Lemniscate of Bernoulli's Map (LBM), which belongs to the family of chaotic maps, in line with the EEG's nonlinear nature. As far as the authors know, LBM has not been utilized for biological signal analysis. Next, the map was characterized using several graphical indices. The feature vector was imposed on the feature selection algorithm while evaluating the role of the feature vector dimension on emotion recognition rates. Finally, the efficiency of the features on emotion recognition was appraised using two conventional classifiers and validated using the Database for Emotion Analysis using Physiological signals (DEAP) and SJTU Emotion EEG Dataset-IV (SEED-IV) benchmark databases. The experimental results showed a maximum accuracy of 92.16% for DEAP and 90.7% for SEED-IV. Achieving higher recognition rates compared to the state-of-art EEG emotion recognition systems suggest the proposed method based on LBM could have potential both in characterizing bio-signal dynamics and detecting affect-deficit disorders.

Combination of sLORETA and Nonlinear Coupling for Emotional EEG Source Localization

The objective of the present study is to investigate the anatomical distribution of the cortical sources of emotional response to music videos by means of electroencephalogram (EEG) analysis. A novel methodology is introduced to determine the nonlinear couplings between different brain regions based on the coherence analysis, nonlinear features of EEG recordings and a source localization method, standard low resolution electromagnetic tomography (sLORETA). 32 channels of EEG time series of 32 subjects available in DEAP database were studied. The Lyapunov exponents and approximate entropy were applied to the EEG. The coherence for Lyapunov exponents and approximate entropy were calculated between each electrode paired to all other electrodes. Considering valence and arousal related effects, the sLORETA was applied to each above mentioned feature to determine emotional processing cortices. Using the proposed methodology, significant differences in sLORETA activity are observed between different emotional states. These changes were dominantly localized in the Brodmann 11 area (frontal lobe). In addition, some evidences provided that the left hemisphere is more activated to valence and arousal-related effects. Results suggest that considering two dimensions of emotions concurrently, a wider brain region was dominated in synchronization: superior frontal gyrus, middle frontal gyrus, and superior parietal lobule. Cooperating nonlinear coupling along with EEG source localization methods could provide an interesting tool for understanding the cortical specialization in emotional processes.