Time-Frequency Representation and Convolutional Neural Network-Based Emotion Recognition

Brain Emotion Perception Inspired EEG Emotion Recognition With Deep Reinforcement Learning

Inspired by the well-known Papez circuit theory and neuroscience knowledge of reinforcement learning, a double dueling deep Q network (DQN) is built incorporating the electroencephalogram (EEG) signals of the frontal lobe as prior information, which is named frontal lobe double dueling DQN (FLD3QN). The framework of FLD3QN is constructed in accord with the brain emotion mechanism which takes the frontal lobe and the thalamus as the core, in which the part of the Papez circuit is simulated by the bifrontal lobe residual convolution neural network (BiFRCNN). Moreover, a step penalty factor is designed to constrain the number of mistakes of the agent. The ablation studies results on the public EEG emotion dataset DEAP verified the important roles of the frontal lobe and the Papez circuit in modeling the procedure of learning rewards during the perception of emotions, with a great increase in the average accuracies by 25.24% and 23.31% in valence and arousal dimensions.

Multimodal insights into granger causality connectivity: Integrating physiological signals and gated eye-tracking data for emotion recognition using convolutional neural network

This study introduces a groundbreaking method to enhance the accuracy and reliability of emotion recognition systems by combining electrocardiogram (ECG) with electroencephalogram (EEG) data, using an eye-tracking gated strategy. Initially, we propose a technique to filter out irrelevant portions of emotional data by employing pupil diameter metrics from eye-tracking data. Subsequently, we introduce an innovative approach for estimating effective connectivity to capture the dynamic interaction between the brain and the heart during emotional states of happiness and sadness. Granger causality (GC) is estimated and utilized to optimize input for a highly effective pre-trained convolutional neural network (CNN), specifically ResNet-18. To assess this methodology, we employed EEG and ECG data from the publicly available MAHNOB-HCI database, using a 5-fold cross-validation approach. Our method achieved an impressive average accuracy and area under the curve (AUC) of 91.00 % and 0.97, respectively, for GC-EEG-ECG images processed with ResNet-18. Comparative analysis with state-of-the-art studies clearly shows that augmenting ECG with EEG and refining data with an eye-tracking strategy significantly enhances emotion recognition performance across various emotions.

Judgment Prediction Based on Tensor Decomposition With Optimized Neural Networks

In the field of smart justice, handling legal cases through artificial intelligence technology is a research hotspot. Traditional judgment prediction methods are mainly based on feature models and classification algorithms. The former is difficult to describe cases from multiple angles and capture the correlation information between different case modules, while requires a wealth of legal expertise and manual labeling. The latter is unable to accurately extract the most useful information from case documents and produce fine-grained predictions. This article proposes a judgment prediction method based on tensor decomposition with optimized neural networks, which consists of OTenr, GTend, and RnEla. OTenr represents cases as normalized tensors. GTend decomposes normalized tensors into core tensors using the guidance tensor. RnEla intervenes in a case modeling process in GTend by optimizing the guidance tensor, so that core tensors represent tensor structural and elemental information, which is most conducive to improving the accuracy of judgment prediction. RnEla consists of the similarity correlation Bi-LSTM and optimized Elastic-Net regression. RnEla takes the similarity between cases as an important factor for judgment prediction. Experimental results on real legal case dataset show that the accuracy of our method is higher than that of the previous judgment prediction methods.

Effective Emotion Recognition by Learning Discriminative Graph Topologies in EEG Brain Networks

Multichannel electroencephalogram (EEG) is an array signal that represents brain neural networks and can be applied to characterize information propagation patterns for different emotional states. To reveal these inherent spatial graph features and increase the stability of emotion recognition, we propose an effective emotion recognition model that performs multicategory emotion recognition with multiple emotion-related spatial network topology patterns (MESNPs) by learning discriminative graph topologies in EEG brain networks. To evaluate the performance of our proposed MESNP model, we conducted single-subject and multisubject four-class classification experiments on two public datasets, MAHNOB-HCI and DEAP. Compared with existing feature extraction methods, the MESNP model significantly enhances the multiclass emotional classification performance in the single-subject and multisubject conditions. To evaluate the online version of the proposed MESNP model, we designed an online emotion monitoring system. We recruited 14 participants to conduct the online emotion decoding experiments. The average online experimental accuracy of the 14 participants was 84.56%, indicating that our model can be applied in affective brain-computer interface (aBCI) systems. The offline and online experimental results demonstrate that the proposed MESNP model effectively captures discriminative graph topology patterns and significantly improves emotion classification performance. Moreover, the proposed MESNP model provides a new scheme for extracting features from strongly coupled array signals.

Lemniscate of Bernoulli's map quantifiers: innovative measures for EEG emotion recognition

Thanks to the advent of affective computing, designing an automatic human emotion recognition system for clinical and non-clinical applications has attracted the attention of many researchers. Currently, multi-channel electroencephalogram (EEG)-based emotion recognition is a fundamental but challenging issue. This experiment envisioned developing a new scheme for automated EEG affect recognition. An innovative nonlinear feature engineering approach was presented based on Lemniscate of Bernoulli's Map (LBM), which belongs to the family of chaotic maps, in line with the EEG's nonlinear nature. As far as the authors know, LBM has not been utilized for biological signal analysis. Next, the map was characterized using several graphical indices. The feature vector was imposed on the feature selection algorithm while evaluating the role of the feature vector dimension on emotion recognition rates. Finally, the efficiency of the features on emotion recognition was appraised using two conventional classifiers and validated using the Database for Emotion Analysis using Physiological signals (DEAP) and SJTU Emotion EEG Dataset-IV (SEED-IV) benchmark databases. The experimental results showed a maximum accuracy of 92.16% for DEAP and 90.7% for SEED-IV. Achieving higher recognition rates compared to the state-of-art EEG emotion recognition systems suggest the proposed method based on LBM could have potential both in characterizing bio-signal dynamics and detecting affect-deficit disorders.

Assessment of impaired consciousness using EEG-based connectivity features and convolutional neural networks

Growing electroencephalogram (EEG) studies have linked the abnormities of functional brain networks with disorders of consciousness (DOC). However, due to network data's high-dimensional and non-Euclidean properties, it is difficult to exploit the brain connectivity information that can effectively detect the consciousness levels of DOC patients via deep learning. To take maximum advantage of network information in assessing impaired consciousness, we utilized the functional connectivity with convolutional neural network (CNN) and employed three rearrangement schemes to improve the evaluation performance of brain networks. In addition, the gradient-weighted class activation mapping (Grad-CAM) was adopted to visualize the classification contributions of connections among different areas. We demonstrated that the classification performance was significantly enhanced by applying network rearrangement techniques compared to those obtained by the original connectivity matrix (with an accuracy of 75.0%). The highest classification accuracy (87.2%) was achieved by rearranging the alpha network based on the anatomical regions. The inter-region connections (i.e., frontal-parietal and frontal-occipital connectivity) played dominant roles in the classification of patients with different consciousness states. The effectiveness of functional connectivity in revealing individual differences in brain activity was further validated by the correlation between behavioral performance and connections among specific regions. These findings suggest that our proposed assessment model could detect the residual consciousness of patients.

Synthesizing affective neurophysiological signals using generative models: A review paper

The integration of emotional intelligence in machines is an important step in advancing human-computer interaction. This demands the development of reliable end-to-end emotion recognition systems. However, the scarcity of public affective datasets presents a challenge. In this literature review, we emphasize the use of generative models to address this issue in neurophysiological signals, particularly Electroencephalogram (EEG) and Functional Near-Infrared Spectroscopy (fNIRS). We provide a comprehensive analysis of different generative models used in the field, examining their input formulation, deployment strategies, and methodologies for evaluating the quality of synthesized data. This review serves as a comprehensive overview, offering insights into the advantages, challenges, and promising future directions in the application of generative models in emotion recognition systems. Through this review, we aim to facilitate the progression of neurophysiological data augmentation, thereby supporting the development of more efficient and reliable emotion recognition systems.

EEG-based emotion recognition systems; comprehensive study

Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.

An Integrated Multitasking Intelligent Bearing Fault Diagnosis Scheme Based on Representation Learning Under Imbalanced Sample Condition

Accurate bearing fault diagnosis is of great significance of the safety and reliability of rotary mechanical system. In practice, the sample proportion between faulty data and healthy data in rotating mechanical system is imbalanced. Furthermore, there are commonalities between the bearing fault detection, classification, and identification tasks. Based on these observations, this article proposes a novel integrated multitasking intelligent bearing fault diagnosis scheme with the aid of representation learning under imbalanced sample condition, which realizes bearing fault detection, classification, and unknown fault identification. Specifically, in the unsupervised condition, a bearing fault detection approach based on modified denoising autoencoder (DAE) with self-attention mechanism for bottleneck layer (MDAE-SAMB) is proposed in the integrated scheme, which only uses the healthy data for training. The self-attention mechanism is introduced into the neurons in the bottleneck layer, which can assign different weights to the neurons in the bottleneck layer. Moreover, the transfer learning based on representation learning is proposed for few-shot fault classification. Only a few fault samples are used for offline training, and high-accuracy online bearing fault classification is achieved. Finally, according to the known fault data, the unknown bearing faults can be effectively identified. A bearing dataset generated by rotor dynamics experiment rig (RDER) and a public bearing dataset demonstrates the applicability of the proposed integrated fault diagnosis scheme.

Amplitude-Time Dual-View Fused EEG Temporal Feature Learning for Automatic Sleep Staging

Electroencephalogram (EEG) plays an important role in studying brain function and human cognitive performance, and the recognition of EEG signals is vital to develop an automatic sleep staging system. However, due to the complex nonstationary characteristics and the individual difference between subjects, how to obtain the effective signal features of the EEG for practical application is still a challenging task. In this article, we investigate the EEG feature learning problem and propose a novel temporal feature learning method based on amplitude-time dual-view fusion for automatic sleep staging. First, we explore the feature extraction ability of convolutional neural networks for the EEG signal from the perspective of interpretability and construct two new representation signals for the raw EEG from the views of amplitude and time. Then, we extract the amplitude-time signal features that reflect the transformation between different sleep stages from the obtained representation signals by using conventional 1-D CNNs. Furthermore, a hybrid dilation convolution module is used to learn the long-term temporal dependency features of EEG signals, which can overcome the shortcoming that the small-scale convolution kernel can only learn the local signal variation information. Finally, we conduct attention-based feature fusion for the learned dual-view signal features to further improve sleep staging performance. To evaluate the performance of the proposed method, we test 30-s-epoch EEG signal samples for healthy subjects and subjects with mild sleep disorders. The experimental results from the most commonly used datasets show that the proposed method has better sleep staging performance and has the potential for the development and application of an EEG-based automatic sleep staging system.

Electroencephalographic Signal Data Augmentation Based on Improved Generative Adversarial Network

EEG signals combined with deep learning play an important role in the study of human-computer interaction. However, the limited dataset makes it challenging to study EEG signals using deep learning methods. Inspired by the GAN network in image generation, this paper presents an improved generative adversarial network model L-C-WGAN-GP to generate artificial EEG data to augment training sets and improve the application of BCI in various fields. The generator consists of a long short-term memory (LSTM) network and the discriminator consists of a convolutional neural network (CNN) which uses the gradient penalty-based Wasserstein distance as the loss function in model training. The model can learn the statistical features of EEG signals and generate EEG data that approximate real samples. In addition, the performance of the compressed sensing reconstruction model can be improved by using augmented datasets. Experiments show that, compared with the existing advanced data amplification techniques, the proposed model produces EEG signals closer to the real EEG signals as measured by RMSE, FD and WTD indicators. In addition, in the compressed reconstruction of EEG signals, adding the new data reduces the loss by about 15% compared with the original data, which greatly improves the reconstruction accuracy of the EEG signals' compressed sensing.

EEGProgress: A fast and lightweight progressive convolution architecture for EEG classification

Because of the intricate topological structure and connection of the human brain, extracting deep spatial features from electroencephalograph (EEG) signals is a challenging and time-consuming task. The extraction of topological spatial information plays a crucial role in EEG classification, and the architecture of the spatial convolution greatly affects the performance and complexity of convolutional neural network (CNN) based EEG classification models. In this study, a progressive convolution CNN architecture named EEGProgress is proposed, aiming to efficiently extract the topological spatial information of EEG signals from multi-scale levels (electrode, brain region, hemisphere, global) with superior speed. To achieve this, the raw EEG data is permuted using the empirical topological permutation rule, integrating the EEG data with numerous topological properties. Subsequently, the spatial features are extracted by a progressive feature extractor including prior, electrode, region, and hemisphere convolution blocks, progressively extracting the deep spatial features with reduced parameters and speed. Finally, the comparison and ablation experiments under both cross-subject and within-subject scenarios are conducted on a public dataset to verify the performance of the proposed EEGProgress and the effectiveness of the topological permutation. The results demonstrate the superior feature extraction ability of the proposed EEGProgress, with an average increase of 4.02% compared to other CNN-based EEG classification models under both cross-subject and within-subject scenarios. Furthermore, with the obtained average testing time, FLOPs, and parameters, the proposed EEGProgress outperforms other comparison models in terms of model complexity.

Learning Skill Characteristics From Manipulations

Percutaneous coronary intervention (PCI) has increasingly become the main treatment for coronary artery disease. The procedure requires high experienced skills and dexterous manipulations. However, there are few techniques to model PCI skill so far. In this study, a learning framework with local and ensemble learning is proposed to learn skill characteristics of different skill-level subjects from their PCI manipulations. Ten interventional cardiologists (four experts and six novices) were recruited to deliver a medical guidewire to two target arteries on a porcine model for in vivo studies. Simultaneously, translation and twist manipulations of thumb, forefinger, and wrist are acquired with electromagnetic (EM) and fiber-optic bend (FOB) sensors, respectively. These behavior data are then processed with wavelet packet decomposition (WPD) under 1-10 levels for feature extraction. The feature vectors are further fed into three candidate individual classifiers in the local learning layer. Furthermore, the local learning results from different manipulation behaviors are fused in the ensemble learning layer with three rule-based ensemble learning algorithms. In subject-dependent skill characteristics learning, the ensemble learning can achieve 100% accuracy, significantly outperforming the best local result (90%). Furthermore, ensemble learning can also maintain 73% accuracy in subject-independent schemes. These promising results demonstrate the great potential of the proposed method to facilitate skill learning in surgical robotics and skill assessment in clinical practice.

Emotion recognition in EEG signals using deep learning methods: A review

Emotions are a critical aspect of daily life and serve a crucial role in human decision-making, planning, reasoning, and other mental states. As a result, they are considered a significant factor in human interactions. Human emotions can be identified through various sources, such as facial expressions, speech, behavior (gesture/position), or physiological signals. The use of physiological signals can enhance the objectivity and reliability of emotion detection. Compared with peripheral physiological signals, electroencephalogram (EEG) recordings are directly generated by the central nervous system and are closely related to human emotions. EEG signals have the great spatial resolution that facilitates the evaluation of brain functions, making them a popular modality in emotion recognition studies. Emotion recognition using EEG signals presents several challenges, including signal variability due to electrode positioning, individual differences in signal morphology, and lack of a universal standard for EEG signal processing. Moreover, identifying the appropriate features for emotion recognition from EEG data requires further research. Finally, there is a need to develop more robust artificial intelligence (AI) including conventional machine learning (ML) and deep learning (DL) methods to handle the complex and diverse EEG signals associated with emotional states. This paper examines the application of DL techniques in emotion recognition from EEG signals and provides a detailed discussion of relevant articles. The paper explores the significant challenges in emotion recognition using EEG signals, highlights the potential of DL techniques in addressing these challenges, and suggests the scope for future research in emotion recognition using DL techniques. The paper concludes with a summary of its findings.

Exploring EEG characteristics of multi-level mental stress based on human-machine system

Objective.The understanding of cognitive states is important for the development of human-machine systems (HMSs), and one of the fundamental but challenging issues is the understanding and assessment of the operator's mental stress state in real task scenarios.Approach.In this paper, a virtual unmanned vehicle (UAV) driving task with multi-challenge-level was created to explore the operator's mental stress, and the human brain activity during the task was tracked in real time via electroencephalography (EEG). A mental stress analysis dataset for the virtual UAV task was then developed and used to explore the neural activation patterns associated with mental stress activity. Finally, a multiple attention-based convolutional neural network (MACN) was constructed for automatic stress assessment using the extracted stress-sensitive neural activation features.Main Results.The statistical results of EEG power spectral density (PSD) showed that frontal theta-PSD decreased with increasing task difficulty, and central beta-PSD increased with increasing task difficulty, indicating that neural patterns showed different trends under different levels of mental stress. The performance of the proposed MACN was evaluated based on the dimensional model, and results showed that average three-class classification accuracies of 89.49%/89.88% were respectively achieved for arousal/valence.Significance.The results of this paper suggest that objective assessment of mental stress in a HMS based on a virtual UAV scenario is feasible, and the proposed method provides a promising solution for cognitive computing and applications in human-machine tasks.

Classification of User Emotional Experiences on B2C Websites Utilizing Infrared Thermal Imaging

The acquisition of physiological signals for analyzing emotional experiences has been intrusive, and potentially yields inaccurate results. This study employed infrared thermal images (IRTIs), a noninvasive technique, to classify user emotional experiences while interacting with business-to-consumer (B2C) websites. By manipulating the usability and aesthetics of B2C websites, the facial thermal images of 24 participants were captured as they engaged with the different websites. Machine learning techniques were leveraged to classify their emotional experiences, with participants' self-assessments serving as the ground truth. The findings revealed significant fluctuations in emotional valence, while the participants' arousal levels remained consistent, enabling the categorization of emotional experiences into positive and negative states. The support vector machine (SVM) model performed well in distinguishing between baseline and emotional experiences. Furthermore, this study identified key regions of interest (ROIs) and effective classification features in machine learning. These findings not only established a significant connection between user emotional experiences and IRTIs but also broadened the research perspective on the utility of IRTIs in the field of emotion analysis.

A novel feature fusion network for multimodal emotion recognition from EEG and eye movement signals

Emotion recognition is a challenging task, and the use of multimodal fusion methods for emotion recognition has become a trend. Fusion vectors can provide a more comprehensive representation of changes in the subject's emotional state, leading to more accurate emotion recognition results. Different fusion inputs or feature fusion methods have varying effects on the final fusion outcome. In this paper, we propose a novel Multimodal Feature Fusion Neural Network model (MFFNN) that effectively extracts complementary information from eye movement signals and performs feature fusion with EEG signals. We construct a dual-branch feature extraction module to extract features from both modalities while ensuring temporal alignment. A multi-scale feature fusion module is introduced, which utilizes cross-channel soft attention to adaptively select information from different spatial scales, enabling the acquisition of features at different spatial scales for effective fusion. We conduct experiments on the publicly available SEED-IV dataset, and our model achieves an accuracy of 87.32% in recognizing four emotions (happiness, sadness, fear, and neutrality). The results demonstrate that the proposed model can better explore complementary information from EEG and eye movement signals, thereby improving accuracy, and stability in emotion recognition.

High accuracy distinction of shockable and non-shockable arrhythmias in abnormal classes through wavelet transform with pseudo differential like operators

Arrhythmia is an abnormal rhythm of the heart which leads to sudden death. Among these arrhythmias, some are shockable, and some are non-shockable arrhythmias with external defibrillation. The automated external defibrillator (AED) is used as the automated arrhythmia diagnosis system and requires an accurate and rapid decision to increase the survival rate. Therefore, a precise and quick decision by the AED has become essential in improving the survival rate. This paper presents an arrhythmia diagnosis system for the AED by engineering methods and generalized function theories. In the arrhythmia diagnosis system, the proposed wavelet transform with pseudo-differential like operators-based method effectively generates a distinguishable scalogram for the shockable and non-shockable arrhythmia in the abnormal class signals, which leads to the decision algorithm getting the best distinction. Then, a new quality parameter is introduced to get more details by quantizing the statistical features on the scalogram. Finally, design a simple AED shock and non-shock advice method by following this information to improve the precision and rapid decision. Here, an adequate topology (metric function) is adopted to the space of the scatter plot, where we can give different scales to select the best area of the scatter plot for the test sample. As a consequence, the proposed decision method gives the highest accuracy and rapid decision between shockable and non-shockable arrhythmias. The proposed arrhythmia diagnosis system increases the accuracy to 97.98%, with a gain of 11.75% compared to the conventional approach in the abnormal class signals. Therefore, the proposed method contributes an additional 11.75% possibility for increasing the survival rate. The proposed arrhythmia diagnosis system is general and could be applied to distinguish different arrhythmia-based applications. Also, each contribution could be used independently in various applications.

DeePay: deep learning decodes EEG to predict consumer's willingness to pay for neuromarketing

There is an increasing demand within consumer-neuroscience (or neuromarketing) for objective neural measures to quantify consumers' subjective valuations and predict responses to marketing campaigns. However, the properties of EEG raise difficulties for these aims: small datasets, high dimensionality, elaborate manual feature extraction, intrinsic noise, and between-subject variations. We aimed to overcome these limitations by combining unique techniques of Deep Learning Networks (DLNs), while providing interpretable results for neuroscientific and decision-making insight. In this study, we developed a DLN to predict subjects' willingness to pay (WTP) based on their EEG data. In each trial, 213 subjects observed a product's image, from 72 possible products, and then reported their WTP for the product. The DLN employed EEG recordings from product observation to predict the corresponding reported WTP values. Our results showed 0.276 test root-mean-square-error and 75.09% test accuracy in predicting high vs. low WTP, surpassing other models and a manual feature extraction approach. Network visualizations provided the predictive frequencies of neural activity, their scalp distributions, and critical timepoints, shedding light on the neural mechanisms involved with evaluation. In conclusion, we show that DLNs may be the superior method to perform EEG-based predictions, to the benefit of decision-making researchers and marketing practitioners alike.

Emotion Recognition Using a Novel Granger Causality Quantifier and Combined Electrodes of EEG

Electroencephalogram (EEG) connectivity patterns can reflect neural correlates of emotion. However, the necessity of evaluating bulky data for multi-channel measurements increases the computational cost of the EEG network. To date, several approaches have been presented to pick the optimal cerebral channels, mainly depending on available data. Consequently, the risk of low data stability and reliability has increased by reducing the number of channels. Alternatively, this study suggests an electrode combination approach in which the brain is divided into six areas. After extracting EEG frequency bands, an innovative Granger causality-based measure was introduced to quantify brain connectivity patterns. The feature was subsequently subjected to a classification module to recognize valence-arousal dimensional emotions. A Database for Emotion Analysis Using Physiological Signals (DEAP) was used as a benchmark database to evaluate the scheme. The experimental results revealed a maximum accuracy of 89.55%. Additionally, EEG-based connectivity in the beta-frequency band was able to effectively classify dimensional emotions. In sum, combined EEG electrodes can efficiently replicate 32-channel EEG information.

Optimized EEG based mood detection with signal processing and deep neural networks for brain-computer interface

Electroencephalogram (EEG) is a very promising and widely implemented procedure to study brain signals and activities by amplifying and measuring the post-synaptical potential arising from electrical impulses produced by neurons and detected by specialized electrodes attached to specific points in the scalp. It can be studied for detecting brain abnormalities, headaches, and other conditions. However, there are limited studies performed to establish a smart decision-making model to identify EEG's relation with the mood of the subject. In this experiment, EEG signals of 28 healthy human subjects have been observed with consent and attempts have been made to study and recognise moods. Savitzky-Golay band-pass filtering and Independent Component Analysis have been used for data filtration.Different neural network algorithms have been implemented to analyze and classify the EEG data based on the mood of the subject. The model is further optimised by the usage of Blackman window-based Fourier Transformation and extracting the most significant frequencies for each electrode. Using these techniques, up to 96.01% detection accuracy has been obtained.

SchizoNET: a robust and accurate Margenau-Hill time-frequency distribution based deep neural network model for schizophrenia detection using EEG signals

Objective.Schizophrenia (SZ) is a severe chronic illness characterized by delusions, cognitive dysfunctions, and hallucinations that impact feelings, behaviour, and thinking. Timely detection and treatment of SZ are necessary to avoid long-term consequences. Electroencephalogram (EEG) signals are one form of a biomarker that can reveal hidden changes in the brain during SZ. However, the EEG signals are non-stationary in nature with low amplitude. Therefore, extracting the hidden information from the EEG signals is challenging.Approach.The time-frequency domain is crucial for the automatic detection of SZ. Therefore, this paper presents the SchizoNET model combining the Margenau-Hill time-frequency distribution (MH-TFD) and convolutional neural network (CNN). The instantaneous information of EEG signals is captured in the time-frequency domain using MH-TFD. The time-frequency amplitude is converted to two-dimensional plots and fed to the developed CNN model.Results.The SchizoNET model is developed using three different validation techniques, including holdout, five-fold cross-validation, and ten-fold cross-validation techniques using three separate public SZ datasets (Dataset 1, 2, and 3). The proposed model achieved an accuracy of 97.4%, 99.74%, and 96.35% on Dataset 1 (adolescents: 45 SZ and 39 HC subjects), Dataset 2 (adults: 14 SZ and 14 HC subjects), and Dataset 3 (adults: 49 SZ and 32 HC subjects), respectively. We have also evaluated six performance parameters and the area under the curve to evaluate the performance of our developed model.Significance.The SchizoNET is robust, effective, and accurate, as it performed better than the state-of-the-art techniques. To the best of our knowledge, this is the first work to explore three publicly available EEG datasets for the automated detection of SZ. Our SchizoNET model can help neurologists detect the SZ in various scenarios.

Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review

In recent years, the rapid development of sensors and information technology has made it possible for machines to recognize and analyze human emotions. Emotion recognition is an important research direction in various fields. Human emotions have many manifestations. Therefore, emotion recognition can be realized by analyzing facial expressions, speech, behavior, or physiological signals. These signals are collected by different sensors. Correct recognition of human emotions can promote the development of affective computing. Most existing emotion recognition surveys only focus on a single sensor. Therefore, it is more important to compare different sensors or unimodality and multimodality. In this survey, we collect and review more than 200 papers on emotion recognition by literature research methods. We categorize these papers according to different innovations. These articles mainly focus on the methods and datasets used for emotion recognition with different sensors. This survey also provides application examples and developments in emotion recognition. Furthermore, this survey compares the advantages and disadvantages of different sensors for emotion recognition. The proposed survey can help researchers gain a better understanding of existing emotion recognition systems, thus facilitating the selection of suitable sensors, algorithms, and datasets.

Coarse-to-Fine: Progressive Knowledge Transfer-Based Multitask Convolutional Neural Network for Intelligent Large-Scale Fault Diagnosis

In modern industry, large-scale fault diagnosis of complex systems is emerging and becoming increasingly important. Most deep learning-based methods perform well on small number of fault diagnosis, but cannot converge to satisfactory results when handling large-scale fault diagnosis because the huge number of fault types will lead to the problems of intra/inter-class distance unbalance and poor local minima in neural networks. To address the above problems, a progressive knowledge transfer-based multitask convolutional neural network (PKT-MCNN) is proposed. First, to construct the coarse-to-fine knowledge structure intelligently, a structure learning algorithm is proposed via clustering fault types in different coarse-grained nodes. Thus, the intra/inter-class distance unbalance problem can be mitigated by spreading similar tasks into different nodes. Then, an MCNN architecture is designed to learn the coarse and fine-grained task simultaneously and extract more general fault information, thereby pushing the algorithm away from poor local minima. Last but not least, a PKT algorithm is proposed, which can not only transfer the coarse-grained knowledge to the fine-grained task and further alleviate the intra/inter-class distance unbalance in feature space, but also regulate different learning stages by adjusting the attention weight to each task progressively. To verify the effectiveness of the proposed method, a dataset of a nuclear power system with 66 fault types was collected and analyzed. The results demonstrate that the proposed method can be a promising tool for large-scale fault diagnosis.

Exploration of Effective Time-Velocity Distribution for Doppler-Radar-Based Personal Gait Identification Using Deep Learning

Personal identification based on radar gait measurement is an important application of biometric technology because it enables remote and continuous identification of people, irrespective of the lighting conditions and subjects' outfits. This study explores an effective time-velocity distribution and its relevant parameters for Doppler-radar-based personal gait identification using deep learning. Most conventional studies on radar-based gait identification used a short-time Fourier transform (STFT), which is a general method to obtain time-velocity distribution for motion recognition using Doppler radar. However, the length of the window function that controls the time and velocity resolutions of the time-velocity image was empirically selected, and several other methods for calculating high-resolution time-velocity distributions were not considered. In this study, we compared four types of representative time-velocity distributions calculated from the Doppler-radar-received signals: STFT, wavelet transform, Wigner-Ville distribution, and smoothed pseudo-Wigner-Ville distribution. In addition, the identification accuracies of various parameter settings were also investigated. We observed that the optimally tuned STFT outperformed other high-resolution distributions, and a short length of the window function in the STFT process led to a reasonable accuracy; the best identification accuracy was 99% for the identification of twenty-five test subjects. These results indicate that STFT is the optimal time-velocity distribution for gait-based personal identification using the Doppler radar, although the time and velocity resolutions of the other methods were better than those of the STFT.

Classification of Heart Sounds Using Chaogram Transform and Deep Convolutional Neural Network Transfer Learning

Heart sounds convey important information regarding potential heart diseases. Currently, heart sound classification attracts many researchers from the fields of telemedicine, digital signal processing, and machine learning-among others-mainly to identify cardiac pathology as quickly as possible. This article proposes chaogram as a new transform to convert heart sound signals to colour images. In the proposed approach, the output image is, therefore, the projection of the reconstructed phase space representation of the phonocardiogram (PCG) signal on three coordinate planes. This has two major benefits: (1) it makes possible to apply deep convolutional neural networks to heart sounds and (2) it is also possible to employ a transfer learning scheme by converting a heart sound signal to an image. The performance of the proposed approach was verified on the PhysioNet dataset. Due to the imbalanced data on this dataset, it is common to assess the results quality using the average of sensitivity and specificity, which is known as score, instead of accuracy. In this study, the best results were achieved using the InceptionV3 model, which achieved a score of 88.06%.

An Ensemble Learning Method for Emotion Charting Using Multimodal Physiological Signals

Emotion charting using multimodal signals has gained great demand for stroke-affected patients, for psychiatrists while examining patients, and for neuromarketing applications. Multimodal signals for emotion charting include electrocardiogram (ECG) signals, electroencephalogram (EEG) signals, and galvanic skin response (GSR) signals. EEG, ECG, and GSR are also known as physiological signals, which can be used for identification of human emotions. Due to the unbiased nature of physiological signals, this field has become a great motivation in recent research as physiological signals are generated autonomously from human central nervous system. Researchers have developed multiple methods for the classification of these signals for emotion detection. However, due to the non-linear nature of these signals and the inclusion of noise, while recording, accurate classification of physiological signals is a challenge for emotion charting. Valence and arousal are two important states for emotion detection; therefore, this paper presents a novel ensemble learning method based on deep learning for the classification of four different emotional states including high valence and high arousal (HVHA), low valence and low arousal (LVLA), high valence and low arousal (HVLA) and low valence high arousal (LVHA). In the proposed method, multimodal signals (EEG, ECG, and GSR) are preprocessed using bandpass filtering and independent components analysis (ICA) for noise removal in EEG signals followed by discrete wavelet transform for time domain to frequency domain conversion. Discrete wavelet transform results in spectrograms of the physiological signal and then features are extracted using stacked autoencoders from those spectrograms. A feature vector is obtained from the bottleneck layer of the autoencoder and is fed to three classifiers SVM (support vector machine), RF (random forest), and LSTM (long short-term memory) followed by majority voting as ensemble classification. The proposed system is trained and tested on the AMIGOS dataset with k-fold cross-validation. The proposed system obtained the highest accuracy of 94.5% and shows improved results of the proposed method compared with other state-of-the-art methods.

Automated robust human emotion classification system using hybrid EEG features with ICBrainDB dataset

Emotion identification is an essential task for human-computer interaction systems. Electroencephalogram (EEG) signals have been widely used in emotion recognition. So far, there have been several EEG-based emotion recognition datasets that the researchers have used to validate their developed models. Hence, we have used a new ICBrainDB EEG dataset to classify angry, neutral, happy, and sad emotions in this work. Signal processing-based wavelet transform (WT), tunable Q-factor wavelet transform (TQWT), and image processing-based histogram of oriented gradients (HOG), local binary pattern (LBP), and convolutional neural network (CNN) features have been used extracted from the EEG signals. The WT is used to extract the rhythms from each channel of the EEG signal. The instantaneous frequency and spectral entropy are computed from each EEG rhythm signal. The average, and standard deviation of instantaneous frequency, and spectral entropy of each rhythm of the signal are the final feature vectors. The spectral entropy in each channel of the EEG signal after performing the TQWT is used to create the feature vectors in the second signal side method. Each EEG channel is transformed into time-frequency plots using the synchrosqueezed wavelet transform. Then, the feature vectors are constructed individually using windowed HOG and LBP features. Also, each channel of the EEG data is fed to a pretrained CNN to extract the features. In the feature selection process, the ReliefF feature selector is employed. Various feature classification algorithms namely, k-nearest neighbor (KNN), support vector machines, and neural networks are used for the automated classification of angry, neutral, happy, and sad emotions. Our developed model obtained an average accuracy of 90.7% using HOG features and a KNN classifier with a tenfold cross-validation strategy.

Multi-Layer Graph Attention Network for Sleep Stage Classification Based on EEG

Graph neural networks have been successfully applied to sleep stage classification, but there are still challenges: (1) How to effectively utilize epoch information of EEG-adjacent channels owing to their different interaction effects. (2) How to extract the most representative features according to confused transitional information in confused stages. (3) How to improve classification accuracy of sleep stages compared with existing models. To address these shortcomings, we propose a multi-layer graph attention network (MGANet). Node-level attention prompts the graph attention convolution and GRU to focus on and differentiate the interaction between channels in the time-frequency domain and the spatial domain, respectively. The multi-head spatial-temporal mechanism balances the channel weights and dynamically adjusts channel features, and a multi-layer graph attention network accurately expresses the spatial sleep information. Moreover, stage-level attention is applied to easily confused sleep stages, which effectively improves the limitations of a graph convolutional network in large-scale graph sleep stages. The experimental results demonstrated classification accuracy; MF1 and Kappa reached 0.825, 0.814, and 0.775 and 0.873, 0.801, and 0.827 for the ISRUC and SHHS datasets, respectively, which showed that MGANet outperformed the state-of-the-art baselines.

Exploration of effective electroencephalography features for the recognition of different valence emotions

Recent studies have shown that the recognition and monitoring of different valence emotions can effectively avoid the occurrence of human errors due to the decline in cognitive ability. The quality of features directly affects emotion recognition results, so this manuscript explores the effective electroencephalography (EEG) features for the recognition of different valence emotions. First, 110 EEG features were extracted from the time domain, frequency domain, time-frequency domain, spatial domain, and brain network, including all the current mainly used features. Then, the classification performance, computing time, and important electrodes of each feature were systematically compared and analyzed on the self-built dataset involving 40 subjects and the public dataset DEAP. The experimental results show that the first-order difference, second-order difference, high-frequency power, and high-frequency differential entropy features perform better in the recognition of different valence emotions. Also, the time-domain features, especially the first-order difference features and second-order difference features, have less computing time, so they are suitable for real-time emotion recognition applications. Besides, the features extracted from the frontal, temporal, and occipital lobes are more effective than others for the recognition of different valence emotions. Especially, when the number of electrodes is reduced by 3/4, the classification accuracy of using features from 16 electrodes located in these brain regions is 91.8%, which is only about 2% lower than that of using all electrodes. The study results can provide an important reference for feature extraction and selection in emotion recognition based on EEG.

Emotional Cognitive Expression in Lacquer Colors Based on Prior Knowledge

Since lacquer painting first appeared in the world of art, research into it has grown steadily. People have developed a keen interest in modern lacquer painting as a result of the extensive study of lacquer culture in both domestic and international academic circles. Many artists and art enthusiasts have contributed significantly to the study and research of lacquer painting and have made helpful attempts at modern lacquer painting. But it is challenging to describe the emotion that a lacquer painting's color conveys. This paper presents a decision-making framework for emotional cognitive learning based on the theory of emotional cognitive evaluation because there are relatively few researchers who have specifically studied the relationship between the creation of lacquer paintings and emotions and because there are also few research materials and documents for reference. The assessment of an emotional state is the central component of this framework. The observation module in the model framework is used to gather the emotional data that the lacquer painting expresses. The issue of emotional expression in lacquer painting is resolved by the emotional evaluation system, which combines the preprocessed information with prior knowledge to evaluate. The importance of affective cognitive expression in lacquer painting and the necessity of affective computing in the fields of machine learning and decision control is obtained on the basis of discussing the research status and content of affective cognition and affective computing. The efficiency of expression was increased by 1.3 percent as a result.

A Classroom Emotion Recognition Model Based on a Convolutional Neural Network Speech Emotion Algorithm

In this paper, we construct a model of convolutional neural network speech emotion algorithm, analyze the classroom identified by the neural network with a certain degree of confidence together with the school used in the dataset, find the characteristics and rules of teachers' control of classroom emotion nowadays using big data, find the parts of classroom emotion, and design a classroom emotion recognition model based on convolutional neural network speech emotion algorithm according to these characteristics. This paper will investigate the factors and patterns of teachers' emotional control in the classroom. In this paper, the existing neural network is adapted and improved, and some preprocessing is performed on the current dataset to train the network. The network used in this paper is a combination of convolutional neural network (CNN) and recurrent neural network (RNN), which takes advantage of both CNN for feature extraction and RNN for memory capability in the sequence model. This network has a good effect on both object labeling and speech recognition. For the problem of extracting emotion features of whole-sentence speech, we propose an attention mechanism-based emotion recognition algorithm for variable-length speech and design a spatiotemporal attention module for the speech emotion algorithm and a convolutional channel attention module for the CNN network to reduce the contribution of the spatiotemporal data of the speech emotion algorithm and the unimportant parts of the CNN convolutional channel feature data in the subsequent recognition by the attention mechanism. In turn, the weight of core key data and features is increased to improve the model recognition accuracy.

Accurate emotion recognition using Bayesian model based EEG sources as dynamic graph convolutional neural network nodes

Due to the effect of emotions on interactions, interpretations, and decisions, automatic detection and analysis of human emotions based on EEG signals has an important role in the treatment of psychiatric diseases. However, the low spatial resolution of EEG recorders poses a challenge. In order to overcome this problem, in this paper we model each emotion by mapping from scalp sensors to brain sources using Bernoulli–Laplace-based Bayesian model. The standard low-resolution electromagnetic tomography (sLORETA) method is used to initialize the source signals in this algorithm. Finally, a dynamic graph convolutional neural network (DGCNN) is used to classify emotional EEG in which the sources of the proposed localization model are considered as the underlying graph nodes. In the proposed method, the relationships between the EEG source signals are encoded in the DGCNN adjacency matrix. Experiments on our EEG dataset recorded at the Brain-Computer Interface Research Laboratory, University of Tabriz as well as publicly available SEED and DEAP datasets show that brain source modeling by the proposed algorithm significantly improves the accuracy of emotion recognition, such that it achieve a classification accuracy of 99.25% during the classification of the two classes of positive and negative emotions. These results represent an absolute 1–2% improvement in terms of classification accuracy over subject-dependent and subject-independent scenarios over the existing approaches.

Car engine sounds recognition based on deformable feature map residual network

Aiming at the difficulty in extracting the features of time–frequency images for the recognition of car engine sounds, we propose a method to recognize them based on a deformable feature map residual network. A deformable feature map residual block includes offset and convolutional layers. The offset layers shift the pixels of the input feature map. The shifted feature map is superimposed on the feature map extracted by the convolutional layers through shortcut connections to concentrate the network to the sampling in the region of interest, and to transmit the information of the offset feature map to the lower network. Then, a deformable convolution residual network is designed, and the features extracted through this network are fused with the Mel frequency cepstral coefficients of car engine sounds. After recalibration by the squeeze and excitation block, the fused results are fed into the fully connected layer for classification. Experiments on a car engine sound dataset show that the accuracy of the proposed method is 84.28%. Compared with the existing state-of-the-art methods, in terms of the accuracy of recognizing car engine sounds under various operating conditions, the proposed method represents an improvement over the method based on dictionary learning and a convolutional neural network.

Emotional Analysis Model for Social Hot Topics of Professional Migrant Workers

Text makes up a large portion of network data because it is the vehicle for people's direct expression of emotions and opinions. How to analyze and mine these emotional text data has become a hot topic of concern in academia and industry in recent years. The online LDA (Latent Dirichlet Allocation) model is used in this paper to train the social hot topic data of professional migrant workers on the same time slice, and the subtopic evolution and intensity are obtained. The topic development is divided into four categories, and the classification model is created using SVM (Support Vector Machine). Instead of decision makers, a virtual human with sensibility and rationality is built using a hierarchical emotional cognitive model to solve multiobjective optimization problems interactively. It analyzes human body structure and emotional signals, and then combines them with visual and physiological signals to create multimodal emotional data. An example is used to demonstrate the effectiveness of the proposed model.

A Dataset for Emotion Recognition Using Virtual Reality and EEG (DER-VREEG): Emotional State Classification Using Low-Cost Wearable VR-EEG Headsets

Emotions are viewed as an important aspect of human interactions and conversations, and allow effective and logical decision making. Emotion recognition uses low-cost wearable electroencephalography (EEG) headsets to collect brainwave signals and interpret these signals to provide information on the mental state of a person, with the implementation of a virtual reality environment in different applications; the gap between human and computer interaction, as well as the understanding process, would shorten, providing an immediate response to an individual’s mental health. This study aims to use a virtual reality (VR) headset to induce four classes of emotions (happy, scared, calm, and bored), to collect brainwave samples using a low-cost wearable EEG headset, and to run popular classifiers to compare the most feasible ones that can be used for this particular setup. Firstly, we attempt to build an immersive VR database that is accessible to the public and that can potentially assist with emotion recognition studies using virtual reality stimuli. Secondly, we use a low-cost wearable EEG headset that is both compact and small, and can be attached to the scalp without any hindrance, allowing freedom of movement for participants to view their surroundings inside the immersive VR stimulus. Finally, we evaluate the emotion recognition system by using popular machine learning algorithms and compare them for both intra-subject and inter-subject classification. The results obtained here show that the prediction model for the four-class emotion classification performed well, including the more challenging inter-subject classification, with the support vector machine (SVM Class Weight kernel) obtaining 85.01% classification accuracy. This shows that using less electrode channels but with proper parameter tuning and selection features affects the performance of the classifications.

Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation

BACKGROUND

The aim of this paper is to implement a system to facilitate the diagnosis of multiple sclerosis (MS) in its initial stages. It does so using a convolutional neural network (CNN) to classify images captured with swept-source optical coherence tomography (SS-OCT).

METHODS

SS-OCT images from 48 control subjects and 48 recently diagnosed MS patients have been used. These images show the thicknesses (45 × 60 points) of the following structures: complete retina, retinal nerve fiber layer, two ganglion cell layers (GCL+, GCL++) and choroid. The Cohen distance is used to identify the structures and the regions within them with greatest discriminant capacity. The original database of OCT images is augmented by a deep convolutional generative adversarial network to expand the CNN's training set.

RESULTS

The retinal structures with greatest discriminant capacity are the GCL++ (44.99% of image points), complete retina (26.71%) and GCL+ (22.93%). Thresholding these images and using them as inputs to a CNN comprising two convolution modules and one classification module obtains sensitivity = specificity = 1.0.

CONCLUSIONS

Feature pre-selection and the use of a convolutional neural network may be a promising, nonharmful, low-cost, easy-to-perform and effective means of assisting the early diagnosis of MS based on SS-OCT thickness data.

Hardware Acceleration of EEG-Based Emotion Classification Systems: A Comprehensive Survey

Recent years have witnessed a growing interest in EEG-based wearable classifiers of emotions, which could enable the real-time monitoring of patients suffering from neurological disorders such as Amyotrophic Lateral Sclerosis (ALS), Autism Spectrum Disorder (ASD), or Alzheimer's. The hope is that such wearable emotion classifiers would facilitate the patients' social integration and lead to improved healthcare outcomes for them and their loved ones. Yet in spite of their direct relevance to neuro-medicine, the hardware platforms for emotion classification have yet to fill up some important gaps in their various approaches to emotion classification in a healthcare context. In this paper, we present the first hardware-focused critical review of EEG-based wearable classifiers of emotions and survey their implementation perspectives, their algorithmic foundations, and their feature extraction methodologies. We further provide a neuroscience-based analysis of current hardware accelerators of emotion classifiers and use it to map out several research opportunities, including multi-modal hardware platforms, accelerators with tightly-coupled cores operating robustly in the near/supra-threshold region, and pre-processing libraries for universal EEG-based datasets.

Identification of sloshing noises using convolutional neural network

Sloshing in fuel tanks has become a new source of noise in hybrid and high-end vehicles in the wake of reduced noise from major sources like the engine. It occurs due to the interactions of fluid inside the tank under various driving conditions of the vehicle. Interactions of fluid with the tank walls cause hit noise, and the fluid-fluid interactions cause splash noise. As the generation mechanism is different, the hit and splash noises demand different noise controlling strategies. Thus, identifying these noises during the design stage is important for implementing effective solutions in designing a quieter fuel tank. This paper presents a convolutional neural network (CNN) based methodology for the identification of sloshing noises under different conditions of fill level, excitation, baffle configuration, etc. Data for training and testing the network are collected using a reciprocating test setup, which facilitates the generation of hit and splash noises in a rectangular tank. The identification accuracy of the features learned by CNN is compared with the hand-crafted features using support vector machines. The applicability of the proposed CNN model is tested for practical scenarios like vehicle braking, where different types of sloshing noises occur in quick succession.

A Novel Medical Diagnosis model for COVID-19 infection detection based on Deep Features and Bayesian Optimization

A pneumonia of unknown causes, which was detected in Wuhan, China, and spread rapidly throughout the world, was declared as Coronavirus disease 2019 (COVID-19). Thousands of people have lost their lives to this disease. Its negative effects on public health are ongoing. In this study, an intelligence computer-aided model that can automatically detect positive COVID-19 cases is proposed to support daily clinical applications. The proposed model is based on the convolution neural network (CNN) architecture and can automatically reveal discriminative features on chest X-ray images through its convolution with rich filter families, abstraction, and weight-sharing characteristics. Contrary to the generally used transfer learning approach, the proposed deep CNN model was trained from scratch. Instead of the pre-trained CNNs, a novel serial network consisting of five convolution layers was designed. This CNN model was utilized as a deep feature extractor. The extracted deep discriminative features were used to feed the machine learning algorithms, which were k-nearest neighbor, support vector machine (SVM), and decision tree. The hyperparameters of the machine learning models were optimized using the Bayesian optimization algorithm. The experiments were conducted on a public COVID-19 radiology database. The database was divided into two parts as training and test sets with 70% and 30% rates, respectively. As a result, the most efficient results were ensured by the SVM classifier with an accuracy of 98.97%, a sensitivity of 89.39%, a specificity of 99.75%, and an F-score of 96.72%. Consequently, a cheap, fast, and reliable intelligence tool has been provided for COVID-19 infection detection. The developed model can be used to assist field specialists, physicians, and radiologists in the decision-making process. Thanks to the proposed tool, the misdiagnosis rates can be reduced, and the proposed model can be used as a retrospective evaluation tool to validate positive COVID-19 infection cases.

Tunable Q wavelet transform based emotion classification in Parkinson's disease using Electroencephalography

Parkinson's disease (PD) is a severe incurable neurological disorder. It is mostly characterized by non-motor symptoms like fatigue, dementia, anxiety, speech and communication problems, depression, and so on. Electroencephalography (EEG) play a key role in the detection of the true emotional state of a person. Various studies have been proposed for the detection of emotional impairment in PD using filtering, Fourier transforms, wavelet transforms, and non-linear methods. However, these methods require a selection of basis and are confined in terms of accuracy. In this paper, tunable Q wavelet transform (TQWT) is proposed for the classification of emotions in PD and normal controls (NC). EEG signals of six emotional states namely happiness, sadness, fear, anger, surprise, and disgust are studied. Power, entropy, and statistical moments based features are elicited from the highpass and lowpass sub-bands of TQWT. Six features selected by statistical analysis are classified with a k-nearest neighbor, probabilistic neural network, random forest, decision tree, and extreme learning machine. Three performance measures are obtained, maximum mean accuracy, sensitivity, and specificity of 96.16%, 97.59%, and 88.51% for NC and 93.88%, 96.33%, and 81.67% for PD are achieved with a probabilistic neural network. The proposed method proved to be very effective such that it classifies emotions in PD and could be used as a potential tool for diagnosing emotional impairment in hospitals.

Two-stepped majority voting for efficient EEG-based emotion classification

In this paper, a novel approach that is based on two-stepped majority voting is proposed for efficient EEG-based emotion classification. Emotion recognition is important for human–machine interactions. Facial features- and body gestures-based approaches have been generally proposed for emotion recognition. Recently, EEG-based approaches become more popular in emotion recognition. In the proposed approach, the raw EEG signals are initially low-pass filtered for noise removal and band-pass filters are used for rhythms extraction. For each rhythm, the best performed EEG channels are determined based on wavelet-based entropy features and fractal dimension-based features. The k-nearest neighbor (KNN) classifier is used in classification. The best five EEG channels are used in majority voting for getting the final predictions for each EEG rhythm. In the second majority voting step, the predictions from all rhythms are used to get a final prediction. The DEAP dataset is used in experiments and classification accuracy, sensitivity and specificity are used for performance evaluation metrics. The experiments are carried out to classify the emotions into two binary classes such as high valence (HV) vs low valence (LV) and high arousal (HA) vs low arousal (LA). The experiments show that 86.3% HV vs LV discrimination accuracy and 85.0% HA vs LA discrimination accuracy is obtained. The obtained results are also compared with some of the existing methods. The comparisons show that the proposed method has potential in the use of EEG-based emotion classification.

Emotion classification using EEG signal for women safety application based on deep learning

An emotion is a conscious logical response that varies for different situations in women’s life. These mental responses are caused by physiological, cognitive, and behavioral changes. Gender-based violence undermines the participation of women in decision-making, resulting in a decline in their quality of life. More accurate and automatic classification of women’s emotions can enhance human-computer interfaces and security in real time. There are some wearable technologies and mobile applications that claim to ensure the safety of women. However, they rely on limited social action and are ineffective at ensuring women’s safety when and where it is needed. In this work, a novel CDB-LSTM network has been proposed to accurately classify the emotions of women in seven different classes. The electroencephalogram (EEG) offers non-radioactive methods of identifying emotions. Initially, the EEG signals are preprocessed and they are converted into images via Time-Frequency Representation (TPR). A smoothed pseudo-Wigner-Ville distribution (SPWVD) is employed to convert the EEG time-domain signals into input images. Consequently, these converted images are given as input to the Convolutional Deep Belief Network (CDBN) for extracting the most relevant features. Finally, Bi-directional LSTM is used for classifying the emotions of women into seven classes namely: happy, relax, sad, fear, anxiety, anger, and stress. The proposed CDB-LSTM network preserves the high accuracy range of 97.27% in the validation phase. The proposed CDB-LSTM network improves the overall accuracy by 6.20% 32.98% 6.85% and 3.30% better than CNN-LSTM, Multi-domain feature fusion model, GCNN-LSTM and CNN with SVM and DT respectively.