Cross corpus multi-lingual speech emotion recognition using ensemble learning

Emotion recognition for human-computer interaction using high-level descriptors

Recent research has focused extensively on employing Deep Learning (DL) techniques, particularly Convolutional Neural Networks (CNN), for Speech Emotion Recognition (SER). This study addresses the burgeoning interest in leveraging DL for SER, specifically focusing on Punjabi language speakers. The paper presents a novel approach to constructing and preprocessing a labeled speech corpus using diverse social media sources. By utilizing spectrograms as the primary feature representation, the proposed algorithm effectively learns discriminative patterns for emotion recognition. The method is evaluated on a custom dataset derived from various Punjabi media sources, including films and web series. Results demonstrate that the proposed approach achieves an accuracy of 69%, surpassing traditional methods like decision trees, Naïve Bayes, and random forests, which achieved accuracies of 49%, 52%, and 61% respectively. Thus, the proposed method improves accuracy in recognizing emotions from Punjabi speech signals.

Speech emotion recognition via graph-based representations

Speech emotion recognition (SER) has gained an increased interest during the last decades as part of enriched affective computing. As a consequence, a variety of engineering approaches have been developed addressing the challenge of the SER problem, exploiting different features, learning algorithms, and datasets. In this paper, we propose the application of the graph theory for classifying emotionally-colored speech signals. Graph theory provides tools for extracting statistical as well as structural information from any time series. We propose to use the mentioned information as a novel feature set. Furthermore, we suggest setting a unique feature-based identity for each emotion belonging to each speaker. The emotion classification is performed by a Random Forest classifier in a Leave-One-Speaker-Out Cross Validation (LOSO-CV) scheme. The proposed method is compared with two state-of-the-art approaches involving well known hand-crafted features as well as deep learning architectures operating on mel-spectrograms. Experimental results on three datasets, EMODB (German, acted) and AESDD (Greek, acted), and DEMoS (Italian, in-the-wild), reveal that our proposed method outperforms the comparative methods in these datasets. Specifically, we observe an average UAR increase of almost [Formula: see text], [Formula: see text] and [Formula: see text], respectively.

Cross-Corpus Speech Emotion Recognition Based on Transfer Learning and Multi-Loss Dynamic Adjustment

In this paper, we do research on cross-corpus speech emotion recognition (SER), in which the training and testing speech signals come from different speech corpus. The mismatched feature distribution between the training and testing sets makes many classical algorithms unable to achieve better results. To deal with this issue, a transfer learning and multi-loss dynamic adjustment (TLMLDA) algorithm is initiatively proposed in this paper. The proposed algorithm first builds a novel deep network model based on a deep auto-encoder and fully connected layers to improve the representation ability of features. Subsequently, global domain and subdomain adaptive algorithms are jointly adopted to implement features transfer. Finally, dynamic weighting factors are constructed to adjust the contribution of different loss functions to prevent optimization offset of model training, which effectively improve the generalization ability of the whole system. The results of simulation experiments on Berlin, eNTERFACE, and CASIA speech corpora show that the proposed algorithm can achieve excellent recognition results, and it is competitive with most of the state-of-the-art algorithms.

Mobile Robot Application with Hierarchical Start Position DQN

Advances in deep learning significantly affect reinforcement learning, which results in the emergence of Deep RL (DRL). DRL does not need a data set and has the potential beyond the performance of human experts, resulting in significant developments in the field of artificial intelligence. However, because a DRL agent has to interact with the environment a lot while it is trained, it is difficult to be trained directly in the real environment due to the long training time, high cost, and possible material damage. Therefore, most or all of the training of DRL agents for real-world applications is conducted in virtual environments. This study focused on the difficulty in a mobile robot to reach its target by making a path plan in a real-world environment. The Minimalistic Gridworld virtual environment has been used for training the DRL agent, and to our knowledge, we have implemented the first real-world implementation for this environment. A DRL algorithm with higher performance than the classical Deep Q-network algorithm was created with the expanded environment. A mobile robot was designed for use in a real-world application. To match the virtual environment with the real environment, algorithms that can detect the position of the mobile robot and the target, as well as the rotation of the mobile robot, were created. As a result, a DRL-based mobile robot was developed that uses only the top view of the environment and can reach its target regardless of its initial position and rotation.

Global and local feature fusion via long and short-term memory mechanism for dance emotion recognition in robot

In recent years, there are more and more intelligent machines in people's life, such as intelligent wristbands, sweeping robots, intelligent learning machines and so on, which can simply complete a single execution task. We want robots to be as emotional as humans. In this way, human-computer interaction can be more natural, smooth and intelligent. Therefore, emotion research has become a hot topic that researchers pay close attention to. In this paper, we propose a new dance emotion recognition based on global and local feature fusion method. If the single feature of audio is extracted, the global information of dance cannot be reflected. And the dimension of data features is very high. In this paper, an improved long and short-term memory (LSTM) method is used to extract global dance information. Linear prediction coefficient is used to extract local information. Considering the complementarity of different features, a global and local feature fusion method based on discriminant multi-canonical correlation analysis is proposed in this paper. Experimental results on public data sets show that the proposed method can effectively identify dance emotion compared with other state-of-the-art emotion recognition methods.

Cardiovascular Disease Detection using Ensemble Learning

One of the most challenging tasks for clinicians is detecting symptoms of cardiovascular disease as earlier as possible. Many individuals worldwide die each year from cardiovascular disease. Since heart disease is a major concern, it must be dealt with timely. Multiple variables affecting health, such as excessive blood pressure, elevated cholesterol, an irregular pulse rate, and many more, make it challenging to diagnose cardiac disease. Thus, artificial intelligence can be useful in identifying and treating diseases early on. This paper proposes an ensemble-based approach that uses machine learning (ML) and deep learning (DL) models to predict a person’s likelihood of developing cardiovascular disease. We employ six classification algorithms to predict cardiovascular disease. Models are trained using a publicly available dataset of cardiovascular disease cases. We use random forest (RF) to extract important cardiovascular disease features. The experiment results demonstrate that the ML ensemble model achieves the best disease prediction accuracy of 88.70%.

Authorship identification using ensemble learning

With time, textual data is proliferating, primarily through the publications of articles. With this rapid increase in textual data, anonymous content is also increasing. Researchers are searching for alternative strategies to identify the author of an unknown text. There is a need to develop a system to identify the actual author of unknown texts based on a given set of writing samples. This study presents a novel approach based on ensemble learning, DistilBERT, and conventional machine learning techniques for authorship identification. The proposed approach extracts the valuable characteristics of the author using a count vectorizer and bi-gram Term frequency-inverse document frequency (TF-IDF). An extensive and detailed dataset, “All the news” is used in this study for experimentation. The dataset is divided into three subsets (article1, article2, and article3). We limit the scope of the dataset and selected ten authors in the first scope and 20 authors in the second scope for experimentation. The experimental results of proposed ensemble learning and DistilBERT provide better performance for all the three subsets of the “All the news” dataset. In the first scope, the experimental results prove that the proposed ensemble learning approach from 10 authors provides a better accuracy gain of 3.14% and from DistilBERT 2.44% from the article1 dataset. Similarly, in the second scope from 20 authors, the proposed ensemble learning approach provides a better accuracy gain of 5.25% and from DistilBERT 7.17% from the article1 dataset, which is better than previous state-of-the-art studies.

Social relationship analysis using state-of-the-art embeddings

Detection of human relationships from their interactions on social media is a challenging problem with a wide range of applications in different areas like targeted marketing, cyber-crime, fraud, defense, planning, human resource, to name a few. All previous work in this area has only dealt with the most basic types of relationships. The proposed approach goes beyond the previous work to efficiently handle the hierarchy of social relationships. This paper introduces a novel technique named Quantifiable Social Relationship (QSR) analysis for quantifying social relationships to analyze relationships between agents from their textual conversations. QSR uses cross-disciplinary techniques from computational linguistics and cognitive psychology to identify relationships. QSR utilizes sentiment and behavioral styles displayed in the conversations for mapping them onto level II relationship categories. Then, for identifying the level III relationship categories, QSR uses level II relationships, sentiments, interactions, and word embeddings as key features. QSR employs natural language processing techniques for feature engineering and state-of-the-art embeddings generated by word2vec, global vectors (glove), and bidirectional encoder representations from transformers (bert). QSR combines the intrinsic conversational features with word embeddings for classifying relationships. QSR achieves an accuracy of up to 89% for classifying relationship subtypes. The evaluation shows that QSR can accurately identify the hierarchical relationships between agents by extracting intrinsic and extrinsic features from textual conversations between agents.

Hypertuned Deep Convolutional Neural Network for Sign Language Recognition

Sign language plays a pivotal role in the lives of impaired people having speaking and hearing disabilities. They can convey messages using hand gesture movements. American Sign Language (ASL) recognition is challenging due to the increasing intra-class similarity and high complexity. This paper used a deep convolutional neural network for ASL alphabet recognition to overcome ASL recognition challenges. This paper presents an ASL recognition approach using a deep convolutional neural network. The performance of the DeepCNN model improves with the amount of given data; for this purpose, we applied the data augmentation technique to expand the size of training data from existing data artificially. According to the experiments, the proposed DeepCNN model provides consistent results for the ASL dataset. Experiments prove that the DeepCNN gives a better accuracy gain of 19.84%, 8.37%, 16.31%, 17.17%, 5.86%, and 3.26% as compared to various state-of-the-art approaches.

A Novel Deep Auto-Encoder Based Linguistics Clustering Model for Social Text

The wide adoption of media and social media has increased the amount of digital content to an enormous level. Natural language processing (NLP) techniques provide an opportunity to extract and explore meaningful information from a large amount of text. Among natural languages, Urdu is one of the widely used languages worldwide for spoken and written communications. Due to its wide adopt-ability, digital content in the Urdu language is increasing briskly, especially with social media and online NEWS feeds. Government agencies and advertisers must filter and understand the content to analyze the trends and cohorts in their interest and national prerogative. Clustering is considered a baseline and one of the first steps in natural language understanding. There are many state-of-the-art clustering techniques specifically for English, French, and Arabic, but no significant research has been conducted in Urdu language processing. Doing it for short text segments is challenging because of limited features and the absence of meaningful language discourse and nuance. Many rule-based NLP techniques are adopted to overcome these issues, relying on human-designed features and rules. Therefore, these methods do not promise remarkable results. Alongside NLP, deep learning techniques are pretty efficient in capturing contextual information with minimal noise compared to other traditional methods. By taking on this challenging job, we develop a deep learning-based technique for Urdu short text clustering for the very first time without a human-designed feature. In this paper, we propose a method of short text clustering using a deep neural network that automatically learns feature representations and clustering assignments simultaneously. This method learns clustering objectives by converting the high dimensional feature space to a low dimensional feature space. Our experiments on the Urdu NEWS headlines dataset show remarkable results compared to state-of-the-art methods.

Context Aware Emotion Detection from Low Resource Urdu Language using Deep Neural Network

Emotion detection (ED) plays a vital role in determining individual interest in any field. Humans use gestures, facial expressions, voice pitch, and choose words to describe their emotions. Significant work has been done to detect emotions from the textual data in English, French, Chinese, and other high-resource languages. However, emotion classification has not been well studied in low-resource languages (i.e., Urdu) due to the lack of labeled corpora. This paper presents a publicly available Urdu Nastalique Emotions Dataset ( UNED ) of sentences and paragraphs annotated with different emotions and proposes a deep learning (DL) based technique for classifying emotions in the UNED corpus. Our annotated UNED corpus has six emotions for both paragraphs and sentences. We perform extensive experimentation to evaluate the quality of the corpus and further classify it using machine learning and DL approaches. Experimental results show that the developed DL-based model performs better than generic machine learning approaches with an F1 score of 85% on UNED sentence-based corpus and 50% on UNED paragraph-based corpus.

The Emotion Probe: On the Universality of Cross-Linguistic and Cross-Gender Speech Emotion Recognition via Machine Learning

Machine Learning (ML) algorithms within a human–computer framework are the leading force in speech emotion recognition (SER). However, few studies explore cross-corpora aspects of SER; this work aims to explore the feasibility and characteristics of a cross-linguistic, cross-gender SER. Three ML classifiers (SVM, Naïve Bayes and MLP) are applied to acoustic features, obtained through a procedure based on Kononenko’s discretization and correlation-based feature selection. The system encompasses five emotions (disgust, fear, happiness, anger and sadness), using the Emofilm database, comprised of short clips of English movies and the respective Italian and Spanish dubbed versions, for a total of 1115 annotated utterances. The results see MLP as the most effective classifier, with accuracies higher than 90% for single-language approaches, while the cross-language classifier still yields accuracies higher than 80%. The results show cross-gender tasks to be more difficult than those involving two languages, suggesting greater differences between emotions expressed by male versus female subjects than between different languages. Four feature domains, namely, RASTA, F0, MFCC and spectral energy, are algorithmically assessed as the most effective, refining existing literature and approaches based on standard sets. To our knowledge, this is one of the first studies encompassing cross-gender and cross-linguistic assessments on SER.

Short-Text Classification Detector: A Bert-Based Mental Approach

With the continuous development of the Internet, social media based on short text has become popular. However, the sparsity and shortness of essays will restrict the accuracy of text classification. Therefore, based on the Bert model, we capture the mental feature of reviewers and apply them for short text classification to improve its classification accuracy. Specifically, we construct a model text at the language level and fine tune the model to better embed mental features. To verify the accuracy of this method, we compare a variety of machine learning methods, such as support vector machine, convolution neural networks, and recurrent neural networks. The results show the following: (1) Through feature comparison, it is found that mental features can significantly improve the accuracy of short text classification. (2) Combining mental features and text as input vectors can provide more classification accuracy than separating them as two independent vectors. (3) Through model comparison, it can be found that Bert model can integrate mental features and short text. Bert can better capture mental features to improve the accuracy of classification results. This will help to promote the development of short text classification.

Deep Learning for Depression Detection from Textual Data

Depression is a prevalent sickness, spreading worldwide with potentially serious implications. Timely recognition of emotional responses plays a pivotal function at present, with the profound expansion of social media and users of the internet. Mental illnesses are highly hazardous, stirring more than three hundred million people. Moreover, that is why research is focused on this subject. With the advancements of machine learning and the availability of sample data relevant to depression, there is the possibility of developing an early depression diagnostic system, which is key to lessening the number of afflicted individuals. This paper proposes a productive model by implementing the Long-Short Term Memory (LSTM) model, consisting of two hidden layers and large bias with Recurrent Neural Network (RNN) with two dense layers, to predict depression from text, which can be beneficial in protecting individuals from mental disorders and suicidal affairs. We train RNN on textual data to identify depression from text, semantics, and written content. The proposed framework achieves 99.0% accuracy, higher than its counterpart, frequency-based deep learning models, whereas the false positive rate is reduced. We also compare the proposed model with other models regarding its mean accuracy. The proposed approach indicates the feasibility of RNN and LSTM by achieving exceptional results for early recognition of depression in the emotions of numerous social media subscribers.