A Survey on Physiological Signal-Based Emotion Recognition

Machine learning model with output correction: Towards reliable bradycardia detection in neonates

Bradycardia is a commonly occurring condition in premature infants, often causing serious consequences and cardiovascular complications. Reliable and accurate detection of bradycardia events is pivotal for timely intervention and effective treatment. Excessive false alarms pose a critical problem in bradycardia event detection, eroding trust in machine learning (ML)-based clinical decision support tools designed for such detection. This could result in disregarding the algorithm's accurate recommendations and disrupting workflows, potentially compromising the quality of patient care. This article introduces an ML-based approach incorporating an output correction element, designed to minimise false alarms. The approach has been applied to bradycardia detection in preterm infants. We applied five ML-based autoencoder techniques, using recurrent neural network (RNN), long-short-term memory (LSTM), gated recurrent unit (GRU), 1D convolutional neural network (1D CNN), and a combination of 1D CNN and LSTM. The analysis is performed on ∼440 hours of real-time preterm infant data. The proposed approach achieved 0.978, 0.73, 0.992, 0.671 and 0.007 in AUC-ROC, AUC-PRC, recall, F1 score, and false positive rate (FPR) respectively and a false alarms reduction of 36% when compared with methods without the correction approach. This study underscores the imperative of cultivating solutions that alleviate alarm fatigue and encourage active engagement among healthcare professionals.

Facial expression recognition (FER) survey: a vision, architectural elements, and future directions

With the cutting-edge advancements in computer vision, facial expression recognition (FER) is an active research area due to its broad practical applications. It has been utilized in various fields, including education, advertising and marketing, entertainment and gaming, health, and transportation. The facial expression recognition-based systems are rapidly evolving due to new challenges, and significant research studies have been conducted on both basic and compound facial expressions of emotions; however, measuring emotions is challenging. Fueled by the recent advancements and challenges to the FER systems, in this article, we have discussed the basics of FER and architectural elements, FER applications and use-cases, FER-based global leading companies, interconnection between FER, Internet of Things (IoT) and Cloud computing, summarize open challenges in-depth to FER technologies, and future directions through utilizing Preferred Reporting Items for Systematic reviews and Meta Analyses Method (PRISMA). In the end, the conclusion and future thoughts are discussed. By overcoming the identified challenges and future directions in this research study, researchers will revolutionize the discipline of facial expression recognition in the future.

A Comparison of Personalized and Generalized Approaches to Emotion Recognition Using Consumer Wearable Devices: Machine Learning Study

BACKGROUND

There are a wide range of potential adverse health effects, ranging from headaches to cardiovascular disease, associated with long-term negative emotions and chronic stress. Because many indicators of stress are imperceptible to observers, the early detection of stress remains a pressing medical need, as it can enable early intervention. Physiological signals offer a noninvasive method for monitoring affective states and are recorded by a growing number of commercially available wearables.

OBJECTIVE

We aim to study the differences between personalized and generalized machine learning models for 3-class emotion classification (neutral, stress, and amusement) using wearable biosignal data.

METHODS

We developed a neural network for the 3-class emotion classification problem using data from the Wearable Stress and Affect Detection (WESAD) data set, a multimodal data set with physiological signals from 15 participants. We compared the results between a participant-exclusive generalized, a participant-inclusive generalized, and a personalized deep learning model.

RESULTS

For the 3-class classification problem, our personalized model achieved an average accuracy of 95.06% and an F1-score of 91.71%; our participant-inclusive generalized model achieved an average accuracy of 66.95% and an F1-score of 42.50%; and our participant-exclusive generalized model achieved an average accuracy of 67.65% and an F1-score of 43.05%.

CONCLUSIONS

Our results emphasize the need for increased research in personalized emotion recognition models given that they outperform generalized models in certain contexts. We also demonstrate that personalized machine learning models for emotion classification are viable and can achieve high performance.

Emotion differentiation through features of eye-tracking and pupil diameter for monitoring well-being

Emotions are an important contributor to human self-expression and well-being. However, many populations express their emotions differently from what is considered "typical". Previous literature has indicated a possible relationship between emotion and eye-movement. The objective of this paper is to further explore this proposed relationship by identifying specific features of eye-movement that relate to six emotion categories: joy, surprise, indifference, disgust, sadness, and fear. Features of eye-movement are extracted from measurements of pupil diameter, saccades, and fixations. These measurements are collected as participants view images from the International Affective Picture System, a validated image deck used to evoke known levels of pleasure, arousal, and dominance. Example features of eye-movement measurements such as pupil diameter include maximum or minimum values, means, and standard deviations. Statistical analyses indicate that the extracted features of eye-tracking in this paper can identify fear and sadness with relative accuracy, while more work is needed to differentiate among joy, indifference, disgust, and surprise. Future work aims to understand differences between typically developing populations such as the individuals included in this analysis, and clinical populations such as individuals with cerebral palsy.Clinical Relevance- This pilot study suggests a link between emotion and features of eye-movement. The information will later be used to develop assistive communication devices that better meet the self-expression needs of individuals with motor and communication challenges.

Multi-Input Speech Emotion Recognition Model Using Mel Spectrogram and GeMAPS

The existing research on emotion recognition commonly uses mel spectrogram (MelSpec) and Geneva minimalistic acoustic parameter set (GeMAPS) as acoustic parameters to learn the audio features. MelSpec can represent the time-series variations of each frequency but cannot manage multiple types of audio features. On the other hand, GeMAPS can handle multiple audio features but fails to provide information on their time-series variations. Thus, this study proposes a speech emotion recognition model based on a multi-input deep neural network that simultaneously learns these two audio features. The proposed model comprises three parts, specifically, for learning MelSpec in image format, learning GeMAPS in vector format, and integrating them to predict the emotion. Additionally, a focal loss function is introduced to address the imbalanced data problem among the emotion classes. The results of the recognition experiments demonstrate weighted and unweighted accuracies of 0.6657 and 0.6149, respectively, which are higher than or comparable to those of the existing state-of-the-art methods. Overall, the proposed model significantly improves the recognition accuracy of the emotion "happiness", which has been difficult to identify in previous studies owing to limited data. Therefore, the proposed model can effectively recognize emotions from speech and can be applied for practical purposes with future development.