Development and Progress in Sensors and Technologies for Human Emotion Recognition

Leveraging Wearable Sensors in Virtual Reality Driving Simulators: A Review of Techniques and Applications

Virtual reality (VR) driving simulators are very promising tools for driver assessment since they provide a controlled and adaptable setting for behavior analysis. At the same time, wearable sensor technology provides a well-suited and valuable approach to evaluating the behavior of drivers and their physiological or psychological state. This review paper investigates the potential of wearable sensors in VR driving simulators. Methods: A literature search was performed on four databases (Scopus, Web of Science, Science Direct, and IEEE Xplore) using appropriate search terms to retrieve scientific articles from a period of eleven years, from 2013 to 2023. Results: After removing duplicates and irrelevant papers, 44 studies were selected for analysis. Some important aspects were extracted and presented: the number of publications per year, countries of publication, the source of publications, study aims, characteristics of the participants, and types of wearable sensors. Moreover, an analysis and discussion of different aspects are provided. To improve car simulators that use virtual reality technologies and boost the effectiveness of particular driver training programs, data from the studies included in this systematic review and those scheduled for the upcoming years may be of interest.

Assessment of a Person's Emotional State Based on His or Her Posture Parameters

This article is devoted to the study of the correlation between the emotional state of a person and the posture of his or her body in the sitting position. In order to carry out the study, we developed the first version of the hardware-software system based on a posturometric armchair, allowing the characteristics of the posture of a sitting person to be evaluated using strain gauges. Using this system, we revealed the correlation between sensor readings and human emotional states. We showed that certain readings of a sensor group are formed for a certain emotional state of a person. We also found that the groups of triggered sensors, their composition, their number, and their location are related to the states of a particular person, which led to the need to build personalized digital pose models for each person. The intellectual component of our hardware-software complex is based on the concept of co-evolutionary hybrid intelligence. The system can be used during medical diagnostic procedures and rehabilitation processes, as well as in controlling people whose professional activity is connected with increased psycho-emotional load and can cause cognitive disorders, fatigue, and professional burnout and can lead to the development of diseases.

The Use of Wearable Technologies in the Assessment of Physical Activity in Preschool- and School-Age Youth: Systematic Review

In recent years, physical activity assessment has increasingly relied on wearable monitors to provide measures for surveillance, intervention, and epidemiological research. This present systematic review aimed to examine the current research about the utilization of wearable technology in the evaluation in physical activities of preschool- and school-age children. A database search (Web of Science, PubMed and Scopus) for original research articles was performed. A total of twenty-one articles met the inclusion criteria, and the Cochrane risk of bias tool was used. Wearable technology can actually be a very important instrument/tool to detect the movements and monitor the physical activity of children and adolescents. The results revealed that there are a few studies on the influence of these technologies on physical activity in schools, and most of them are descriptive. In line with previous research, the wearable devices can be used as a motivational tool to improve PA behaviors and in the evaluation of PA interventions. However, the different reliability levels of the different devices used in the studies can compromise the analysis and understanding of the results.

Applying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals

The use of machine learning (ML) techniques in affective computing applications focuses on improving the user experience in emotion recognition. The collection of input data (e.g., physiological signals), together with expert annotations are part of the established standard supervised learning methodology used to train human emotion recognition models. However, these models generally require large amounts of labeled data, which is expensive and impractical in the healthcare context, in which data annotation requires even more expert knowledge. To address this problem, this paper explores the use of the self-supervised learning (SSL) paradigm in the development of emotion recognition methods. This approach makes it possible to learn representations directly from unlabeled signals and subsequently use them to classify affective states. This paper presents the key concepts of emotions and how SSL methods can be applied to recognize affective states. We experimentally analyze and compare self-supervised and fully supervised training of a convolutional neural network designed to recognize emotions. The experimental results using three emotion datasets demonstrate that self-supervised representations can learn widely useful features that improve data efficiency, are widely transferable, are competitive when compared to their fully supervised counterparts, and do not require the data to be labeled for learning.

Machine Learning Algorithms for Detection and Classifications of Emotions in Contact Center Applications

Over the past few years, virtual assistant solutions used in Contact Center systems are gaining popularity. One of the main tasks of the virtual assistant is to recognize the intentions of the customer. It is important to note that quite often the actual intention expressed in a conversation is also directly influenced by the emotions that accompany that conversation. Unfortunately, scientific literature has not identified what specific types of emotions in Contact Center applications are relevant to the activities they perform. Therefore, the main objective of this work was to develop an Emotion Classification for Machine Detection of Affect-Tinged Conversational Contents dedicated directly to the Contact Center industry. In the conducted study, Contact Center voice and text channels were considered, taking into account the following families of emotions: anger, fear, happiness, sadness vs. affective neutrality of the statements. The obtained results confirmed the usefulness of the proposed classification—for the voice channel, the highest efficiency was obtained using the Convolutional Neural Network (accuracy, 67.5%; precision, 80.3; F1-Score, 74.5%), while for the text channel, the Support Vector Machine algorithm proved to be the most efficient (accuracy, 65.9%; precision, 58.5; F1-Score, 61.7%).

EmotIoT: An IoT System to Improve Users’ Wellbeing

IoT provides applications and possibilities to improve people’s daily lives and business environments. However, most of these technologies have not been exploited in the field of emotions. With the amount of data that can be collected through IoT, emotions could be detected and anticipated. Since the study of related works indicates a lack of methodological approaches in designing IoT systems from the perspective of emotions and smart adaption rules, we introduce a methodology that can help design IoT systems quickly in this scenario, where the detection of users is valuable. In order to test the methodology presented, we apply the proposed stages to design an IoT smart recommender system named EmotIoT. The system allows anticipating and predicting future users’ emotions using parameters collected from IoT devices. It recommends new activities for the user in order to obtain a final state. Test results validate our recommender system as it has obtained more than 80% accuracy in predicting future user emotions.

An Approach to Assessing Shopper Acceptance of Beacon Triggered Promotions in Smart Retail

This paper studies shopper acceptance for using beacons in the purchase process. The main goal is to examine shopper response to beacon-triggered promotions and propose a model that would help retail practitioners plan the implementation of beacons in stores. The model was evaluated via an in-market test to examine the effects of beacon-triggered promotion on shopper attention, technology acceptance, and the decision to purchase. The test was conducted in Belgrade, Serbia in 10 representative stores where beacons were implemented with 10 twin control stores. The SimplyTastly mobile application was used for sending notifications. Furthermore, two more in-market beacon activations were analysed in Croatia and Bulgaria. The results showed that shoppers accepted beacon technology and that beacon-triggered promotion had a positive impact on shopper attention, purchase behaviour, and the decision to purchase. The results show that the proposed model could serve as a sound basis for the implementation of beacon technology in retail.

The Relationship between Landscape Metrics and Facial Expressions in 18 Urban Forest Parks of Northern China

Urban forests are an important green infrastructure that positively impacts human well-being by improving emotions and reducing psychological stress. Questionnaires have been used frequently to study the influence of forest experiences on mental health; however, they have poor controllability and low accuracy for detecting immediate emotions. This study used the alternative approach of facial reading, detecting the facial expressions of urban forest visitors and their relationships with the landscape metrics. Using the microblogging site, Sina Weibo, we collected facial photos of 2031 people visiting 18 different forest parks across Northern China in 2020. We used satellite imagery analysis to assess the elevation and pattern sizes of green space and blue space areas. Age and location were taken as independent variables affecting facial expressions, which were categorized as happy or sad. With increases in green space and intact park areas, people showed a higher frequency of expressing happy scores. The results showed that the forest experience frequently elicited positive emotions, suggesting that creating and maintaining urban green spaces enhance people’s quality of life.

Dimensioning of Wide-Area Alternate Wetting and Drying (AWD) System for IoT-Based Automation

Water, one of the most valuable resources, is underutilized in irrigated rice production. The yield of rice, a staple food across the world, is highly dependent on having proper irrigation systems. Alternate wetting and drying (AWD) is an effective irrigation method mainly used for irrigated rice production. However, unattended, manual, small-scale, and discrete implementations cannot achieve the maximum benefit of AWD. Automation of large-scale (over 1000 acres) implementation of AWD can be carried out using wide-area wireless sensor network (WSN). An automated AWD system requires three different WSNs: one for water level and environmental monitoring, one for monitoring of the irrigation system, and another for controlling the irrigation system. Integration of these three different WSNs requires proper dimensioning of the AWD edge elements (sensor and actuator nodes) to reduce the deployment cost and make it scalable. Besides field-level monitoring, the integration of external control parameters, such as real-time weather forecasts, plant physiological data, and input from farmers, can further enhance the performance of the automated AWD system. Internet of Things (IoT) can be used to interface the WSNs with external data sources. This research focuses on the dimensioning of the AWD system for the multilayer WSN integration and the required algorithms for the closed loop control of the irrigation system using IoT. Implementation of the AWD for 25,000 acres is shown as a possible use case. Plastic pipes are proposed as the means to transport and control proper distribution of water in the field, which significantly helps to reduce conveyance loss. This system utilizes 250 pumps, grouped into 10 clusters, to ensure equal water distribution amongst the users (field owners) in the wide area. The proposed automation algorithm handles the complexity of maintaining proper water pressure throughout the pipe network, scheduling the pump, and controlling the water outlets. Mathematical models are presented for proper dimensioning of the AWD. A low-power and long-range sensor node is developed due to the lack of cellular data coverage in rural areas, and its functionality is tested using an IoT platform for small-scale field trials.