DRER: Deep Learning-Based Driver's Real Emotion Recognizer

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, A Systematic Review

Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper's scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified.

Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review

In recent years, the rapid development of sensors and information technology has made it possible for machines to recognize and analyze human emotions. Emotion recognition is an important research direction in various fields. Human emotions have many manifestations. Therefore, emotion recognition can be realized by analyzing facial expressions, speech, behavior, or physiological signals. These signals are collected by different sensors. Correct recognition of human emotions can promote the development of affective computing. Most existing emotion recognition surveys only focus on a single sensor. Therefore, it is more important to compare different sensors or unimodality and multimodality. In this survey, we collect and review more than 200 papers on emotion recognition by literature research methods. We categorize these papers according to different innovations. These articles mainly focus on the methods and datasets used for emotion recognition with different sensors. This survey also provides application examples and developments in emotion recognition. Furthermore, this survey compares the advantages and disadvantages of different sensors for emotion recognition. The proposed survey can help researchers gain a better understanding of existing emotion recognition systems, thus facilitating the selection of suitable sensors, algorithms, and datasets.

A Study of Bibliometric Trends in Automotive Human–Machine Interfaces

With the development of autonomous driving technology and the internet, automotive human–machine interface (HMI) technology has become an important part of contemporary automotive design. Currently, global automakers are designing a variety of innovative in-car HMIs that illustrate the direction of automotive design in the new era from the perspective of technological aesthetics and experience design. However, sleek designs and innovative experience methods must be built on the basis of safety. Therefore, it is necessary to summarize existing research in the field of automotive HMI and construct a literature review of automotive design research. In this paper, literature on automotive HMI from the Scopus database was analyzed using bibliometric methods such as descriptive analysis, keyword co-occurrence, and literature co-citation network analysis. The final mapping analysis revealed that the current automotive HMI research literature primarily focuses on user research, interface research, external environment research, and technology implementation research related to automotive HMI. The three main stages of automotive HMI research include conceptual construction, system and technology refinement, and user perception research from the perspective of driver assistance and information recognition. Additionally, burst detection suggests that future research should focus on driver assistance, trust levels, and e-HMI information communication.

Multimodal Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-World Driving

As vehicles provide various services to drivers, research on driver emotion recognition has been expanding. However, current driver emotion datasets are limited by inconsistencies in collected data and inferred emotional state annotations by others. To overcome this limitation, we propose a data collection system that collects multimodal datasets during real-world driving. The proposed system includes a self-reportable HMI application into which a driver directly inputs their current emotion state. Data collection was completed without any accidents for over 122 h of real-world driving using the system, which also considers the minimization of behavioral and cognitive disturbances. To demonstrate the validity of our collected dataset, we also provide case studies for statistical analysis, driver face detection, and personalized driver emotion recognition. The proposed data collection system enables the construction of reliable large-scale datasets on real-world driving and facilitates research on driver emotion recognition. The proposed system is avaliable on GitHub.

A New Event-Triggered Type-3 Fuzzy Control System for Multi-Agent Systems: Optimal Economic Efficient Approach for Actuator Activating

This study presents a new approach for multi-agent systems (MASs). The agent dynamics are approximated by the suggested type-3 (T3) fuzzy logic system (FLS). Some sufficient conditions based on the event-triggered scheme are presented to ensure the stability under less activation of the actuators. New tuning rules are obtained for T3-FLSs form the stability and robustness examination. The effect of perturbations, actuator failures and approximation errors are compensated by the designed adaptive compensators. Simulation results show that the output of all agents well converged to the leader agent under disturbances and faulty conditions. Additionally, it is shown that the suggested event-triggered scheme is effective and the actuators are updated about 20–40% of total sample times.