Exploration of Emotion Dynamics Sensing Using Trapezius EMG and Fingertip Temperature

Sensing emotional valence and arousal dynamics through automated facial action unit analysis

Information about the concordance between dynamic emotional experiences and objective signals is practically useful. Previous studies have shown that valence dynamics can be estimated by recording electrical activity from the muscles in the brows and cheeks. However, whether facial actions based on video data and analyzed without electrodes can be used for sensing emotion dynamics remains unknown. We investigated this issue by recording video of participants' faces and obtaining dynamic valence and arousal ratings while they observed emotional films. Action units (AUs) 04 (i.e., brow lowering) and 12 (i.e., lip-corner pulling), detected through an automated analysis of the video data, were negatively and positively correlated with dynamic ratings of subjective valence, respectively. Several other AUs were also correlated with dynamic valence or arousal ratings. Random forest regression modeling, interpreted using the SHapley Additive exPlanation tool, revealed non-linear associations between the AUs and dynamic ratings of valence or arousal. These results suggest that an automated analysis of facial expression video data can be used to estimate dynamic emotional states, which could be applied in various fields including mental health diagnosis, security monitoring, and education.

Advancements in Sensors and Analyses for Emotion Sensing

Exploring the objective signals associated with subjective emotional states has practical significance [...].

Machine Learning-Based Interpretable Modeling for Subjective Emotional Dynamics Sensing Using Facial EMG

Understanding the association between subjective emotional experiences and physiological signals is of practical and theoretical significance. Previous psychophysiological studies have shown a linear relationship between dynamic emotional valence experiences and facial electromyography (EMG) activities. However, whether and how subjective emotional valence dynamics relate to facial EMG changes nonlinearly remains unknown. To investigate this issue, we re-analyzed the data of two previous studies that measured dynamic valence ratings and facial EMG of the corrugator supercilii and zygomatic major muscles from 50 participants who viewed emotional film clips. We employed multilinear regression analyses and two nonlinear machine learning (ML) models: random forest and long short-term memory. In cross-validation, these ML models outperformed linear regression in terms of the mean squared error and correlation coefficient. Interpretation of the random forest model using the SHapley Additive exPlanation tool revealed nonlinear and interactive associations between several EMG features and subjective valence dynamics. These findings suggest that nonlinear ML models can better fit the relationship between subjective emotional valence dynamics and facial EMG than conventional linear models and highlight a nonlinear and complex relationship. The findings encourage emotion sensing using facial EMG and offer insight into the subjective-physiological association.

Delivery of pleasant stroke touch via robot in older adults

Touch care has clinically positive effects on older adults. Touch can be delivered using robots, addressing the lack of caregivers. A recent study of younger participants showed that stroke touch delivered via robot produced subjective and physiologically positive emotional responses similar to those evoked by human touch. However, whether robotic touch can elicit similar responses in older adults remains unknown. We investigated this topic by assessing subjective rating (valence and arousal) and physiological signals [corrugator and zygomatic electromyography (EMG) and skin conductance response (SCR)] to gentle stroking motions delivered to the backs of older participants by robot and human agents at two different speeds: 2.6 and 8.5 cm/s. Following the recent study, the participants were informed that only the robot strokes them. We compared the difference between the younger (their data from the previous study) and the older participants in their responses when the two agents (a robot and a human) stroked them. Subjectively, data from both younger and older participants showed that 8.5 cm/s stroking was more positive and arousing than 2.6 cm/s stroking for both human and robot agents. Physiologically, data from both younger and older participants showed that 8.5 cm/s stroking induced weaker corrugator EMG activity and stronger SCR activity than the 2.6 cm/s stroking for both agents. These results demonstrate that the overall patterns of the older groups responses were similar to those of the younger group, and suggest that robot-delivered stroke touch can elicit pleasant emotional responses in older adults.

Affective Impressions Recognition under Different Colored Lights Based on Physiological Signals and Subjective Evaluation Method

The design of the light environment plays a critical role in the interaction between people and visual objects in space. Adjusting the space's light environment to regulate emotional experience is more practical for the observers under lighting conditions. Although lighting plays a vital role in spatial design, the effects of colored lights on individuals' emotional experiences are still unclear. This study combined physiological signal (galvanic skin response (GSR) and electrocardiography (ECG)) measurements and subjective assessments to detect the changes in the mood states of observers under four sets of lighting conditions (green, blue, red, and yellow). At the same time, two sets of abstract and realistic images were designed to discuss the relationship between light and visual objects and their influence on individuals' impressions. The results showed that different light colors significantly affected mood, with red light having the most substantial emotional arousal, then blue and green. In addition, GSR and ECG measurements were significantly correlated with impressions evaluation results of interest, comprehension, imagination, and feelings in subjective evaluation. Therefore, this study explores the feasibility of combining the measurement of GSR and ECG signals with subjective evaluations as an experimental method of light, mood, and impressions, which provided empirical evidence for regulating individuals' emotional experiences.