Automated prediction of preferences using facial expressions

Predicting Activation Liking of People With Dementia

Physical, social and cognitive activation is an important cornerstone in non-pharmacological therapy for People with Dementia (PwD). To support long-term motivation and well-being, activation contents first need to be perceived positively. Prompting for explicit feedback, however, is intrusive and interrupts the activation flow. Automated analyses of verbal and non-verbal signals could provide an unobtrusive means of recommending suitable contents based on implicit feedback. In this study, we investigate the correlation between engagement responses and self-reported activation ratings. Subsequently, we predict ratings of PwD based on verbal and non-verbal signals in an unconstrained care setting. Applying Long-Short-Term-Memory (LSTM) networks, we can show that our classifier outperforms chance level. We further investigate which features are the most promising indicators for the prediction of activation ratings of PwD.

Opportunities and Challenges for Using Automatic Human Affect Analysis in Consumer Research

The ability to automatically assess emotional responses via contact-free video recording taps into a rapidly growing market aimed at predicting consumer choices. If consumer attention and engagement are measurable in a reliable and accessible manner, relevant marketing decisions could be informed by objective data. Although significant advances have been made in automatic affect recognition, several practical and theoretical issues remain largely unresolved. These concern the lack of cross-system validation, a historical emphasis of posed over spontaneous expressions, as well as more fundamental issues regarding the weak association between subjective experience and facial expressions. To address these limitations, the present paper argues that extant commercial and free facial expression classifiers should be rigorously validated in cross-system research. Furthermore, academics and practitioners must better leverage fine-grained emotional response dynamics, with stronger emphasis on understanding naturally occurring spontaneous expressions, and in naturalistic choice settings. We posit that applied consumer research might be better situated to examine facial behavior in socio-emotional contexts rather than decontextualized, laboratory studies, and highlight how AHAA can be successfully employed in this context. Also, facial activity should be considered less as a single outcome variable, and more as a starting point for further analyses. Implications of this approach and potential obstacles that need to be overcome are discussed within the context of consumer research.

Contributions of shape and reflectance information to social judgments from faces

Face perception is based on both shape and reflectance information. However, we know little about the relative contribution of these kinds of information to social judgments of faces. In Experiment 1, we generated faces using validated computational models of attractiveness, competence, dominance, extroversion, and trustworthiness. Faces were manipulated orthogonally on five levels of shape and reflectance for each model. Both kinds of information had linear and additive effects on participants' social judgments. Shape information was more predictive of dominance, extroversion, and trustworthiness judgments, whereas reflectance information was more predictive of competence judgments. In Experiment 2, to test whether the amount of visual information alters the relative contribution of shape and reflectance information, we presented faces - varied on attractiveness, competence, and dominance - for five different durations (33-500 ms). For all judgments, the linear effect of both shape and reflectance increased as duration increased. Importantly, the relative contribution did not change across durations. These findings show that that the judged dimension is critical for which kind of information is weighted more heavily in judgments and that the relative contribution of shape and reflectance is stable across the amount of visual information available.