Automated measurement of infant and mother Duchenne facial expressions in the Face-to-Face/Still-Face

Automated facial expression measurement in a longitudinal sample of 4- and 8-month-olds: Baby FaceReader 9 and manual coding of affective expressions

Facial expressions are among the earliest behaviors infants use to express emotional states, and are crucial to preverbal social interaction. Manual coding of infant facial expressions, however, is laborious and poses limitations to replicability. Recent developments in computer vision have advanced automated facial expression analyses in adults, providing reproducible results at lower time investment. Baby FaceReader 9 is commercially available software for automated measurement of infant facial expressions, but has received little validation. We compared Baby FaceReader 9 output to manual micro-coding of positive, negative, or neutral facial expressions in a longitudinal dataset of 58 infants at 4 and 8 months of age during naturalistic face-to-face interactions with the mother, father, and an unfamiliar adult. Baby FaceReader 9's global emotional valence formula yielded reasonable classification accuracy (AUC = .81) for discriminating manually coded positive from negative/neutral facial expressions; however, the discrimination of negative from neutral facial expressions was not reliable (AUC = .58). Automatically detected a priori action unit (AU) configurations for distinguishing positive from negative facial expressions based on existing literature were also not reliable. A parsimonious approach using only automatically detected smiling (AU12) yielded good performance for discriminating positive from negative/neutral facial expressions (AUC = .86). Likewise, automatically detected brow lowering (AU3+AU4) reliably distinguished neutral from negative facial expressions (AUC = .79). These results provide initial support for the use of selected automatically detected individual facial actions to index positive and negative affect in young infants, but shed doubt on the accuracy of complex a priori formulas.

The origins and development of aesthetics

All people (and some other animals) have aesthetic responses to sensory stimulation, responses of emotional pleasure or displeasure. These emotions vary from one person and culture to another, yet they share a common mechanism. To survive, an adaptive animal (as opposed to a tropic animal) needs to become comfortable with normality and to have slight abnormalities draw attention to themselves. Walking through a jungle you need to notice a tiger from a single stripe: if you must wait to see the whole animal, you are unlikely to survive. In Homo sapiens, the brain's adaptive neurochemistry does this naturally, partly because the brain's neuronal networks are structured to react efficiently to fractal structures, structures that shape much of nature. In addition, previous associations may turn a slight variation from normal into feelings of either pleasure or danger. The details of these responses-what is normal and what variations feel like-will depend upon an individual's experience, but the mechanism is the same, no matter whether a person is tasting a wine, seeing a face or landscape, or hearing a song. This article is part of the theme issue 'Sensing and feeling: an integrative approach to sensory processing and emotional experience'.