Automatic Recognition of Macaque Facial Expressions for Detection of Affective States

Quantifying Facial Gestures Using Deep Learning in a New World Monkey

Facial gestures are a crucial component of primate multimodal communication. However, current methodologies for extracting facial data from video recordings are labor-intensive and prone to human subjectivity. Although automatic tools for this task are still in their infancy, deep learning techniques are revolutionizing animal behavior research. This study explores the distinctiveness of facial gestures in cotton-top tamarins, quantified using markerless pose estimation algorithms. From footage of captive individuals, we extracted and manually labeled frames to develop a model that can recognize a custom set of landmarks positioned on the face of the target species. The trained model predicted landmark positions and subsequently transformed them into distance matrices representing landmarks' spatial distributions within each frame. We employed three competitive machine learning classifiers to assess the ability to automatically discriminate facial configurations that cooccur with vocal emissions and are associated with different behavioral contexts. Initial analysis showed correct classification rates exceeding 80%, suggesting that voiced facial configurations are highly distinctive from unvoiced ones. Our findings also demonstrated varying context specificity of facial gestures, with the highest classification accuracy observed during yawning, social activity, and resting. This study highlights the potential of markerless pose estimation for advancing the study of primate multimodal communication, even in challenging species such as cotton-top tamarins. The ability to automatically distinguish facial gestures in different behavioral contexts represents a critical step in developing automated tools for extracting behavioral cues from raw video data.

GorillaFACS: The Facial Action Coding System for the Gorilla spp

The Facial Action Coding System (FACS) is an objective observation tool for measuring human facial behaviour. It avoids subjective attributions of meaning by objectively measuring independent movements linked to facial muscles, called Action Units (AUs). FACS has been adapted to 11 other taxa, including most apes, macaques and domestic animals, but not yet gorillas. To carry out cross species studies of facial expressions within and beyond apes, gorillas need to be included in such studies. Hence, we developed the GorillaFACS for the Gorilla spp. We followed similar methodology as previous FACS: First, we examined the facial muscular plan of the gorilla. Second, we analysed gorilla videos in a wide variety of contexts to identify their spontaneous facial movements. Third, we classified the individual facial movements according to appearance changes produced by the corresponding underlying musculature. A diverse repertoire of 42 facial movements was identified in the gorilla, including 28 AUs and 14 Action Descriptors, with several new movements not identified in the HumanFACS. Although some of the movements in gorillas differ from humans, the total number of AUs is comparable to the HumanFACS (32 AUs). Importantly, the gorilla's range of facial movements was larger than expected, suggesting a more relevant role in social interactions than what was previously assumed. GorillaFACS is a scientific tool to measure facial movements, and thus, will allow us to better understand the gorilla's expressions and communication. Furthermore, GorillaFACS has the potential be used as an important tool to evaluate this species welfare, particularly in settings of close proximity to humans.

Probing the processing of facial expressions in monkeys via time perception and eye tracking

Accurately recognizing facial expressions is essential for effective social interactions. Non-human primates (NHPs) are widely used in the study of the neural mechanisms underpinning facial expression processing, yet it remains unclear how well monkeys can recognize the facial expressions of other species such as humans. In this study, we systematically investigated how monkeys process the facial expressions of conspecifics and humans using eye-tracking technology and sophisticated behavioral tasks, namely the temporal discrimination task (TDT) and face scan task (FST). We found that monkeys showed prolonged subjective time perception in response to Negative facial expressions in monkeys while showing longer reaction time to Negative facial expressions in humans. Monkey faces also reliably induced divergent pupil contraction in response to different expressions, while human faces and scrambled monkey faces did not. Furthermore, viewing patterns in the FST indicated that monkeys only showed bias toward emotional expressions upon observing monkey faces. Finally, masking the eye region marginally decreased the viewing duration for monkey faces but not for human faces. By probing facial expression processing in monkeys, our study demonstrates that monkeys are more sensitive to the facial expressions of conspecifics than those of humans, thus shedding new light on inter-species communication through facial expressions between NHPs and humans.

Chemogenetic dissection of a prefrontal-hypothalamic circuit for socially subjective reward valuation in macaques

The value of one's own reward is affected by the reward of others, serving as a source for envy. However, it is not known which neural circuits mediate such socially subjective value modulation. Here, we chemogenetically dissected the circuit from the medial prefrontal cortex (MPFC) to the lateral hypothalamus (LH) while male macaques were presented with visual stimuli that concurrently signaled the prospects of one's own and others' rewards. We found that functional disconnection between the MPFC and LH rendered animals significantly less susceptible to others' but not one's own reward prospects. In parallel with this behavioral change, inter-areal coordination, as indexed by coherence and Granger causality, decreased primarily in the delta and theta bands. These findings demonstrate that the MPFC-to-LH circuit plays a crucial role in carrying information about upcoming other-rewards for subjective reward valuation in social contexts.

Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions

Advances in animal motion tracking and pose recognition have been a game changer in the study of animal behavior. Recently, an increasing number of works go ‘deeper’ than tracking, and address automated recognition of animals’ internal states such as emotions and pain with the aim of improving animal welfare, making this a timely moment for a systematization of the field. This paper provides a comprehensive survey of computer vision-based research on recognition of pain and emotional states in animals, addressing both facial and bodily behavior analysis. We summarize the efforts that have been presented so far within this topic—classifying them across different dimensions, highlight challenges and research gaps, and provide best practice recommendations for advancing the field, and some future directions for research.

CalliFACS: The common marmoset Facial Action Coding System

Facial expressions are subtle cues, central for communication and conveying emotions in mammals. Traditionally, facial expressions have been classified as a whole (e.g. happy, angry, bared-teeth), due to automatic face processing in the human brain, i.e., humans categorise emotions globally, but are not aware of subtle or isolated cues such as an eyebrow raise. Moreover, the same facial configuration (e.g. lip corners pulled backwards exposing teeth) can convey widely different information depending on the species (e.g. humans: happiness; chimpanzees: fear). The Facial Action Coding System (FACS) is considered the gold standard for investigating human facial behaviour and avoids subjective interpretations of meaning by objectively measuring independent movements linked to facial muscles, called Action Units (AUs). Following a similar methodology, we developed the CalliFACS for the common marmoset. First, we determined the facial muscular plan of the common marmoset by examining dissections from the literature. Second, we recorded common marmosets in a variety of contexts (e.g. grooming, feeding, play, human interaction, veterinary procedures), and selected clips from online databases (e.g. YouTube) to identify their facial movements. Individual facial movements were classified according to appearance changes produced by the corresponding underlying musculature. A diverse repertoire of 33 facial movements was identified in the common marmoset (15 Action Units, 15 Action Descriptors and 3 Ear Action Descriptors). Although we observed a reduced range of facial movement when compared to the HumanFACS, the common marmoset's range of facial movements was larger than predicted according to their socio-ecology and facial morphology, which indicates their importance for social interactions. CalliFACS is a scientific tool to measure facial movements, and thus, allows us to better understand the common marmoset's expressions and communication. As common marmosets have become increasingly popular laboratory animal models, from neuroscience to cognition, CalliFACS can be used as an important tool to evaluate their welfare, particularly in captivity.