Chimpanzee face recognition from videos in the wild using deep learning

Applying machine learning to primate bioacoustics: Review and perspectives

This paper provides a comprehensive review of the use of computational bioacoustics as well as signal and speech processing techniques in the analysis of primate vocal communication. We explore the potential implications of machine learning and deep learning methods, from the use of simple supervised algorithms to more recent self-supervised models, for processing and analyzing large data sets obtained within the emergence of passive acoustic monitoring approaches. In addition, we discuss the importance of automated primate vocalization analysis in tackling essential questions on animal communication and highlighting the role of comparative linguistics in bioacoustic research. We also examine the challenges associated with data collection and annotation and provide insights into potential solutions. Overall, this review paper runs through a set of common or innovative perspectives and applications of machine learning for primate vocal communication analysis and outlines opportunities for future research in this rapidly developing field.

Deciphering social traits and pathophysiological conditions from natural behaviors in common marmosets

Nonhuman primates (NHPs) are indispensable animal models by virtue of the continuity of behavioral repertoires across primates, including humans. However, behavioral assessment at the laboratory level has so far been limited. Employing the application of three-dimensional (3D) pose estimation and the optimal integration of subsequent analytic methodologies, we demonstrate that our artificial intelligence (AI)-based approach has successfully deciphered the ethological, cognitive, and pathological traits of common marmosets from their natural behaviors. By applying multiple deep neural networks trained with large-scale datasets, we established an evaluation system that could reconstruct and estimate the 3D poses of the marmosets, a small NHP that is suitable for analyzing complex natural behaviors in laboratory setups. We further developed downstream analytic methodologies to quantify a variety of behavioral parameters beyond motion kinematics. We revealed the distinct parental roles of male and female marmosets through automated detections of food-sharing behaviors using a spatial-temporal filter on 3D poses. Employing a recurrent neural network to analyze 3D pose time series data during social interactions, we additionally discovered that marmosets adjusted their behaviors based on others' internal state, which is not directly observable but can be inferred from the sequence of others' actions. Moreover, a fully unsupervised approach enabled us to detect progressively appearing symptomatic behaviors over a year in a Parkinson's disease model. The high-throughput and versatile nature of an AI-driven approach to analyze natural behaviors will open a new avenue for neuroscience research dealing with big-data analyses of social and pathophysiological behaviors in NHPs.

Deep learning for automatic facial detection and recognition in Japanese macaques: illuminating social networks

Individual identification plays a pivotal role in ecology and ethology, notably as a tool for complex social structures understanding. However, traditional identification methods often involve invasive physical tags and can prove both disruptive for animals and time-intensive for researchers. In recent years, the integration of deep learning in research has offered new methodological perspectives through the automatisation of complex tasks. Harnessing object detection and recognition technologies is increasingly used by researchers to achieve identification on video footage. This study represents a preliminary exploration into the development of a non-invasive tool for face detection and individual identification of Japanese macaques (Macaca fuscata) through deep learning. The ultimate goal of this research is, using identification done on the dataset, to automatically generate a social network representation of the studied population. The current main results are promising: (i) the creation of a Japanese macaques' face detector (Faster-RCNN model), reaching an accuracy of 82.2% and (ii) the creation of an individual recogniser for the Kōjima Island macaque population (YOLOv8n model), reaching an accuracy of 83%. We also created a Kōjima population social network by traditional methods, based on co-occurrences on videos. Thus, we provide a benchmark against which the automatically generated network will be assessed for reliability. These preliminary results are a testament to the potential of this approach to provide the scientific community with a tool for tracking individuals and social network studies in Japanese macaques.

Automatic identification of stone-handling behaviour in Japanese macaques using LabGym artificial intelligence

The latest advances in artificial intelligence technology have opened doors to the video analysis of complex behaviours. In light of this, ethologists are actively exploring the potential of these innovations to streamline the time-intensive behavioural analysis process using video data. Several tools have been developed for this purpose in primatology in the past decade. Nonetheless, each tool grapples with technical constraints. To address these limitations, we have established a comprehensive protocol designed to harness the capabilities of a cutting-edge artificial intelligence-assisted software, LabGym. The primary objective of this study was to evaluate the suitability of LabGym for the analysis of primate behaviour, focusing on Japanese macaques as our model subjects. First, we developed a model that accurately detects Japanese macaques, allowing us to analyse their actions using LabGym. Our behavioural analysis model succeeded in recognising stone-handling-like behaviours on video. However, the absence of quantitative data within the specified time frame limits the ability of our study to draw definitive conclusions regarding the quality of the behavioural analysis. Nevertheless, to the best of our knowledge, this study represents the first instance of applying the LabGym tool specifically for the analysis of primate behaviours, with our model focusing on the automated recognition and categorisation of specific behaviours in Japanese macaques. It lays the groundwork for future research in this promising field to complexify our model using the latest version of LabGym and associated tools, such as multi-class detection and interactive behaviour analysis.

Evolution of laughter from play

In this hypothesis, I discuss how laughter from physical play could have evolved to being induced via visual or even verbal stimuli, and serves as a signal to highlight incongruity that could potentially pose a threat to survival. I suggest how laughter's induction could have negated the need for physical contact in play, evolving from its use in tickling, to tickle-misses, and to taunting, and I discuss how the application of deep learning neural networks trained on images of spectra of a variety of laughter types from a variety of individuals or even species, could be used to determine such evolutionary pathways via the use of latent space exploration.

Fine-grained image classification on bats using VGG16-CBAM: a practical example with 7 horseshoe bats taxa (CHIROPTERA: Rhinolophidae: Rhinolophus) from Southern China

BACKGROUND

Rapid identification and classification of bats are critical for practical applications. However, species identification of bats is a typically detrimental and time-consuming manual task that depends on taxonomists and well-trained experts. Deep Convolutional Neural Networks (DCNNs) provide a practical approach for the extraction of the visual features and classification of objects, with potential application for bat classification.

RESULTS

In this study, we investigated the capability of deep learning models to classify 7 horseshoe bat taxa (CHIROPTERA: Rhinolophus) from Southern China. We constructed an image dataset of 879 front, oblique, and lateral targeted facial images of live individuals collected during surveys between 2012 and 2021. All images were taken using a standard photograph protocol and setting aimed at enhancing the effectiveness of the DCNNs classification. The results demonstrated that our customized VGG16-CBAM model achieved up to 92.15% classification accuracy with better performance than other mainstream models. Furthermore, the Grad-CAM visualization reveals that the model pays more attention to the taxonomic key regions in the decision-making process, and these regions are often preferred by bat taxonomists for the classification of horseshoe bats, corroborating the validity of our methods.

CONCLUSION

Our finding will inspire further research on image-based automatic classification of chiropteran species for early detection and potential application in taxonomy.

Individual recognition of Eurasian beavers (Castor fiber) by their tail patterns using a computer-assisted pattern-identification algorithm

Individual recognition of animals is an important aspect of ecological sciences. Photograph-based individual recognition options are of particular importance since these represent a non-invasive method to distinguish and identify individual animals. Recent developments and improvements in computer-based approaches make possible a faster semi-automated evaluation of large image databases than was previously possible. We tested the Scale Invariant Feature Transform (SIFT) algorithm, which extracts distinctive invariant features of images robust to illumination, rotation or scaling of images. We applied this algorithm to a dataset of 800 tail pattern images from 100 individual Eurasian beavers (Castor fiber) collected as part of the Norwegian Beaver Project (NBP). Images were taken using a single-lens reflex camera and the pattern of scales on the tail, similar to a human fingerprint, was extracted using freely accessible image processing programs. The focus for individual recognition was not on the shape or the scarring of the tail, but purely on the individual scale pattern on the upper (dorsal) surface of the tail. The images were taken from two different heights above ground, and the largest possible area of the tail was extracted. The available data set was split in a ratio of 80% for training and 20% for testing. Overall, our study achieved an accuracy of 95.7%. We show that it is possible to distinguish individual beavers from their tail scale pattern images using the SIFT algorithm.

Camera traps unveil demography, social structure, and home range of six unhabituated Western chimpanzee groups in the Moyen Bafing National Park, Guinea

Precise estimates of population dynamics and social grouping patterns are required for effective conservation of wild animal populations. It is difficult to obtain such information on non-human great apes as they have slow reproductive rates. To gain a better understanding of demography in these populations, previous research has typically involved habituation\, a process that requires years. Here, we collected data continuously over year-long periods to monitor an unhabituated population of critically endangered Western chimpanzees (Pan troglodytes verus) in the Moyen Bafing National Park, Guinea. We used two arrays of 100 camera traps that were placed opportunistically in two distinct 100 km2 sites, named Bakoun and Koukoutamba. We identified 227 individuals in Bakoun and 207 in Koukoutamba through their unique facial features. Our camera trap data make clear that these individuals belong to six and seven closed groups, respectively. Six of those groups were near-completely sampled with an average minimum size of 46.8 individuals (range: 37-58), and a mean adult sex ratio of 1.32 (range: 0.93-2.10). We described the demographic composition of these groups and use Bayesian social network analysis to understand population structure. The network analyses suggested that the social bonds within the two populations were structured by sex homophily, with male chimpanzees being more or equally likely to be observed together than other adult associations. Through estimation of minimum convex polygons, we described the minimum home range for those groups. Compared to other chimpanzee groups living in a similar environment (mosaic savanna-forest), the Moyen Bafing region seems to host a high-density of chimpanzees with small home ranges for their group size. Our research highlights the potential of camera traps for studying the demographic composition of chimpanzee populations with high resolution and obtaining crucial information on several groups in a time-efficient and cost-effective way.

GesturalOrigins: A bottom-up framework for establishing systematic gesture data across ape species

Current methodologies present significant hurdles to understanding patterns in the gestural communication of individuals, populations, and species. To address this issue, we present a bottom-up data collection framework for the study of gesture: GesturalOrigins. By "bottom-up", we mean that we minimise a priori structural choices, allowing researchers to define larger concepts (such as 'gesture types', 'response latencies', or 'gesture sequences') flexibly once coding is complete. Data can easily be re-organised to provide replication of, and comparison with, a wide range of datasets in published and planned analyses. We present packages, templates, and instructions for the complete data collection and coding process. We illustrate the flexibility that our methodological tool offers with worked examples of (great ape) gestural communication, demonstrating differences in the duration of action phases across distinct gesture action types and showing how species variation in the latency to respond to gestural requests may be revealed or masked by methodological choices. While GesturalOrigins is built from an ape-centred perspective, the basic framework can be adapted across a range of species and potentially to other communication systems. By making our gesture coding methods transparent and open access, we hope to enable a more direct comparison of findings across research groups, improve collaborations, and advance the field to tackle some of the long-standing questions in comparative gesture research.

Facial Expression of TIPI Personality and CHMP-Tri Psychopathy Traits in Chimpanzees (Pan troglodytes) : Evidence for Honest Signalling?

Honest signalling theory suggests that humans and chimpanzees can extract socially relevant information relating to personality from the faces of their conspecifics. Humans are also able to extract information from chimpanzees' faces. Here, we examine whether personality characteristics of chimpanzees, including measures of psychopathy, can be discerned based purely on facial morphology in photographs. Twenty-one chimpanzees were given naïve and expert personality ratings on the Ten Item Personality Inventory (TIPI) and the Chimpanzee Triarchic Model of Psychopathy (CHMP-Tri) before and following behavioural observations. Characteristics relating to openness, conscientiousness, extraversion, and disinhibition could be distinguished from the faces of chimpanzees. Individuals higher on disinhibition have lower scores on conscientiousness and emotional stability and higher scores on extraversion, while those higher on meanness have lower conscientiousness and agreeableness. Facial expressions are linked to personality traits present in the TIPI and CHMP-Tri models: the Relaxed Face and the Grooming Face were displayed more by chimpanzees higher on agreeableness, whereas the Compressed Lips Face was observed more in those individuals higher on boldness, and the Full Open Grin was displayed more by chimpanzees higher on extraversion but lower on emotional stability and conscientiousness. Facial expressions were also found to be associated with particular behavioural contexts, namely the Grooming Face in affiliative contexts and the Relaxed and Relaxed Open Mouth Faces in neutral contexts. Dominant chimpanzees display higher levels of boldness and more Compressed Lips Faces, Relaxed Open Mouth Faces, and Grooming Faces than subordinate individuals. These findings support and extend evidence for an honest signalling system and a personality structure shared between humans and chimpanzees. Future research could further explore how personality is conveyed through the face, perhaps through more than just singular aspects of character, and maybe reflecting what chimpanzees themselves are able to do.

Identification of hybrids between the Japanese giant salamander (Andrias japonicus) and Chinese giant salamander (Andrias cf. davidianus) using deep learning and smartphone images

Human-mediated hybridization between native and non-native species is causing biodiversity loss worldwide. Hybridization has contributed to the extinction of many species through direct and indirect processes such as loss of reproductive opportunity and genetic introgression. Therefore, it is essential to manage hybrids to conserve biodiversity. However, specialized knowledge is required to identify the target species based on visual characteristics when two species have similar features. Although image recognition technology can be a powerful tool for identifying hybrids, studies have yet to utilize deep learning approaches. Hence, this study aimed to identify hybrids between the native Japanese giant salamander (Andrias japonicus) and the non-native Chinese giant salamander (Andrias cf. davidianus) using EfficientNetV2 and smartphone images. We used smartphone images of 11 individuals of native A. japonicus (five training and six test images) and 20 individuals of hybrids between A. japonicus and A. cf. davidianus (five training and 15 test images). In our experimental environment, an AI model constructed with EfficientNetV2 exhibited 100% accuracy in identifying hybrids. In addition, gradient-weighted class activation mapping revealed that the AI model was able to classify A. japonicus and hybrids between A. japonicus and A. cf. davidianus on the basis of the dorsal head spot patterning. Our approach thus enables the identification of hybrids against A. japonicus, which was previously considered difficult by non-experts. Furthermore, since this study achieved reliable identification using smartphone images, it is expected to be applied to a wide range of citizen science projects.

Multi-Cat Monitoring System Based on Concept Drift Adaptive Machine Learning Architecture

In multi-cat households, monitoring individual cats' various behaviors is essential for diagnosing their health and ensuring their well-being. This study focuses on the defecation and urination activities of cats, and introduces an adaptive cat identification architecture based on deep learning (DL) and machine learning (ML) methods. The architecture comprises an object detector and a classification module, with the primary focus on the design of the classification component. The DL object detection algorithm, YOLOv4, is used for the cat object detector, with the convolutional neural network, EfficientNetV2, serving as the backbone for our feature extractor in identity classification with several ML classifiers. Additionally, to address changes in cat composition and individual cat appearances in multi-cat households, we propose an adaptive concept drift approach involving retraining the classification module. To support our research, we compile a comprehensive cat body dataset comprising 8934 images of 36 cats. After a rigorous evaluation of different combinations of DL models and classifiers, we find that the support vector machine (SVM) classifier yields the best performance, achieving an impressive identification accuracy of 94.53%. This outstanding result underscores the effectiveness of the system in accurately identifying cats.

Individual identification of endangered amphibians using deep learning and smartphone images: case study of the Japanese giant salamander (Andrias japonicus)

Information obtained via individual identification is invaluable for ecology and conservation. Physical tags, such as PIT tags and GPS, have been used for individual identification; however, these methods could impact on animal behavior and survival rates, and the tags may become lost. Although non-invasive methods that do not affect the target species (such as manual photoidentification) are available, these techniques utilize stripes and spots that are unique to the individual, which requires training, and applying them to large datasets is challenging. Many studies that have applied deep learning for identification have focused on species-level identification, but few have addressed individual-level identification. In this study, we developed an image-based identification method based on deep learning that uses the head spot pattern of the Japanese giant salamander (Andrias japonicus), an endemic and endangered species in Japan. We trained and evaluated a dataset collected over two days from 11 individuals in captivity, which included 7075 images taken by a smartphone camera. Individuals were photographed three times a day at approximately 11:00 (morning), 15:00 (afternoon), and 18:00 (evening). As a result, individual identification by our method, which used the EfficientNetV2 achieved 99.86% accuracy, kappa coefficient of 0.99, and an F1 score of 0.99. Performance was lower for the evening  model than for the morning and afternoon models, which were trained and evaluated using photographs taken at the corresponding time of the day. The proposed method does not require direct contact with the target species, and the effect on the animals is minimal; moreover, individual-level information can be obtained under natural conditions. In the future, smartphone images can be applied to citizen science surveys and individual-level big data collection, which is difficult using current methods.

Editorial: Recent Advances in Animal Cognition and Ethology

Animal cognition and ethology, the scientific study of animal behaviour, have long captivated the human imagination [...].

Classification of multiple emotional states from facial expressions in head-fixed mice using a deep learning-based image analysis

Facial expressions are widely recognized as universal indicators of underlying internal states in most species of animals, thereby presenting as a non-invasive measure for assessing physical and mental conditions. Despite the advancement of artificial intelligence-assisted tools for automated analysis of voluminous facial expression data in human subjects, the corresponding tools for mice still remain limited so far. Considering that mice are the most prevalent model animals for studying human health and diseases, a comprehensive characterization of emotion-dependent patterns of facial expressions in mice could extend our knowledge on the basis of emotions and the related disorders. Here, we present a framework for the development of a deep learning-powered tool for classifying facial expressions in head-fixed mouse. We demonstrate that our machine vision was capable of accurately classifying three different emotional states from lateral facial images in head-fixed mouse. Moreover, we objectively determined how our classifier characterized the differences among the facial images through the use of an interpretation technique called Gradient-weighted Class Activation Mapping. Importantly, our machine vision presumably discerned the data by leveraging multiple facial features. Our approach is likely to facilitate the non-invasive decoding of a variety of emotions from facial images in head-fixed mice.

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions. Traditional tracking methods (e.g., marking each animal with dye or surgically implanting microchips) can be invasive and may have an impact on the social behavior being measured. To overcome these shortcomings, video-based methods for tracking unmarked animals, such as fruit flies and zebrafish, have been developed. However, tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns. In this study, we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network (R-CNN) deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology. The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior. The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.

Emerging technologies for behavioral research in changing environments

The first response exhibited by animals to changing environments is typically behavioral. Behavior is thus central to predicting, and mitigating, the impacts that natural and anthropogenic environmental changes will have on populations and, consequently, ecosystems. Yet the inherently multiscale nature of behavior, as well as the complexities associated with inferring how animals perceive their world, and make decisions, has constrained the scope of behavioral research. Major technological advances in electronics and in machine learning, however, provide increasingly powerful means to see, analyze, and interpret behavior in its natural complexity. We argue that these disruptive technologies will foster new approaches that will allow us to move beyond quantitative descriptions and reveal the underlying generative processes that give rise to behavior.

Information Theory Opens New Dimensions in Experimental Studies of Animal Behaviour and Communication

Over the last 40–50 years, ethology has become increasingly quantitative and computational. However, when analysing animal behavioural sequences, researchers often need help finding an adequate model to assess certain characteristics of these sequences while using a relatively small number of parameters. In this review, I demonstrate that the information theory approaches based on Shannon entropy and Kolmogorov complexity can furnish effective tools to analyse and compare animal natural behaviours. In addition to a comparative analysis of stereotypic behavioural sequences, information theory can provide ideas for particular experiments on sophisticated animal communications. In particular, it has made it possible to discover the existence of a developed symbolic “language” in leader-scouting ant species based on the ability of these ants to transfer abstract information about remote events.

Disgust in animals and the application of disease avoidance to wildlife management and conservation

Disgust is an adaptive system hypothesized to have evolved to reduce the risk of becoming sick. It is associated with behavioural, cognitive and physiological responses tuned to allow animals to avoid and/or get rid of parasites, pathogens and toxins. Little is known about the mechanisms and outcomes of disease avoidance in wild animals. Furthermore, given the escalation of negative human-wildlife interactions, the translation of such knowledge into the design of evolutionarily relevant conservation and wildlife management strategies is becoming urgent. Contemporary methods in animal ecology and related fields, using direct (sensory cues) or indirect (remote sensing technologies and machine learning) means, provide a flexible toolbox for testing and applying disgust at individual and collective levels. In this review/perspective paper, we provide an empirical framework for testing the adaptive function of disgust and its associated disease avoidance behaviours across species, from the least to the most social, in different habitats. We predict various trade-offs to be at play depending on the social system and ecology of the species. We propose five contexts in which disgust-related avoidance behaviours could be applied, including endangered species rehabilitation, invasive species, crop-raiding, urban pests and animal tourism. We highlight some of the perspectives and current challenges of testing disgust in the wild. In particular, we recommend future studies to consider together disease, predation and competition risks. We discuss the ethics associated with disgust experiments in the above contexts. Finally, we promote the creation of a database gathering disease avoidance evidence in animals and its applications.

PolarBearVidID: A Video-Based Re-Identification Benchmark Dataset for Polar Bears

Automated monitoring systems have become increasingly important for zoological institutions in the study of their animals' behavior. One crucial processing step for such a system is the re-identification of individuals when using multiple cameras. Deep learning approaches have become the standard methodology for this task. Especially video-based methods promise to achieve a good performance in re-identification, as they can leverage the movement of an animal as an additional feature. This is especially important for applications in zoos, where one has to overcome specific challenges such as changing lighting conditions, occlusions or low image resolutions. However, large amounts of labeled data are needed to train such a deep learning model. We provide an extensively annotated dataset including 13 individual polar bears shown in 1431 sequences, which is an equivalent of 138,363 images. PolarBearVidID is the first video-based re-identification dataset for a non-human species to date. Unlike typical human benchmark re-identification datasets, the polar bears were filmed in a range of unconstrained poses and lighting conditions. Additionally, a video-based re-identification approach is trained and tested on this dataset. The results show that the animals can be identified with a rank-1 accuracy of 96.6%. We thereby show that the movement of individual animals is a characteristic feature and it can be utilized for re-identification.

Animal linguistics: a primer

The evolution of language has been investigated by several research communities, including biologists and linguists, striving to highlight similar linguistic capacities across species. To date, however, no consensus exists on the linguistic capacities of non-human species. Major controversies remain on the use of linguistic terminology, analysis methods and behavioural data collection. The field of 'animal linguistics' has emerged to overcome these difficulties and attempt to reach uniform methods and terminology. This primer is a tutorial review of 'animal linguistics'. It describes the linguistic concepts of semantics, pragmatics and syntax, and proposes minimal criteria to be fulfilled to claim that a given species displays a particular linguistic capacity. Second, it reviews relevant methods successfully applied to the study of communication in animals and proposes a list of useful references to detect and overcome major pitfalls commonly observed in the collection of animal behaviour data. This primer represents a step towards mutual understanding and fruitful collaborations between linguists and biologists.

Dynamic Curriculum Learning for Great Ape Detection in the Wild

We propose a novel end-to-end curriculum learning approach for sparsely labelled animal datasets leveraging large volumes of unlabelled data to improve supervised species detectors. We exemplify the method in detail on the task of finding great apes in camera trap footage taken in challenging real-world jungle environments. In contrast to previous semi-supervised methods, our approach adjusts learning parameters dynamically over time and gradually improves detection quality by steering training towards virtuous self-reinforcement. To achieve this, we propose integrating pseudo-labelling with curriculum learning policies and show how learning collapse can be avoided. We discuss theoretical arguments, ablations, and significant performance improvements against various state-of-the-art systems when evaluating on the Extended PanAfrican Dataset holding approx. 1.8M frames. We also demonstrate our method can outperform supervised baselines with significant margins on sparse label versions of other animal datasets such as Bees and Snapshot Serengeti. We note that performance advantages are strongest for smaller labelled ratios common in ecological applications. Finally, we show that our approach achieves competitive benchmarks for generic object detection in MS-COCO and PASCAL-VOC indicating wider applicability of the dynamic learning concepts introduced. We publish all relevant source code, network weights, and data access details for full reproducibility.

Individual automatic detection and identification of big cats with the combination of different body parts

The development of facial recognition technology has become an increasingly powerful tool in wild animal individual recognition. In this paper, we develop an automatic detection and recognition method with the combinations of body features of big cats based on the deep convolutional neural network (CNN). We collected dataset including 12 244 images from 47 individual Amur tigers (Panthera tigris altaica) at the Siberian Tiger Park by mobile phones and digital camera and 1940 images and videos of 12 individual wild Amur leopard (Panthera pardus orientalis) by infrared cameras. First, the single shot multibox detector algorithm is used to perform the automatic detection process of feature regions in each image. For the different feature regions of the image, like face stripe or spots, CNNs and multi-layer perceptron models were applied to automatically identify tiger and leopard individuals, independently. Our results show that the identification accuracy of Amur tiger can reach up to 93.27% for face front, 93.33% for right body stripe, and 93.46% for left body stripe. Furthermore, the combination of right face, left body stripe, and right body stripe achieves the highest accuracy rate, up to 95.55%. Consequently, the combination of different body parts can improve the individual identification accuracy. However, it is not the higher the number of body parts, the higher the accuracy rate. The combination model with 3 body parts has the highest accuracy. The identification accuracy of Amur leopard can reach up to 86.90% for face front, 89.13% for left body spots, and 88.33% for right body spots. The accuracy of different body parts combination is lower than the independent part. For wild Amur leopard, the combination of face with body spot part is not helpful for the improvement of identification accuracy. The most effective identification part is still the independent left or right body spot part. It can be applied in long-term monitoring of big cats, including big data analysis for animal behavior, and be helpful for the individual identification of other wildlife species.

Wild Terrestrial Animal Re-Identification Based on an Improved Locally Aware Transformer with a Cross-Attention Mechanism

The wildlife re-identification recognition methods based on the camera trap were used to identify different individuals of the same species using the fur, stripes, facial features and other features of the animal body surfaces in the images, which is an important way to count the individual number of a species. Re-identification of wild animals can provide solid technical support for the in-depth study of the number of individuals and living conditions of rare wild animals, as well as provide accurate and timely data support for population ecology and conservation biology research. However, due to the difficulty of recording the shy wild animals and distinguishing the similar fur of different individuals, only a few papers have focused on the re-identification recognition of wild animals. In order to fill this gap, we improved the locally aware transformer (LA transformer) network structure for the re-identification recognition of wild terrestrial animals. First of all, at the stage of feature extraction, we replaced the self-attention module of the LA transformer with a cross-attention block (CAB) in order to calculate the inner-patch attention and cross-patch attention, so that we could efficiently capture the global information of the animal body's surface and local feature differences of fur, colors, textures, or faces. Then, the locally aware network of the LA transformer was used to fuse the local and global features. Finally, the classification layer of the network realized wildlife individual recognition. In order to evaluate the performance of the model, we tested it on a dataset of Amur tiger torsos and the face datasets of six different species, including lions, golden monkeys, meerkats, red pandas, tigers, and chimpanzees. The experimental results showed that our wildlife re-identification model has good generalization ability and is superior to the existing methods in mAP (mean average precision), and obtained comparable results in the metrics Rank 1 and Rank 5.

Group-based, autonomous, individualized training and testing of long-tailed macaques (Macaca fascicularis) in their home enclosure to a visuo-acoustic discrimination task

In recent years, the utility and efficiency of automated procedures for cognitive assessment in psychology and neuroscience have been demonstrated in non-human primates (NHP). This approach mimics conventional shaping principles of breaking down a final desired behavior into smaller components that can be trained in a staircase manner. When combined with home-cage-based approaches, this could lead to a reduction in human workload, enhancement in data quality, and improvement in animal welfare. However, to our knowledge, there are no reported attempts to develop automated training and testing protocols for long-tailed macaques (Macaca fascicularis), a ubiquitous NHP model in neuroscience and pharmaceutical research. In the current work, we present the results from 6 long-tailed macaques that were trained using an automated unsupervised training (AUT) protocol for introducing the animals to the basics of a two-alternative choice (2 AC) task where they had to discriminate a conspecific vocalization from a pure tone relying on images presented on a touchscreen to report their response. We found that animals (1) consistently engaged with the device across several months; (2) interacted in bouts of high engagement; (3) alternated peacefully to interact with the device; and (4) smoothly ascended from step to step in the visually guided section of the procedure, in line with previous results from other NHPs. However, we also found (5) that animals' performance remained at chance level as soon as the acoustically guided steps were reached; and (6) that the engagement level decreased significantly with decreasing performance during the transition from visual to acoustic-guided sections. We conclude that with an autonomous approach, it is possible to train long-tailed macaques in their social group using computer vision techniques and without dietary restriction to solve a visually guided discrimination task but not an acoustically guided task. We provide suggestions on what future attempts could take into consideration to instruct acoustically guided discrimination tasks successfully.

Yet Another Non-Unique Human Behaviour: Leave-Taking in Wild Chacma Baboons (Papio ursinus)

Simple Summary

Although greeting is well-studied across animal species, its counterpart, leave taking, is little studied in nonhumans. Here, we review the previous limitations of leave-taking research and use this to develop a new method for studying leave taking in nonhumans. Using videos of chacma baboons in Gorongosa National Park, Mozambique, we compared behaviours at the end of social departures to nonsocial departures. We found that shifting orientation towards the direction of parting was significantly more likely in social departures compared to nonsocial departures. As the first evidence of leave taking in a wild nonhuman species, we suggest that leave taking is not uniquely human as previously argued, and that our method could be used to further explore the presence of leave taking in other nonhuman species.

Abstract

Leave taking is a common, possibly universal, feature of human social behaviour that has undergone very little empirical research. Although the importance remains unknown, it has been suggested to play an important role in managing separations, mitigating the risk, and increasing social bonding beyond the interaction itself. In nonhuman species, the literature is virtually absent, but identifying leave taking beyond humans may provide unique insights into the evolutionary history of this behaviour and shed light onto its proximate and ultimate function(s). Methods to study leave taking are not well-established, and the variation in definitions, measures, and control variables presented in past studies poses additional challenges. Baboons are a valuable model for investigating human behavioural evolution: as a flexible, highly adaptable, and social primate whose radiation is, similarly to humans, associated with the emergence of the African savannah biome. Using the framework and definition proposed by Baehren, we investigated the presence of leave taking in a wild, generalist primate and tested a range of candidate behaviours on prerecorded video footage: (1) self-scratching, (2) eye gaze, and (3) orientation in the direction of parting. Using multivariate analysis, controlling for interaction duration and individual variation, our results show that orientation in the direction of parting occurs predominantly before social separation events. These results indicate evidence of leave taking in a wild nonhuman population and contrast with previous ideas that this is a uniquely human behaviour. The presence of leave taking in baboons suggests a deep evolutionary history of this behaviour, warranting further investigation into its function and presence across other nonhuman primate species.

The Future of Artificial Intelligence in Monitoring Animal Identification, Health, and Behaviour

Simple Summary

Due to climate change and human interference, many species are now without habitats and on the brink of extinction. Zoos and other conservation spaces allow for non-human animal preservation and public education about endangered species and ecosystems. Monitoring the health and well-being of animals in care, while providing species-specific environments, is critical for zoo and conservation staff. In order to best provide such care, keepers and researchers need to gather as much information as possible about individual animals and species as a whole. This paper focuses on existing technology to monitor animals, providing a review on the history of technology, including recent technological advancements and current limitations. Subsequently, we provide a brief introduction to our proposed novel computer software: an artificial intelligence software capable of unobtrusively and non-invasively tracking individuals’ location, estimating position, and analyzing behaviour. This innovative technology is currently being trained with orangutans at the Toronto Zoo and will allow for mass data collection, permitting keepers and researchers to closely monitor individual animal welfare, learn about the variables impacting behaviour and provide additional enrichment or interventions accordingly.

Abstract

With many advancements, technologies are now capable of recording non-human animals’ location, heart rate, and movement, often using a device that is physically attached to the monitored animals. However, to our knowledge, there is currently no technology that is able to do this unobtrusively and non-invasively. Here, we review the history of technology for use with animals, recent technological advancements, current limitations, and a brief introduction to our proposed novel software. Canadian tech mogul EAIGLE Inc. has developed an artificial intelligence (AI) software solution capable of determining where people and assets are within public places or attractions for operational intelligence, security, and health and safety applications. The solution also monitors individual temperatures to reduce the potential spread of COVID-19. This technology has been adapted for use at the Toronto Zoo, initiated with a focus on Sumatran orangutans (Pongo abelii) given the close physical similarity between orangutans and humans as great ape species. This technology will be capable of mass data collection, individual identification, pose estimation, behaviour monitoring and tracking orangutans’ locations, in real time on a 24/7 basis, benefitting both zookeepers and researchers looking to review this information.

Using Pruning-Based YOLO v3 Deep Learning Algorithm for Accurate Detection of Sheep Face

Simple Summary

The identification of individual animals is an important step in the history of precision breeding. It has a great role in both breeding and genetic management. The continuous development of computer vision and deep learning technologies provides new possibilities for the establishment of accurate breeding models. This helps to achieve high productivity and precise management in precision agriculture. Here, we demonstrate that sheep faces can be recognized based on the YOLOv3 target detection network. A model compression method based on K-means clustering algorithm and combined with channel pruning and layer pruning is applied to individual sheep identification. In addition, the results show that the proposed non-contact sheep face recognition method can identify sheep quickly and accurately.

Abstract

Accurate identification of sheep is important for achieving precise animal management and welfare farming in large farms. In this study, a sheep face detection method based on YOLOv3 model pruning is proposed, abbreviated as YOLOv3-P in the text. The method is used to identify sheep in pastures, reduce stress and achieve welfare farming. Specifically, in this study, we chose to collect Sunit sheep face images from a certain pasture in Xilin Gol League Sunit Right Banner, Inner Mongolia, and used YOLOv3, YOLOv4, Faster R-CNN, SSD and other classical target recognition algorithms to train and compare the recognition results, respectively. Ultimately, the choice was made to optimize YOLOv3. The mAP was increased from 95.3% to 96.4% by clustering the anchor frames in YOLOv3 using the sheep face dataset. The mAP of the compressed model was also increased from 96.4% to 97.2%. The model size was also reduced to 1/4 times the size of the original model. In addition, we restructured the original dataset and performed a 10-fold cross-validation experiment with a value of 96.84% for mAP. The results show that clustering the anchor boxes and compressing the model using this dataset is an effective method for identifying sheep. The method is characterized by low memory requirement, high-recognition accuracy and fast recognition speed, which can accurately identify sheep and has important applications in precision animal management and welfare farming.

LAEO-Net++: Revisiting People Looking at Each Other in Videos

Capturing the 'mutual gaze' of people is essential for understanding and interpreting the social interactions between them. To this end, this paper addresses the problem of detecting people Looking At Each Other (LAEO) in video sequences. For this purpose, we propose LAEO-Net++, a new deep CNN for determining LAEO in videos. In contrast to previous works, LAEO-Net++ takes spatio-temporal tracks as input and reasons about the whole track. It consists of three branches, one for each character's tracked head and one for their relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and AVA-LAEO. A thorough experimental evaluation demonstrates the ability of LAEO-Net++ to successfully determine if two people are LAEO and the temporal window where it happens. Our model achieves state-of-the-art results on the existing TVHID-LAEO video dataset, significantly outperforming previous approaches. Finally, we apply LAEO-Net++ to a social network, where we automatically infer the social relationship between pairs of people based on the frequency and duration that they LAEO, and show that LAEO can be a useful tool for guided search of human interactions in videos.

Evaluating Cognitive Enrichment for Zoo-Housed Gorillas Using Facial Recognition

The use of computer technology within zoos is becoming increasingly popular to help achieve high animal welfare standards. However, despite its various positive applications to wildlife in recent years, there has been little uptake of machine learning in zoo animal care. In this paper, we describe how a facial recognition system, developed using machine learning, was embedded within a cognitive enrichment device (a vertical, modular finger maze) for a troop of seven Western lowland gorillas (Gorilla gorilla gorilla) at Bristol Zoo Gardens, UK. We explored whether machine learning could automatically identify individual gorillas through facial recognition, and automate the collection of device-use data including the order, frequency and duration of use by the troop. Concurrent traditional video recording and behavioral coding by eye was undertaken for comparison. The facial recognition system was very effective at identifying individual gorillas (97% mean average precision) and could automate specific downstream tasks (for example, duration of engagement). However, its development was a heavy investment, requiring specialized hardware and interdisciplinary expertise. Therefore, we suggest a system like this is only appropriate for long-term projects. Additionally, researcher input was still required to visually identify which maze modules were being used by gorillas and how. This highlights the need for additional technology, such as infrared sensors, to fully automate cognitive enrichment evaluation. To end, we describe a future system that combines machine learning and sensor technology which could automate the collection of data in real-time for use by researchers and animal care staff.

SEALNET: Facial recognition software for ecological studies of harbor seals

Methods for long‐term monitoring of coastal species such as harbor seals (Phoca vitulina) are often costly, time‐consuming, and highly invasive, underscoring the need for improved techniques for data collection and analysis. Here, we propose the use of automated facial recognition technology for identification of individual seals and demonstrate its utility in ecological and population studies. We created a software package, SealNet, that automates photo identification of seals, using a graphical user interface (GUI) software to detect, align, and chip seal faces from photographs and a deep convolutional neural network (CNN) suitable for small datasets (e.g., 100 seals with five photos per seal) to classify individual seals. We piloted the SealNet technology with a population of harbor seals located within Casco Bay on the coast of Maine, USA. Across two years of sampling, 2019 and 2020, at seven haul‐out sites in Middle Bay, we obtained a dataset optimized for the development and testing of SealNet. We processed 1752 images representing 408 individual seals and achieved 88% Rank‐1 and 96% Rank‐5 accuracy in closed set seal identification. In identifying individual seals, SealNet software outperformed a similar face recognition method, PrimNet, developed for primates but retrained on seals. The ease and wealth of image data that can be processed using SealNet software contributes a vital tool for ecological and behavioral studies of marine mammals in the developing field of conservation technology.

An Adaptive Embedding Network with Spatial Constraints for the Use of Few-Shot Learning in Endangered-Animal Detection

Image recording is now ubiquitous in the fields of endangered-animal conservation and GIS. However, endangered animals are rarely seen, and, thus, only a few samples of images of them are available. In particular, the study of endangered-animal detection has a vital spatial component. We propose an adaptive, few-shot learning approach to endangered-animal detection through data augmentation by applying constraints on the mixture of foreground and background images based on species distributions. First, the pre-trained, salient network U2-Net segments the foregrounds and backgrounds of images of endangered animals. Then, the pre-trained image completion network CR-Fill is used to repair the incomplete environment. Furthermore, our approach identifies a foreground–background mixture of different images to produce multiple new image examples, using the relation network to permit a more realistic mixture of foreground and background images. It does not require further supervision, and it is easy to embed into existing networks, which learn to compensate for the uncertainties and nonstationarities of few-shot learning. Our experimental results are in excellent agreement with theoretical predictions by different evaluation metrics, and they unveil the future potential of video surveillance to address endangered-animal detection in studies of their behavior and conservation.

Multi-animal pose estimation, identification and tracking with DeepLabCut

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios. To take up this challenge, we build on DeepLabCut, an open-source pose estimation toolbox, and provide high-performance animal assembly and tracking-features required for multi-animal scenarios. Furthermore, we integrate the ability to predict an animal's identity to assist tracking (in case of occlusions). We illustrate the power of this framework with four datasets varying in complexity, which we release to serve as a benchmark for future algorithm development.