A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping

Unsupervised decomposition of natural monkey behavior into a sequence of motion motifs

Nonhuman primates (NHPs) exhibit complex and diverse behavior that typifies advanced cognitive function and social communication, but quantitative and systematical measure of this natural nonverbal processing has been a technical challenge. Specifically, a method is required to automatically segment time series of behavior into elemental motion motifs, much like finding meaningful words in character strings. Here, we propose a solution called SyntacticMotionParser (SMP), a general-purpose unsupervised behavior parsing algorithm using a nonparametric Bayesian model. Using three-dimensional posture-tracking data from NHPs, SMP automatically outputs an optimized sequence of latent motion motifs classified into the most likely number of states. When applied to behavioral datasets from common marmosets and rhesus monkeys, SMP outperformed conventional posture-clustering models and detected a set of behavioral ethograms from publicly available data. SMP also quantified and visualized the behavioral effects of chemogenetic neural manipulations. SMP thus has the potential to dramatically improve our understanding of natural NHP behavior in a variety of contexts.

Mapping the Behavioral Signatures of Shank3b Mice in Both Sexes

Autism spectrum disorders (ASD) are characterized by social and repetitive abnormalities. Although the ASD mouse model with Shank3b mutations is widely used in ASD research, the behavioral phenotype of this model has not been fully elucidated. Here, a 3D-motion capture system and linear discriminant analysis were used to comprehensively record and analyze the behavioral patterns of male and female Shank3b mutant mice. It was found that both sexes replicated the core and accompanied symptoms of ASD, with significant sex differences. Further, Shank3b heterozygous knockout mice exhibited distinct autistic behaviors, that were significantly different from those those observed in the wild type and homozygous knockout groups. Our findings provide evidence for the inclusion of both sexes and experimental approaches to efficiently characterize heterozygous transgenic models, which are more clinically relevant in autistic studies.

Study of tree shrew biology and models: A booming and prosperous field for biomedical research

The tree shrew ( Tupaia belangeri) has long been proposed as a suitable alternative to non-human primates (NHPs) in biomedical and laboratory research due to its close evolutionary relationship with primates. In recent years, significant advances have facilitated tree shrew studies, including the determination of the tree shrew genome, genetic manipulation using spermatogonial stem cells, viral vector-mediated gene delivery, and mapping of the tree shrew brain atlas. However, the limited availability of tree shrews globally remains a substantial challenge in the field. Additionally, determining the key questions best answered using tree shrews constitutes another difficulty. Tree shrew models have historically been used to study hepatitis B virus (HBV) and hepatitis C virus (HCV) infection, myopia, and psychosocial stress-induced depression, with more recent studies focusing on developing animal models for infectious and neurodegenerative diseases. Despite these efforts, the impact of tree shrew models has not yet matched that of rodent or NHP models in biomedical research. This review summarizes the prominent advancements in tree shrew research and reflects on the key biological questions addressed using this model. We emphasize that intensive dedication and robust international collaboration are essential for achieving breakthroughs in tree shrew studies. The use of tree shrews as a unique resource is expected to gain considerable attention with the application of advanced techniques and the development of viable animal models, meeting the increasing demands of life science and biomedical research.

Microglial sex differences in innate high anxiety and modulatory effects of minocycline

Microglia modulate synaptic refinement in the central nervous system (CNS). We have previously shown that a mouse model with innate high anxiety-related behavior (HAB) displays higher CD68+ microglia density in the key regions of anxiety circuits compared to mice with normal anxiety-related behavior (NAB) in males, and that minocycline treatment attenuated the enhanced anxiety of HAB male. Given that a higher prevalence of anxiety is widely reported in females compared to males, little is known concerning sex differences at the cellular level. Herein, we address this by analyzing microglia heterogeneity and function in the HAB and NAB brains of both sexes. Single-cell RNA sequencing revealed ten distinct microglia clusters varied by their frequency and gene expression profile. We report striking sex differences, especially in the major microglia clusters of HABs, indicating a higher expression of genes associated with phagocytosis and synaptic engulfment in the female compared to the male. On a functional level, we show that female HAB microglia engulfed a greater amount of hippocampal vGLUT1+ excitatory synapses compared to the male. We moreover show that female HAB microglia engulfed more synaptosomes compared to the male HAB in vitro. Due to previously reported effects of minocycline on microglia, we finally administered oral minocycline to HABs of both sexes and showed a significant reduction in the engulfment of synapses by female HAB microglia. In parallel to our microglia-specific findings, we further showed an anxiolytic effect of minocycline on female HABs, which is complementary to our previous findings in the male HABs. Our study, therefore, identifies the altered function of synaptic engulfment by microglia as a potential avenue to target and resolve microglia heterogeneity in mice with innate high anxiety.

Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience

The study of complex behaviors is often challenging when using manual annotation due to the absence of quantifiable behavioral definitions and the subjective nature of behavioral annotation. Integration of supervised machine learning approaches mitigates some of these issues through the inclusion of accessible and explainable model interpretation. To decrease barriers to access, and with an emphasis on accessible model explainability, we developed the open-source Simple Behavioral Analysis (SimBA) platform for behavioral neuroscientists. SimBA introduces several machine learning interpretability tools, including SHapley Additive exPlanation (SHAP) scores, that aid in creating explainable and transparent behavioral classifiers. Here we show how the addition of explainability metrics allows for quantifiable comparisons of aggressive social behavior across research groups and species, reconceptualizing behavior as a sharable reagent and providing an open-source framework. We provide an open-source, graphical user interface (GUI)-driven, well-documented package to facilitate the movement toward improved automation and sharing of behavioral classification tools across laboratories.

ARBUR, a machine learning-based analysis system for relating behaviors and ultrasonic vocalizations of rats

Deciphering how different behaviors and ultrasonic vocalizations (USVs) of rats interact can yield insights into the neural basis of social interaction. However, the behavior-vocalization interplay of rats remains elusive because of the challenges of relating the two communication media in complex social contexts. Here, we propose a machine learning-based analysis system (ARBUR) that can cluster without bias both non-step (continuous) and step USVs, hierarchically detect eight types of behavior of two freely behaving rats with high accuracy, and locate the vocal rat in 3-D space. ARBUR reveals that rats communicate via distinct USVs during different behaviors. Moreover, we show that ARBUR can indicate findings that are long neglected by former manual analysis, especially regarding the non-continuous USVs during easy-to-confuse social behaviors. This work could help mechanistically understand the behavior-vocalization interplay of rats and highlights the potential of machine learning algorithms in automatic animal behavioral and acoustic analysis.

Three-dimensional surface motion capture of multiple freely moving pigs using MAMMAL

Understandings of the three-dimensional social behaviors of freely moving large-size mammals are valuable for both agriculture and life science, yet challenging due to occlusions in close interactions. Although existing animal pose estimation methods captured keypoint trajectories, they ignored deformable surfaces which contained geometric information essential for social interaction prediction and for dealing with the occlusions. In this study, we develop a Multi-Animal Mesh Model Alignment (MAMMAL) system based on an articulated surface mesh model. Our self-designed MAMMAL algorithms automatically enable us to align multi-view images into our mesh model and to capture 3D surface motions of multiple animals, which display better performance upon severe occlusions compared to traditional triangulation and allow complex social analysis. By utilizing MAMMAL, we are able to quantitatively analyze the locomotion, postures, animal-scene interactions, social interactions, as well as detailed tail motions of pigs. Furthermore, experiments on mouse and Beagle dogs demonstrate the generalizability of MAMMAL across different environments and mammal species.

Modeling SHANK3-associated autism spectrum disorder in Beagle dogs via CRISPR/Cas9 gene editing

Despite intensive studies in modeling neuropsychiatric disorders especially autism spectrum disorder (ASD) in animals, many challenges remain. Genetic mutant mice have contributed substantially to the current understanding of the molecular and neural circuit mechanisms underlying ASD. However, the translational value of ASD mouse models in preclinical studies is limited to certain aspects of the disease due to the apparent differences in brain and behavior between rodents and humans. Non-human primates have been used to model ASD in recent years. However, a low reproduction rate due to a long reproductive cycle and a single birth per pregnancy, and an extremely high cost prohibit a wide use of them in preclinical studies. Canine model is an appealing alternative because of its complex and effective dog-human social interactions. In contrast to non-human primates, dog has comparable drug metabolism as humans and a high reproduction rate. In this study, we aimed to model ASD in experimental dogs by manipulating the Shank3 gene as SHANK3 mutations are one of most replicated genetic defects identified from ASD patients. Using CRISPR/Cas9 gene editing, we successfully generated and characterized multiple lines of Beagle Shank3 (bShank3) mutants that have been propagated for a few generations. We developed and validated a battery of behavioral assays that can be used in controlled experimental setting for mutant dogs. bShank3 mutants exhibited distinct and robust social behavior deficits including social withdrawal and reduced social interactions with humans, and heightened anxiety in different experimental settings (n = 27 for wild-type controls and n = 44 for mutants). We demonstrate the feasibility of producing a large number of mutant animals in a reasonable time frame. The robust and unique behavioral findings support the validity and value of a canine model to investigate the pathophysiology and develop treatments for ASD and potentially other psychiatric disorders.

REVEALS: An Open Source Multi Camera GUI For Rodent Behavior Acquisition

Understanding the rich behavioral data generated by mice is essential for deciphering the function of the healthy and diseased brain. However, the current landscape lacks effective, affordable, and accessible methods for acquiring such data, especially when employing multiple cameras simultaneously. We have developed REVEALS (Rodent BEhaVior Multi-camErA Laboratory AcquiSition), a graphical user interface (GUI) written in python for acquiring rodent behavioral data via commonly used USB3 cameras. REVEALS allows for user-friendly control of recording from one or multiple cameras simultaneously while streamlining the data acquisition process, enabling researchers to collect and analyze large datasets efficiently. We release this software package as a stand-alone, open-source framework for researchers to use and modify according to their needs. We describe the details of the GUI implementation, including the camera control software and the video recording functionality. We validate results demonstrating the GUI's stability, reliability, and accuracy for capturing and analyzing rodent behavior using DeepLabCut pose estimation in both an object and social interaction assay. REVEALS can also be incorporated into other custom pipelines to analyze complex behavior, such as MoSeq. In summary, REVEALS provides an interface for collecting behavioral data from one or multiple perspectives that, combined with deep learning algorithms, will allow the scientific community to discover and characterize complex behavioral phenotypes to understand brain function better.

Automated phenotyping of postoperative delirium-like behaviour in mice reveals the therapeutic efficacy of dexmedetomidine

Postoperative delirium (POD) is a complicated and harmful clinical syndrome. Traditional behaviour analysis mostly focuses on static parameters. However, animal behaviour is a bottom-up and hierarchical organizational structure composed of time-varying posture dynamics. Spontaneous and task-driven behaviours are used to conduct comprehensive profiling of behavioural data of various aspects of model animals. A machine-learning based method is used to assess the effect of dexmedetomidine. Fourteen statistically different spontaneous behaviours are used to distinguish the non-POD group from the POD group. In the task-driven behaviour, the non-POD group has greater deep versus shallow investigation preference, with no significant preference in the POD group. Hyperactive and hypoactive subtypes can be distinguished through pose evaluation. Dexmedetomidine at a dose of 25 μg kg-1 reduces the severity and incidence of POD. Here we propose a multi-scaled clustering analysis framework that includes pose, behaviour and action sequence evaluation. This may represent the hierarchical dynamics of delirium-like behaviours.

PiE: an open-source pipeline for home cage behavioral analysis

Over the last two decades a growing number of neuroscience labs are conducting behavioral assays in rodents. The equipment used to collect this behavioral data must effectively limit environmental and experimenter disruptions, to avoid confounding behavior data. Proprietary behavior boxes are expensive, offer limited compatible sensors, and constrain analysis with closed-source hardware and software. Here, we introduce PiE, an open-source, end-to-end, user-configurable, scalable, and inexpensive behavior assay system. The PiE system includes the custom-built behavior box to hold a home cage, as well as software enabling continuous video recording and individual behavior box environmental control. To limit experimental disruptions, the PiE system allows the control and monitoring of all aspects of a behavioral experiment using a remote web browser, including real-time video feeds. To allow experiments to scale up, the PiE system provides a web interface where any number of boxes can be controlled, and video data easily synchronized to a remote location. For the scoring of behavior video data, the PiE system includes a standalone desktop application that streamlines the blinded manual scoring of large datasets with a focus on quality control and assay flexibility. The PiE system is ideal for all types of behavior assays in which video is recorded. Users are free to use individual components of this setup independently, or to use the entire pipeline from data collection to analysis. Alpha testers have included scientists without prior coding experience. An example pipeline is demonstrated with the PiE system enabling the user to record home cage maternal behavior assays, synchronize the resulting data, conduct blinded scoring, and import the data into R for data visualization and analysis.

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions. Traditional tracking methods (e.g., marking each animal with dye or surgically implanting microchips) can be invasive and may have an impact on the social behavior being measured. To overcome these shortcomings, video-based methods for tracking unmarked animals, such as fruit flies and zebrafish, have been developed. However, tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns. In this study, we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network (R-CNN) deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology. The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior. The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.

Systematic evaluation of a predator stress model of depression in mice using a hierarchical 3D-motion learning framework

Investigation of the neurobiology of depression in humans depends on animal models that attempt to mimic specific features of the human disorder. However, frequently-used paradigms based on social stress cannot be easily applied to female mice which has led to a large sex bias in preclinical studies of depression. Furthermore, most studies focus on one or only a few behavioral assessments, with time and practical considerations prohibiting a comprehensive evaluation. In this study, we demonstrate that predator stress effectively induced depression-like behaviors in both male and female mice. By comparing predator stress and social defeat models, we observed that the former elicited a higher level of behavioral despair and the latter elicited more robust social avoidance. Furthermore, the use of machine learning (ML)-based spontaneous behavioral classification can distinguish mice subjected to one type of stress from another, and from non-stressed mice. We show that related patterns of spontaneous behaviors correspond to depression status as measured by canonical depression-like behaviors, which illustrates that depression-like symptoms can be predicted by ML-classified behavior patterns. Overall, our study confirms that the predator stress induced phenotype in mice is a good reflection of several important aspects of depression in humans and illustrates that ML-supported analysis can simultaneously evaluate multiple behavioral alterations in different animal models of depression, providing a more unbiased and holistic approach for the study of neuropsychiatric disorders.

Intracerebral Hemorrhage Models and Behavioral Tests in Rodents

Intracerebral hemorrhage (ICH) is one of the common types of stroke, which can cause neurological dysfunction. In preclinical ICH studies, researchers often established rodent models by donor/autologous whole blood or a collagenase injection. White matter injury (WMI) can result from primary and secondary injuries after ICH. WMI can lead to short- and long-term neurological impairment, and functional recovery can assess the effect of drug therapy after ICH. Therefore, researchers have devised various behavioral tests to assess dysfunction. This review compares the two ICH modeling methods in rodents and summarizes the pathological mechanisms underlying dysfunction after ICH. We also summarize the functions and characteristics of various behavioral methods, including sensation, motion, emotion, and cognition, to assist researchers in selecting the appropriate tests for preclinical ICH research.

Leveraging big data to uncover the eco-evolutionary factors shaping behavioural development

Mapping the eco-evolutionary factors shaping the development of animals' behavioural phenotypes remains a great challenge. Recent advances in 'big behavioural data' research-the high-resolution tracking of individuals and the harnessing of that data with powerful analytical tools-have vastly improved our ability to measure and model developing behavioural phenotypes. Applied to the study of behavioural ontogeny, the unfolding of whole behavioural repertoires can be mapped in unprecedented detail with relative ease. This overcomes long-standing experimental bottlenecks and heralds a surge of studies that more finely define and explore behavioural-experiential trajectories across development. In this review, we first provide a brief guide to state-of-the-art approaches that allow the collection and analysis of high-resolution behavioural data across development. We then outline how such approaches can be used to address key issues regarding the ecological and evolutionary factors shaping behavioural development: developmental feedbacks between behaviour and underlying states, early life effects and behavioural transitions, and information integration across development.

Male and female mice display consistent lifelong ability to address potential life-threatening cues using different post-threat coping strategies

BACKGROUND

Sex differences ranging from physiological functions to pathological disorders are developmentally hard-wired in a broad range of animals, from invertebrates to humans. These differences ensure that animals can display appropriate behaviors under a variety of circumstances, such as aggression, hunting, sleep, mating, and parental care, which are often thought to be important in the acquisition of resources, including territory, food, and mates. Although there are reports of an absence of sexual dimorphism in the context of innate fear, the question of whether there is sexual dimorphism of innate defensive behavior is still an open question. Therefore, an in-depth investigation to determine whether there are sex differences in developmentally hard-wired innate defensive behaviors in life-threatening circumstances is warranted.

RESULTS

We found that innate defensive behavioral responses to potentially life-threatening stimuli between males and females were indistinguishable over their lifespan. However, by using 3 dimensional (3D)-motion learning framework analysis, we found that males and females showed different behavioral patterns after escaping to the refuge. Specifically, the defensive "freezing" occurred primarily in males, whereas females were more likely to return directly to exploration. Moreover, there were also no estrous phase differences in innate defensive behavioral responses after looming stimuli.

CONCLUSIONS

Our results demonstrate that visually-evoked innate fear behavior is highly conserved throughout the lifespan in both males and females, while specific post-threat coping strategies depend on sex. These findings indicate that innate fear behavior is essential to both sexes and as such, there are no evolutionary-driven sex differences in defensive ability.

Estimation of skeletal kinematics in freely moving rodents

Forming a complete picture of the relationship between neural activity and skeletal kinematics requires quantification of skeletal joint biomechanics during free behavior; however, without detailed knowledge of the underlying skeletal motion, inferring limb kinematics using surface-tracking approaches is difficult, especially for animals where the relationship between the surface and underlying skeleton changes during motion. Here we developed a videography-based method enabling detailed three-dimensional kinematic quantification of an anatomically defined skeleton in untethered freely behaving rats and mice. This skeleton-based model was constrained using anatomical principles and joint motion limits and provided skeletal pose estimates for a range of body sizes, even when limbs were occluded. Model-inferred limb positions and joint kinematics during gait and gap-crossing behaviors were verified by direct measurement of either limb placement or limb kinematics using inertial measurement units. Together we show that complex decision-making behaviors can be accurately reconstructed at the level of skeletal kinematics using our anatomically constrained model.

Selfee, self-supervised features extraction of animal behaviors

Fast and accurately characterizing animal behaviors is crucial for neuroscience research. Deep learning models are efficiently used in laboratories for behavior analysis. However, it has not been achieved to use an end-to-end unsupervised neural network to extract comprehensive and discriminative features directly from social behavior video frames for annotation and analysis purposes. Here, we report a self-supervised feature extraction (Selfee) convolutional neural network with multiple downstream applications to process video frames of animal behavior in an end-to-end way. Visualization and classification of the extracted features (Meta-representations) validate that Selfee processes animal behaviors in a way similar to human perception. We demonstrate that Meta-representations can be efficiently used to detect anomalous behaviors that are indiscernible to human observation and hint in-depth analysis. Furthermore, time-series analyses of Meta-representations reveal the temporal dynamics of animal behaviors. In conclusion, we present a self-supervised learning approach to extract comprehensive and discriminative features directly from raw video recordings of animal behaviors and demonstrate its potential usage for various downstream applications.

An Easily Compatible Eye-tracking System for Freely-moving Small Animals

Measuring eye movement is a fundamental approach in cognitive science as it provides a variety of insightful parameters that reflect brain states such as visual attention and emotions. Combining eye-tracking with multimodal neural recordings or manipulation techniques is beneficial for understanding the neural substrates of cognitive function. Many commercially-available and custom-built systems have been widely applied to awake, head-fixed small animals. However, the existing eye-tracking systems used in freely-moving animals are still limited in terms of their compatibility with other devices and of the algorithm used to detect eye movements. Here, we report a novel system that integrates a general-purpose, easily compatible eye-tracking hardware with a robust eye feature-detection algorithm. With ultra-light hardware and a detachable design, the system allows for more implants to be added to the animal's exposed head and has a precise synchronization module to coordinate with other neural implants. Moreover, we systematically compared the performance of existing commonly-used pupil-detection approaches, and demonstrated that the proposed adaptive pupil feature-detection algorithm allows the analysis of more complex and dynamic eye-tracking data in free-moving animals. Synchronized eye-tracking and electroencephalogram recordings, as well as algorithm validation under five noise conditions, suggested that our system is flexibly adaptable and can be combined with a wide range of neural manipulation and recording technologies.

MouseVenue3D: A Markerless Three-Dimension Behavioral Tracking System for Matching Two-Photon Brain Imaging in Free-Moving Mice

Understanding the connection between brain and behavior in animals requires precise monitoring of their behaviors in three-dimensional (3-D) space. However, there is no available three-dimensional behavior capture system that focuses on rodents. Here, we present MouseVenue3D, an automated and low-cost system for the efficient capture of 3-D skeleton trajectories in markerless rodents. We improved the most time-consuming step in 3-D behavior capturing by developing an automatic calibration module. Then, we validated this process in behavior recognition tasks, and showed that 3-D behavioral data achieved higher accuracy than 2-D data. Subsequently, MouseVenue3D was combined with fast high-resolution miniature two-photon microscopy for synchronous neural recording and behavioral tracking in the freely-moving mouse. Finally, we successfully decoded spontaneous neuronal activity from the 3-D behavior of mice. Our findings reveal that subtle, spontaneous behavior modules are strongly correlated with spontaneous neuronal activity patterns.

Dissection of the relationship between anxiety and stereotyped self-grooming using the Shank3B mutant autistic model, acute stress model and chronic pain model

Self-grooming is an innate, cephalo-caudal progression of body cleaning behaviors that are found in normal rodents but exhibit repetitive and stereotyped patterns in several mouse models, such as autism spectrum disorders (ASDs). It is also recognized as a marker of stress and anxiety. Mice with Shank3B gene knockout (KO) exhibit typical ASD-like behavioral abnormalities, including stereotyped self-grooming and increased levels of anxiety. However, the exact relationship between anxiety and stereotyped self-grooming in certain types of animal models is not clear. We selected three animal models with high anxiety to compare their self-grooming parameters. First, we confirmed that Shank3B KO mice (ASD model), acute restraint stress mouse model (stress model), and chronic inflammatory pain mouse model (pain model) all showed increased anxiety levels in the open field test (OFT) and elevated plus maze (EPM). We found that only the ASD model and the stress model produced increased total grooming duration. The pain model only exhibited an increasing trend of mean self-grooming duration. We used the grooming analysis algorithm to examine the self-grooming microstructure and assess the cephalo-caudal progression of grooming behavior. The results showed distinct self-grooming microstructures in these three models. The anxiolytic drug diazepam relieved the anxiety level and the total time of grooming in the ASD and stress models. The grooming microstructure was not restored in Shank3B KO mice but was partially relieved in the stress model, which suggested that anxiety aggravated stereotyped self-grooming duration but not the grooming microstructure in the ASD mouse model. Our results indicated that stereotyped behavior and anxiety may be shared by separate, but interacting, neural circuits in distinct disease models, which may be useful to understand the mechanisms and develop potential treatments for stereotyped behaviors and anxiety.

DeepBhvTracking: A Novel Behavior Tracking Method for Laboratory Animals Based on Deep Learning

Behavioral measurement and evaluation are broadly used to understand brain functions in neuroscience, especially for investigations of movement disorders, social deficits, and mental diseases. Numerous commercial software and open-source programs have been developed for tracking the movement of laboratory animals, allowing animal behavior to be analyzed digitally. In vivo optical imaging and electrophysiological recording in freely behaving animals are now widely used to understand neural functions in circuits. However, it is always a challenge to accurately track the movement of an animal under certain complex conditions due to uneven environment illumination, variations in animal models, and interference from recording devices and experimenters. To overcome these challenges, we have developed a strategy to track the movement of an animal by combining a deep learning technique, the You Only Look Once (YOLO) algorithm, with a background subtraction algorithm, a method we label DeepBhvTracking. In our method, we first train the detector using manually labeled images and a pretrained deep-learning neural network combined with YOLO, then generate bounding boxes of the targets using the trained detector, and finally track the center of the targets by calculating their centroid in the bounding box using background subtraction. Using DeepBhvTracking, the movement of animals can be tracked accurately in complex environments and can be used in different behavior paradigms and for different animal models. Therefore, DeepBhvTracking can be broadly used in studies of neuroscience, medicine, and machine learning algorithms.

Automated audiovisual behavior recognition in wild primates

Large video datasets of wild animal behavior are crucial to produce longitudinal research and accelerate conservation efforts; however, large-scale behavior analyses continue to be severely constrained by time and resources. We present a deep convolutional neural network approach and fully automated pipeline to detect and track two audiovisually distinctive actions in wild chimpanzees: buttress drumming and nut cracking. Using camera trap and direct video recordings, we train action recognition models using audio and visual signatures of both behaviors, attaining high average precision (buttress drumming: 0.87 and nut cracking: 0.85), and demonstrate the potential for behavioral analysis using the automatically parsed video. Our approach produces the first automated audiovisual action recognition of wild primate behavior, setting a milestone for exploiting large datasets in ethology and conservation.