Learning to recognize rat social behavior: Novel dataset and cross-dataset application

EthoWatcher OS: improving the reproducibility and quality of categorical and morphologic/kinematic data from behavioral recordings in laboratory animals

Behavioral recordings annotated by human observers (HOs) from video recordings are a fundamental component of preclinical animal behavioral models of neurobiological diseases. These models are often criticized for their vulnerability to reproducibility issues. Here, we present the EthoWatcher-Open Source (EW-OS), with tools and procedures for the use of blind-to-condition categorical transcriptions that are simultaneous with tracking, for the assessment of HOs intra- and interobserver reliability during training and data collection, for producing video clips of samples of behavioral categories that are useful for observer training. The use of these tools can inform and optimize the performance of observers, thus favoring the reproducibility of the data obtained. Categorical and machine vision-derived outputs are presented in an open data format for increased interoperability with other applications, where behavioral categories are associated frame-by-frame with tracking, morphological and kinematic attributes of an animal's image. The center of mass (X and Y pixel coordinates), the animal's area in square millimeters, the length and width in millimeters, and the angle in degrees were recorded. It also assesses the variation in each morphological descriptor to produce kinematic descriptors. While the initial measurements are in pixels, they are later converted to millimeters using the scale calibrated by the user via the graphical user interfaces. This process enables the creation of databases suitable for machine learning processing and behavioral pharmacology studies. EW-OS is constructed for continued collaborative development, available through an open-source platform, to support initiatives toward the adoption of good scientific practices in behavioral analysis, including tools for evaluating the quality of the data that can alleviate problems associated with low reproducibility in the behavioral sciences.

ARBUR, a machine learning-based analysis system for relating behaviors and ultrasonic vocalizations of rats

Deciphering how different behaviors and ultrasonic vocalizations (USVs) of rats interact can yield insights into the neural basis of social interaction. However, the behavior-vocalization interplay of rats remains elusive because of the challenges of relating the two communication media in complex social contexts. Here, we propose a machine learning-based analysis system (ARBUR) that can cluster without bias both non-step (continuous) and step USVs, hierarchically detect eight types of behavior of two freely behaving rats with high accuracy, and locate the vocal rat in 3-D space. ARBUR reveals that rats communicate via distinct USVs during different behaviors. Moreover, we show that ARBUR can indicate findings that are long neglected by former manual analysis, especially regarding the non-continuous USVs during easy-to-confuse social behaviors. This work could help mechanistically understand the behavior-vocalization interplay of rats and highlights the potential of machine learning algorithms in automatic animal behavioral and acoustic analysis.

Of Mice and Mates: Automated Classification and Modelling of Mouse Behaviour in Groups Using a Single Model Across Cages

Behavioural experiments often happen in specialised arenas, but this may confound the analysis. To address this issue, we provide tools to study mice in the home-cage environment, equipping biologists with the possibility to capture the temporal aspect of the individual's behaviour and model the interaction and interdependence between cage-mates with minimal human intervention. Our main contribution is the novel Global Behaviour Model (GBM) which summarises the joint behaviour of groups of mice across cages, using a permutation matrix to match the mouse identities in each cage to the model. In support of the above, we also (a) developed the Activity Labelling Module (ALM) to automatically classify mouse behaviour from video, and (b) released two datasets, ABODe for training behaviour classifiers and IMADGE for modelling behaviour.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11263-024-02118-3.

A Multimodal Preclinical Assessment of MDMA in Female and Male Rats: Prohedonic, Cognition Disruptive, and Prosocial Effects

Background

Frontline antidepressants such as selective serotonin reuptake inhibitors (SSRIs) leave many patients with unmet treatment needs. Moreover, even when SSRIs reduce depressive symptoms, anhedonia, the loss of pleasure to previously rewarding activities, often remains unabated. This state of affairs is disheartening and calls for the development of medications to more directly treat anhedonia. The atypical psychedelic 3,4-methylenedioxymethamphetamine (MDMA) might have promise as a prohedonic medication given its efficacious applications for treatment-resistant post-traumatic stress disorder and comorbid depression. However, in addition to its prosocial effects as an entactogen, MDMA is also associated with neurotoxic cognitive deficits. The present studies were designed to examine the relative potency of MDMA in female and male rats across three distinct behavioral domains to assist in defining a preclinical profile of MDMA as a candidate prohedonic therapeutic.

Methods

First, signal detection metrics of reward responsivity were examined using the touchscreen probabilistic reward task (PRT), a reverse-translated assay used to objectively quantify anhedonic phenotypes in humans. Second, to probe potential cognitive deficits, touchscreen-based assays of psychomotor vigilance and delayed matching-to-position were used to examine attentional processes and short-term spatial memory, respectively. Finally, MDMA's entactogenic effects were studied via pairwise assessments of social interaction facilitated by machine-learning analyses.

Results

Findings show (1) dose-dependent increases in reward responsivity as quantified by the PRT, (2) dose-dependent deficits in attention and short-term memory, and (3) dose-dependent increases in aspects of prosocial interaction in male but not female subjects. Neither the desirable (prohedonic) nor undesirable (cognition disruptive) effects of MDMA persisted beyond 24 h.

Conclusions

The present results characterize MDMA as a promising prohedonic treatment, notwithstanding some liability for short-lived cognitive impairment following acute administration.

Markerless Mouse Tracking for Social Experiments

Automated behavior quantification in socially interacting animals requires accurate tracking. While many methods have been very successful and highly generalizable to different settings, issues of mistaken identities and lost information on key anatomical features are common, although they can be alleviated by increased human effort in training or post-processing. We propose a markerless video-based tool to simultaneously track two interacting mice of the same appearance in controlled settings for quantifying behaviors such as different types of sniffing, touching, and locomotion to improve tracking accuracy under these settings without increased human effort. It incorporates conventional handcrafted tracking and deep-learning-based techniques. The tool is trained on a small number of manually annotated images from a basic experimental setup and outputs body masks and coordinates of the snout and tail-base for each mouse. The method was tested on several commonly used experimental conditions including bedding in the cage and fiberoptic or headstage implants on the mice. Results obtained without any human corrections after the automated analysis showed a near elimination of identities switches and a ∼15% improvement in tracking accuracy over pure deep-learning-based pose estimation tracking approaches. Our approach can be optionally ensembled with such techniques for further improvement. Finally, we demonstrated an application of this approach in studies of social behavior of mice by quantifying and comparing interactions between pairs of mice in which some lack olfaction. Together, these results suggest that our approach could be valuable for studying group behaviors in rodents, such as social interactions.

ContrastivePose: A contrastive learning approach for self-supervised feature engineering for pose estimation and behavorial classification of interacting animals

In recent years, supervised machine learning models trained on videos of animals with pose estimation data and behavior labels have been used for automated behavior classification. Applications include, for example, automated detection of neurological diseases in animal models. However, we identify two potential problems of such supervised learning approach. First, such models require a large amount of labeled data but the labeling of behaviors frame by frame is a laborious manual process that is not easily scalable. Second, such methods rely on handcrafted features obtained from pose estimation data that are usually designed empirically. In this paper, we propose to overcome these two problems using contrastive learning for self-supervised feature engineering on pose estimation data. Our approach allows the use of unlabeled videos to learn feature representations and reduce the need for handcrafting of higher-level features from pose positions. We show that this approach to feature representation can achieve better classification performance compared to handcrafted features alone, and that the performance improvement is due to contrastive learning on unlabeled data rather than the neural network architecture. The method has the potential to reduce the bottleneck of scarce labeled videos for training and improve performance of supervised behavioral classification models for the study of interaction behaviors in animals.

Changes in social behavior with MAPK2 and KCTD13/CUL3 pathways alterations in two new outbred rat models for the 16p11.2 syndromes with autism spectrum disorders

Copy number variations (CNVs) of the human 16p11.2 locus are associated with several developmental/neurocognitive syndromes. Particularly, deletion and duplication of this genetic interval are found in patients with autism spectrum disorders, intellectual disability and other psychiatric traits. The high gene density associated with the region and the strong phenotypic variability of incomplete penetrance, make the study of the 16p11.2 syndromes extremely complex. To systematically study the effect of 16p11.2 CNVs and identify candidate genes and molecular mechanisms involved in the pathophysiology, mouse models were generated previously and showed learning and memory, and to some extent social deficits. To go further in understanding the social deficits caused by 16p11.2 syndromes, we engineered deletion and duplication of the homologous region to the human 16p11.2 genetic interval in two rat outbred strains, Sprague Dawley (SD) and Long Evans (LE). The 16p11.2 rat models displayed convergent defects in social behavior and in the novel object test in male carriers from both genetic backgrounds. Interestingly major pathways affecting MAPK1 and CUL3 were found altered in the rat 16p11.2 models with additional changes in males compared to females. Altogether, the consequences of the 16p11.2 genetic region dosage on social behavior are now found in three different species: humans, mice and rats. In addition, the rat models pointed to sexual dimorphism with lower severity of phenotypes in rat females compared to male mutants. This phenomenon is also observed in humans. We are convinced that the two rat models will be key to further investigating social behavior and understanding the brain mechanisms and specific brain regions that are key to controlling social behavior.

Learning Rat-Like Behavioral Interaction Using a Small-Scale Robotic Rat

In this paper, we propose a novel method for emulating rat-like behavioral interactions in robots using reinforcement learning. Specifically, we develop a state decision method to optimize the interaction process among 6 known behavior types that have been identified in previous research on rat interactions. The novelty of our method lies in using the temporal difference (TD) algorithm to optimize the state decision process, which enables the robots to make informed decisions about their behavior choices. To assess the similarity between robot and rat behavior, we use Pearson correlation. We then use TD-λ to update the state value function and make state decisions based on probability. The robots execute these decisions using our dynamics-based controller. Our results demonstrate that our method can generate rat-like behaviors on both short- and long-term timescales, with interaction information entropy comparable to that between real rats. Overall, our approach shows promise for controlling robots in robot-rat interactions and highlights the potential of using reinforcement learning to develop more sophisticated robotic systems.

Anomaly traffic detection based on feature fluctuation for secure industrial internet of things

The detection of anomaly traffic in internet of things (IoT) is mainly based on the original binary data at the traffic packet level and the structured data at the session flow level. This kind of dataset has a single feature extraction method and relies on prior manual knowledge. It is easy to lose critical information during data processing, which reduces the validity and robustness of the dataset. In this paper, we first construct a new anomaly traffic dataset based on the traffic packet and session flow data in the Iot-23 dataset. Second, we propose a feature extraction method based on feature fluctuation. Our proposed method can effectively solve the disadvantage that the data collected in different scenarios have different characteristics, which leads to the feature containing less information. Compared with the traditional anomaly traffic detection model, experiments show that our proposed method based on feature fluctuation has stronger robustness, can improve the accuracy of anomaly traffic detection and the generalization ability of the traditional model, and is more conducive to the detection of anomalous traffic in IoT.

DeepAction: a MATLAB toolbox for automated classification of animal behavior in video

The identification of animal behavior in video is a critical but time-consuming task in many areas of research. Here, we introduce DeepAction, a deep learning-based toolbox for automatically annotating animal behavior in video. Our approach uses features extracted from raw video frames by a pretrained convolutional neural network to train a recurrent neural network classifier. We evaluate the classifier on two benchmark rodent datasets and one octopus dataset. We show that it achieves high accuracy, requires little training data, and surpasses both human agreement and most comparable existing methods. We also create a confidence score for classifier output, and show that our method provides an accurate estimate of classifier performance and reduces the time required by human annotators to review and correct automatically-produced annotations. We release our system and accompanying annotation interface as an open-source MATLAB toolbox.

Murine Motion Behavior Recognition Based on DeepLabCut and Convolutional Long Short-Term Memory Network

Murine behavior recognition is widely used in biology, neuroscience, pharmacology, and other aspects of research, and provides a basis for judging the psychological and physiological state of mice. To solve the problem whereby traditional behavior recognition methods only model behavioral changes in mice over time or space, we propose a symmetrical algorithm that can capture spatiotemporal information based on behavioral changes. The algorithm first uses the improved DeepLabCut keypoint detection algorithm to locate the nose, left ear, right ear, and tail root of the mouse, and then uses the ConvLSTM network to extract spatiotemporal information from the keypoint feature map sequence to classify five behaviors of mice: walking straight, resting, grooming, standing upright, and turning. We developed a murine keypoint detection and behavior recognition dataset, and experiments showed that the method achieved a percentage of correct keypoints (PCK) of 87±1% at three scales and against four backgrounds, while the classification accuracy for the five kinds of behaviors reached 93±1%. The proposed method is thus accurate for keypoint detection and behavior recognition, and is a useful tool for murine motion behavior recognition.

Selfee, self-supervised features extraction of animal behaviors

Fast and accurately characterizing animal behaviors is crucial for neuroscience research. Deep learning models are efficiently used in laboratories for behavior analysis. However, it has not been achieved to use an end-to-end unsupervised neural network to extract comprehensive and discriminative features directly from social behavior video frames for annotation and analysis purposes. Here, we report a self-supervised feature extraction (Selfee) convolutional neural network with multiple downstream applications to process video frames of animal behavior in an end-to-end way. Visualization and classification of the extracted features (Meta-representations) validate that Selfee processes animal behaviors in a way similar to human perception. We demonstrate that Meta-representations can be efficiently used to detect anomalous behaviors that are indiscernible to human observation and hint in-depth analysis. Furthermore, time-series analyses of Meta-representations reveal the temporal dynamics of animal behaviors. In conclusion, we present a self-supervised learning approach to extract comprehensive and discriminative features directly from raw video recordings of animal behaviors and demonstrate its potential usage for various downstream applications.

Measuring Behavior in the Home Cage: Study Design, Applications, Challenges, and Perspectives

The reproducibility crisis (or replication crisis) in biomedical research is a particularly existential and under-addressed issue in the field of behavioral neuroscience, where, in spite of efforts to standardize testing and assay protocols, several known and unknown sources of confounding environmental factors add to variance. Human interference is a major contributor to variability both within and across laboratories, as well as novelty-induced anxiety. Attempts to reduce human interference and to measure more "natural" behaviors in subjects has led to the development of automated home-cage monitoring systems. These systems enable prolonged and longitudinal recordings, and provide large continuous measures of spontaneous behavior that can be analyzed across multiple time scales. In this review, a diverse team of neuroscientists and product developers share their experiences using such an automated monitoring system that combines Noldus PhenoTyper® home-cages and the video-based tracking software, EthoVision® XT, to extract digital biomarkers of motor, emotional, social and cognitive behavior. After presenting our working definition of a "home-cage", we compare home-cage testing with more conventional out-of-cage tests (e.g., the open field) and outline the various advantages of the former, including opportunities for within-subject analyses and assessments of circadian and ultradian activity. Next, we address technical issues pertaining to the acquisition of behavioral data, such as the fine-tuning of the tracking software and the potential for integration with biotelemetry and optogenetics. Finally, we provide guidance on which behavioral measures to emphasize, how to filter, segment, and analyze behavior, and how to use analysis scripts. We summarize how the PhenoTyper has applications to study neuropharmacology as well as animal models of neurodegenerative and neuropsychiatric illness. Looking forward, we examine current challenges and the impact of new developments. Examples include the automated recognition of specific behaviors, unambiguous tracking of individuals in a social context, the development of more animal-centered measures of behavior and ways of dealing with large datasets. Together, we advocate that by embracing standardized home-cage monitoring platforms like the PhenoTyper, we are poised to directly assess issues pertaining to reproducibility, and more importantly, measure features of rodent behavior under more ethologically relevant scenarios.

Let's get wild: A review of free-ranging rat assays as context-enriched supplements to traditional laboratory models

More than 24,000 rodent studies are published annually, with the vast majority of these studies focused on genetically undiverse animals in highly-controlled laboratory settings. However, findings from the laboratory have become increasingly unreliable for predicting outcomes in field and clinical settings, leading to a perceived crisis in translational research. One cause of this disparity might be that most human societies, in contrast to laboratory rodents, are genetically diverse and live in super-enriched environments. Methods for importing wild rats into the laboratory, and also exporting laboratory-style chambers into natural environments are not well-known outside their respective disciplines. Therefore, we have reviewed the current status of supplements to the laboratory rodent assay. We progress logically from highly-controlled experiments with natural breeding colonies to purely naturalistic approaches with free-ranging rats. We then highlight a number of approaches that allow genetically-diverse wild rats to be utilized in context-enriched paradigms. While considering the benefits and shortcomings of each available approach, we detail protocols for random sampling, remote-sensing, and deployment of laboratory chambers in the field. As supplements to standardized laboratory trials, some of these assays could offer key insights to help unify outcomes between laboratory and field studies. However, we note several outstanding questions that must be addressed such as: the trade-off between control and context, possible reductions in sample size, ramifications for the 'standardization fallacy', and ethical dilemmas of working with wild animals. Given these challenges, further innovation will be required before supplemental assays can be made broadly-accessible and thus, transferrable across disciplines.

Biomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation

The failure of animal studies to translate to effective clinical therapeutics has driven efforts to identify underlying cause and develop solutions that improve the reproducibility and translatability of preclinical research. Common issues revolve around study design, analysis, and reporting as well as standardization between preclinical and clinical endpoints. To address these needs, recent advancements in digital technology, including biomonitoring of digital biomarkers, development of software systems and database technologies, as well as application of artificial intelligence to preclinical datasets can be used to increase the translational relevance of preclinical animal research. In this review, we will describe how a number of innovative digital technologies are being applied to overcome recurring challenges in study design, execution, and data sharing as well as improving scientific outcome measures. Examples of how these technologies are applied to specific therapeutic areas are provided. Digital technologies can enhance the quality of preclinical research and encourage scientific collaboration, thus accelerating the development of novel therapeutics.

Improved 3D tracking and automated classification of rodents' behavioral activity using depth-sensing cameras

Analysis of rodents' behavior/activity is of fundamental importance in many research fields. However, many behavioral experiments still rely on manual scoring, with obvious problems in reproducibility. Despite important advances in video-analysis systems and computational ethology, automated behavior quantification is still a challenge. The need for large training datasets, background stability requirements, and reduction to two-dimensional analysis (impairing full posture characterization), limit their use. Here we present a novel integrated solution for behavioral analysis of individual rats, combining video segmentation, tracking of body parts, and automated classification of behaviors, using machine learning and computer vision methods. Low-cost depth cameras (RGB-D) are used to enable three-dimensional tracking and classification in dark conditions and absence of color contrast. Our solution automatically tracks five anatomical landmarks in dynamic environments and recognizes seven distinct behaviors, within the accuracy range of human annotations. The developed free software was validated in experiments where behavioral differences between Wistar Kyoto and Wistar rats were automatically quantified. The results reveal the capability for effective automated phenotyping. An extended annotated RGB-D dataset is also made publicly available. The proposed solution is an easy-to-use tool, with low-cost setup and powerful 3D segmentation methods (in static/dynamic environments). The ability to work in dark conditions means that natural animal behavior is not affected by recording lights. Furthermore, automated classification is possible with only ~30 minutes of annotated videos. By creating conditions for high-throughput analysis and reproducible quantitative measurements of animal behavior experiments, we believe this contribution can greatly improve behavioral analysis research.

Big behavior: challenges and opportunities in a new era of deep behavior profiling

The assessment of rodent behavior forms a cornerstone of preclinical assessment in neuroscience research. Nonetheless, the true and almost limitless potential of behavioral analysis has been inaccessible to scientists until very recently. Now, in the age of machine vision and deep learning, it is possible to extract and quantify almost infinite numbers of behavioral variables, to break behaviors down into subcategories and even into small behavioral units, syllables or motifs. However, the rapidly growing field of behavioral neuroethology is experiencing birthing pains. The community has not yet consolidated its methods, and new algorithms transfer poorly between labs. Benchmarking experiments as well as the large, well-annotated behavior datasets required are missing. Meanwhile, big data problems have started arising and we currently lack platforms for sharing large datasets-akin to sequencing repositories in genomics. Additionally, the average behavioral research lab does not have access to the latest tools to extract and analyze behavior, as their implementation requires advanced computational skills. Even so, the field is brimming with excitement and boundless opportunity. This review aims to highlight the potential of recent developments in the field of behavioral analysis, whilst trying to guide a consensus on practical issues concerning data collection and data sharing.

Deep learning-based behavioral analysis reaches human accuracy and is capable of outperforming commercial solutions

To study brain function, preclinical research heavily relies on animal monitoring and the subsequent analyses of behavior. Commercial platforms have enabled semi high-throughput behavioral analyses by automating animal tracking, yet they poorly recognize ethologically relevant behaviors and lack the flexibility to be employed in variable testing environments. Critical advances based on deep-learning and machine vision over the last couple of years now enable markerless tracking of individual body parts of freely moving rodents with high precision. Here, we compare the performance of commercially available platforms (EthoVision XT14, Noldus; TSE Multi-Conditioning System, TSE Systems) to cross-verified human annotation. We provide a set of videos-carefully annotated by several human raters-of three widely used behavioral tests (open field test, elevated plus maze, forced swim test). Using these data, we then deployed the pose estimation software DeepLabCut to extract skeletal mouse representations. Using simple post-analyses, we were able to track animals based on their skeletal representation in a range of classic behavioral tests at similar or greater accuracy than commercial behavioral tracking systems. We then developed supervised machine learning classifiers that integrate the skeletal representation with the manual annotations. This new combined approach allows us to score ethologically relevant behaviors with similar accuracy to humans, the current gold standard, while outperforming commercial solutions. Finally, we show that the resulting machine learning approach eliminates variation both within and between human annotators. In summary, our approach helps to improve the quality and accuracy of behavioral data, while outperforming commercial systems at a fraction of the cost.

The Neuropharmacology of Impulsive Behaviour, an Update

Neuropharmacological interventions in preclinical translational models of impulsivity have tremendously contributed to a better understanding of the neurochemistry and neural basis of impulsive behaviour. In this regard, much progress has been made over the last years, also due to the introduction of novel techniques in behavioural neuroscience such as optogenetics and chemogenetics. In this chapter, we will provide an update of how the behavioural pharmacology field has progressed and built upon existing data since an earlier review we wrote in 2008. To this aim, we will first give a brief background on preclinical translational models of impulsivity. Next, recent interesting evidence of monoaminergic modulation of impulsivity will be highlighted with a focus on the neurotransmitters dopamine and noradrenaline. Finally, we will close the chapter by discussing some novel directions and drug leads in the neuropharmacological modulation of impulsivity.

A Robust Real-Time Detecting and Tracking Framework for Multiple Kinds of Unmarked Object

A rodent real-time tracking framework is proposed to automatically detect and track multi-objects in real time and output the coordinates of each object, which combines deep learning (YOLO v3: You Only Look Once, v3), the Kalman Filter, improved Hungarian algorithm, and the nine-point position correction algorithm. A model of a Rat-YOLO is trained in our experiment. The Kalman Filter model is established in an acceleration model to predict the position of the rat in the next frame. The predicted data is used to fill the losing position of rats if the Rat-YOLO doesn't work in the current frame, and to associate the ID between the last frame and current frame. The Hungarian assigned algorithm is used to show the relationship between the objects of the last frame and the objects of the current frame and match the ID of the objects. The nine-point position correction algorithm is presented to adjust the correctness of the Rat-YOLO result and the predicted results. As the training of deep learning needs more datasets than our experiment, and it is time-consuming to process manual marking, automatic software for generating labeled datasets is proposed under a fixed scene and the labeled datasets are manually verified in term of their correctness. Besides this, in an off-line experiment, a mask is presented to remove the highlight. In this experiment, we select the 500 frames of the data as the training datasets and label these images with the automatic label generating software. A video (of 2892 frames) is tested by the trained Rat model and the accuracy of detecting all the three rats is around 72.545%, however, the Rat-YOLO combining the Kalman Filter and nine-point position correction arithmetic improved the accuracy to 95.194%.

Rigor and reproducibility in rodent behavioral research

Behavioral neuroscience research incorporates the identical high level of meticulous methodologies and exacting attention to detail as all other scientific disciplines. To achieve maximal rigor and reproducibility of findings, well-trained investigators employ a variety of established best practices. Here we explicate some of the requirements for rigorous experimental design and accurate data analysis in conducting mouse and rat behavioral tests. Novel object recognition is used as an example of a cognitive assay which has been conducted successfully with a range of methods, all based on common principles of appropriate procedures, controls, and statistics. Directors of Rodent Core facilities within Intellectual and Developmental Disabilities Research Centers contribute key aspects of their own novel object recognition protocols, offering insights into essential similarities and less-critical differences. Literature cited in this review article will lead the interested reader to source papers that provide step-by-step protocols which illustrate optimized methods for many standard rodent behavioral assays. Adhering to best practices in behavioral neuroscience will enhance the value of animal models for the multiple goals of understanding biological mechanisms, evaluating consequences of genetic mutations, and discovering efficacious therapeutics.

An automated behavior analysis system for freely moving rodents using depth image

A rodent behavior analysis system is presented, capable of automated tracking, pose estimation, and recognition of nine behaviors in freely moving animals. The system tracks three key points on the rodent body (nose, center of body, and base of tail) to estimate its pose and head rotation angle in real time. A support vector machine (SVM)-based model, including label optimization steps, is trained to classify on a frame-by-frame basis: resting, walking, bending, grooming, sniffing, rearing supported, rearing unsupported, micro-movements, and "other" behaviors. Compared to conventional red-green-blue (RGB) camera-based methods, the proposed system operates on 3D depth images provided by the Kinect infrared (IR) camera, enabling stable performance regardless of lighting conditions and animal color contrast with the background. This is particularly beneficial for monitoring nocturnal animals' behavior. 3D features are designed to be extracted directly from the depth stream and combined with contour-based 2D features to further improve recognition accuracies. The system is validated on three freely behaving rats for 168 min in total. The behavior recognition model achieved a cross-validation accuracy of 86.8% on the rat used for training and accuracies of 82.1 and 83% on the other two "testing" rats. The automated head angle estimation aided by behavior recognition resulted in 0.76 correlation with human expert annotation. Graphical abstract Top view of a rat freely behaving in a standard homecage, captured by Kinect-v2 sensors. The depth image is used for constructing a 3D topography of the animal for pose estimation, behavior recognition, and head angle calculation. Results of the processed data are displayed on the user interface in various forms.