Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Characterizing the structure of mouse behavior using Motion Sequencing

Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.

Identifying dysfunctional cell types and circuits in animal models for psychiatric disorders with calcium imaging

A central goal of neuroscience is to understand how the brain transforms external stimuli and internal bodily signals into patterns of activity that underlie cognition, emotional states, and behavior. Understanding how these patterns of activity may be disrupted in mental illness is crucial for developing novel therapeutics. It is well appreciated that psychiatric disorders are complex, circuit-based disorders that arise from dysfunctional activity patterns generated in discrete cell types and their connections. Recent advances in large-scale, cell-type specific calcium imaging approaches have shed new light on the cellular, circuit, and network-level dysfunction in animal models for psychiatric disorders. Here, we highlight a series of recent findings over the last ~10 years from in vivo calcium imaging studies that show how aberrant patterns of activity in discrete cell types and circuits may underlie behavioral deficits in animal models for several psychiatric disorders, including depression, anxiety, autism spectrum disorders, and schizophrenia. These advances in calcium imaging in pre-clinical models demonstrate the power of cell-type-specific imaging tools in understanding the underlying dysfunction in cell types, activity patterns, and neural circuits that may contribute to disease and provide new blueprints for developing more targeted therapeutics and treatment strategies.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings

Accurate detection and tracking of animals across diverse environments are crucial for studying brain and behavior. Recently, computer vision techniques have become essential for high-throughput behavioral studies; however, localizing animals in complex conditions remains challenging due to intra-class visual variability and environmental diversity. These challenges hinder studies in naturalistic settings, such as when animals are partially concealed within nests. Moreover, current tools are laborious and time-consuming, requiring extensive, setup-specific annotation and training procedures. To address these challenges, we introduce the 'Detect-Any-Mouse-Model' (DAMM), an object detector for localizing mice in complex environments with minimal training. Our approach involved collecting and annotating a diverse dataset of single- and multi-housed mice in complex setups. We trained a Mask R-CNN, a popular object detector in animal studies, to perform instance segmentation and validated DAMM's performance on a collection of downstream datasets using zero-shot and few-shot inference. DAMM excels in zero-shot inference, detecting mice and even rats, in entirely unseen scenarios and further improves with minimal training. Using the SORT algorithm, we demonstrate robust tracking, competitive with keypoint-estimation-based methods. Notably, to advance and simplify behavioral studies, we release our code, model weights, and data, along with a user-friendly Python API and a Google Colab implementation.

Integrating computer vision and molecular neurobiology to bridge the gap between behavior and the brain

The past decade of social insect research has seen rapid development in automated behavioral tracking and molecular profiling of the nervous system, two distinct but complementary lines of inquiry into phenotypic variation across individuals, colonies, populations, and species. These experimental strategies have developed largely in parallel, as automated tracking generates a continuous stream of behavioral data, while, in contrast, 'omics-based profiling provides a single 'snapshot' of the brain. Better integration of these approaches applied to studying variation in social behavior will reveal the underlying genetic and neurobiological mechanisms that shape the evolution and diversification of social life. In this review, we discuss relevant advances in both fields and propose new strategies to better elucidate the molecular and behavioral innovations that generate social life.

Probabilistic modeling reveals coordinated social interaction states and their multisensory bases

Social behavior across animal species ranges from simple pairwise interactions to thousands of individuals coordinating goal-directed movements. Regardless of the scale, these interactions are governed by the interplay between multimodal sensory information and the internal state of each animal. Here, we investigate how animals use multiple sensory modalities to guide social behavior in the highly social zebrafish (Danio rerio) and uncover the complex features of pairwise interactions early in development. To identify distinct behaviors and understand how they vary over time, we developed a new hidden Markov model with constrained linear-model emissions to automatically classify states of coordinated interaction, using the movements of one animal to predict those of another. We discovered that social behaviors alternate between two interaction states within a single experimental session, distinguished by unique movements and timescales. Long-range interactions, akin to shoaling, rely on vision, while mechanosensation underlies rapid synchronized movements and parallel swimming, precursors of schooling. Altogether, we observe spontaneous interactions in pairs of fish, develop novel hidden Markov modeling to reveal two fundamental interaction modes, and identify the sensory systems involved in each. Our modeling approach to pairwise social interactions has broad applicability to a wide variety of naturalistic behaviors and species and solves the challenge of detecting transient couplings between quasi-periodic time series.

Quantifying social roles in multi-animal videos using subject-aware deep-learning

Analyzing social behaviors is critical for many fields, including neuroscience, psychology, and ecology. While computational tools have been developed to analyze videos containing animals engaging in limited social interactions under specific experimental conditions, automated identification of the social roles of freely moving individuals in a multi-animal group remains unresolved. Here we describe a deep-learning-based system - named LabGym2 - for identifying and quantifying social roles in multi-animal groups. This system uses a subject-aware approach: it evaluates the behavioral state of every individual in a group of two or more animals while factoring in its social and environmental surroundings. We demonstrate the performance of subject-aware deep-learning in different species and assays, from partner preference in freely-moving insects to primate social interactions in the field. Our subject-aware deep learning approach provides a controllable, interpretable, and efficient framework to enable new experimental paradigms and systematic evaluation of interactive behavior in individuals identified within a group.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce 'Lightning Pose', an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We released a cloud application that allows users to label data, train networks and process new videos directly from the browser.

Temporally aligned segmentation and clustering (TASC) framework for behavior time series analysis

Behavior exhibits a complex spatiotemporal structure consisting of discrete sub-behaviors, or motifs. Continuous behavior data requires segmentation and clustering to reveal these embedded motifs. The popularity of automatic behavior quantification is growing, but existing solutions are often tailored to specific needs and are not designed for the time scale and precision required in many experimental and clinical settings. Here we propose a generalized framework with an iterative approach to refine both segmentation and clustering. Temporally aligned segmentation and clustering (TASC) uses temporal linear alignment to compute distances between and align the recurring behavior motifs in a multidimensional time series, enabling precise segmentation and clustering. We introduce an alternating-step process: evaluation of temporal neighbors against current cluster centroids using linear alignment, alternating with selecting the best non-overlapping segments and their subsequent re-clustering. The framework is evaluated on semi-synthetic and real-world experimental and clinical data, demonstrating enhanced segmentation and clustering, offering a better foundation for consequent research. The framework may be used to extend existing tools in the field of behavior research and may be applied to other domains requiring high precision of time series segmentation.

Odors in space

As an evolutionarily ancient sense, olfaction is key to learning where to find food, shelter, mates, and important landmarks in an animal's environment. Brain circuitry linking odor and navigation appears to be a well conserved multi-region system among mammals; the anterior olfactory nucleus, piriform cortex, entorhinal cortex, and hippocampus each represent different aspects of olfactory and spatial information. We review recent advances in our understanding of the neural circuits underlying odor-place associations, highlighting key choices of behavioral task design and neural circuit manipulations for investigating learning and memory.

SuperAnimal pretrained pose estimation models for behavioral analysis

Quantification of behavior is critical in diverse applications from neuroscience, veterinary medicine to animal conservation. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present SuperAnimal, a method to develop unified foundation models that can be used on over 45 species, without additional manual labels. These models show excellent performance across six pose estimation benchmarks. We demonstrate how to fine-tune the models (if needed) on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If fine-tuned, SuperAnimal models are 10-100× more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification and kinematic analysis. Collectively, we present a data-efficient solution for animal pose estimation.

Social threat alters the behavioral structure of social motivation and reshapes functional brain connectivity

Traumatic social experiences redefine socially motivated behaviors to enhance safety and survival. Although many brain regions have been implicated in signaling a social threat, the mechanisms by which global neural networks regulate such motivated behaviors remain unclear. To address this issue, we first combined traditional and modern behavioral tracking techniques in mice to assess both approach and avoidance, as well as sub-second behavioral changes, during a social threat learning task. We were able to identify previously undescribed body and tail movements during social threat learning and recognition that demonstrate unique alterations into the behavioral structure of social motivation. We then utilized inter-regional correlation analysis of brain activity after a mouse recognizes a social threat to explore functional communication amongst brain regions implicated in social motivation. Broad brain activity changes were observed within the nucleus accumbens, the paraventricular thalamus, the ventromedial hypothalamus, and the nucleus of reuniens. Inter-regional correlation analysis revealed a reshaping of the functional connectivity across the brain when mice recognize a social threat. Altogether, these findings suggest that reshaping of functional brain connectivity may be necessary to alter the behavioral structure of social motivation when a social threat is encountered.

Combining supervised and unsupervised analyses to quantify behavioral phenotypes and validate therapeutic efficacy in a triple transgenic mouse model of Alzheimer's disease

Behavioral testing is an essential tool for evaluating cognitive function and dysfunction in preclinical research models. This is of special importance in the study of neurological disorders such as Alzheimer's disease. However, the reproducibility of classic behavioral assays is frequently compromised by interstudy variation, leading to ambiguous conclusions about the behavioral markers characterizing the disease. Here, we identify age- and genotype-driven differences between 3xTg-AD and non-transgenic control mice using a low-cost, highly customizable behavioral assay that requires little human intervention. Through behavioral phenotyping combining both supervised and unsupervised behavioral classification methods, we are able to validate the preventative effects of the immunosuppressant cyclosporine A in a rodent model of Alzheimer's disease, as well as the partially ameliorating effects of candidate drugs nebivolol and cabozantinib.

Visual detection of seizures in mice using supervised machine learning

Seizures are caused by abnormally synchronous brain activity that can result in changes in muscle tone, such as twitching, stiffness, limpness, or rhythmic jerking. These behavioral manifestations are clear on visual inspection and the most widely used seizure scoring systems in preclinical models, such as the Racine scale in rodents, use these behavioral patterns in semiquantitative seizure intensity scores. However, visual inspection is time-consuming, low-throughput, and partially subjective, and there is a need for rigorously quantitative approaches that are scalable. In this study, we used supervised machine learning approaches to develop automated classifiers to predict seizure severity directly from noninvasive video data. Using the PTZ-induced seizure model in mice, we trained video-only classifiers to predict ictal events, combined these events to predict an univariate seizure intensity for a recording session, as well as time-varying seizure intensity scores. Our results show, for the first time, that seizure events and overall intensity can be rigorously quantified directly from overhead video of mice in a standard open field using supervised approaches. These results enable high-throughput, noninvasive, and standardized seizure scoring for downstream applications such as neurogenetics and therapeutic discovery.

Pan-cortical 2-photon mesoscopic imaging and neurobehavioral alignment in awake, behaving mice

The flow of neural activity across the neocortex during active sensory discrimination is constrained by task-specific cognitive demands, movements, and internal states. During behavior, the brain appears to sample from a broad repertoire of activation motifs. Understanding how these patterns of local and global activity are selected in relation to both spontaneous and task-dependent behavior requires in-depth study of densely sampled activity at single neuron resolution across large regions of cortex. In a significant advance toward this goal, we developed procedures to record mesoscale 2-photon Ca2+ imaging data from two novel in vivo preparations that, between them, allow for simultaneous access to nearly all 0f the mouse dorsal and lateral neocortex. As a proof of principle, we aligned neural activity with both behavioral primitives and high-level motifs to reveal the existence of large populations of neurons that coordinated their activity across cortical areas with spontaneous changes in movement and/or arousal. The methods we detail here facilitate the identification and exploration of widespread, spatially heterogeneous neural ensembles whose activity is related to diverse aspects of behavior.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, resulting in variability that ultimately reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build models that bridges across time scales that enable a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish exposed to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising and wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive (dark) stimuli or in search for prey. Our method enables us to encode the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies. By doing so, we uncover a hierarchical structure to the phenotypic variability that corresponds to exploration-exploitation trade-offs. Within a wide range of sensory cues, a major source of variation among fish is driven by prior and immediate exposure to prey that induces exploitation phenotypes. However, a large degree of variability is unexplained by environmental cues, pointing to hidden states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, our approach extracts the timescales of motor strategies deployed during navigation, exposing undiscovered structure among individuals and pointing to internal states tuned by prior experience.

The choice-wide behavioral association study: data-driven identification of interpretable behavioral components

Behavior contains rich structure across many timescales, but there is a dearth of methods to identify relevant components, especially over the longer periods required for learning and decision-making. Inspired by the goals and techniques of genome-wide association studies, we present a data-driven method-the choice-wide behavioral association study: CBAS-that systematically identifies such behavioral features. CBAS uses a powerful, resampling-based, method of multiple comparisons correction to identify sequences of actions or choices that either differ significantly between groups or significantly correlate with a covariate of interest. We apply CBAS to different tasks and species (flies, rats, and humans) and find, in all instances, that it provides interpretable information about each behavioral task.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce "Lightning Pose," an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry, and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post-hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We release a cloud application that allows users to label data, train networks, and predict new videos directly from the browser.

FABEL: Forecasting Animal Behavioral Events with Deep Learning-Based Computer Vision

Behavioral neuroscience aims to provide a connection between neural phenomena and emergent organism-level behaviors. This requires perturbing the nervous system and observing behavioral outcomes, and comparing observed post-perturbation behavior with predicted counterfactual behavior and therefore accurate behavioral forecasts. In this study we present FABEL, a deep learning method for forecasting future animal behaviors and locomotion trajectories from historical locomotion alone. We train an offline pose estimation network to predict animal body-part locations in behavioral video; then sequences of pose vectors are input to deep learning time-series forecasting models. Specifically, we train an LSTM network that predicts a future food interaction event in a specified time window, and a Temporal Fusion Transformer that predicts future trajectories of animal body-parts, which are then converted into probabilistic label forecasts. Importantly, accurate prediction of food interaction provides a basis for neurobehavioral intervention in the context of compulsive eating. We show promising results on forecasting tasks between 100 milliseconds and 5 seconds timescales. Because the model takes only behavioral video as input, it can be adapted to any behavioral task and does not require specific physiological readouts. Simultaneously, these deep learning models may serve as extensible modules that can accommodate diverse signals, such as in-vivo fluorescence imaging and electrophysiology, which may improve behavior forecasts and elucidate invervention targets for desired behavioral change.

Pan-cortical 2-photon mesoscopic imaging and neurobehavioral alignment in awake, behaving mice

The flow of neural activity across the neocortex during active sensory discrimination is constrained by task-specific cognitive demands, movements, and internal states. During behavior, the brain appears to sample from a broad repertoire of activation motifs. Understanding how these patterns of local and global activity are selected in relation to both spontaneous and task-dependent behavior requires in-depth study of densely sampled activity at single neuron resolution across large regions of cortex. In a significant advance toward this goal, we developed procedures to record mesoscale 2-photon Ca2+ imaging data from two novel in vivo preparations that, between them, allow simultaneous access to nearly all of the mouse dorsal and lateral neocortex. As a proof of principle, we aligned neural activity with both behavioral primitives and high-level motifs to reveal the existence of large populations of neurons that coordinated their activity across cortical areas with spontaneous changes in movement and/or arousal. The methods we detail here facilitate the identification and exploration of widespread, spatially heterogeneous neural ensembles whose activity is related to diverse aspects of behavior.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings

Accurate detection and tracking of animals across diverse environments are crucial for behavioral studies in various disciplines, including neuroscience. Recently, machine learning and computer vision techniques have become integral to the neuroscientist's toolkit, enabling high-throughput behavioral studies. Despite advancements in localizing individual animals in simple environments, the task remains challenging in complex conditions due to intra-class visual variability and environmental diversity. These limitations hinder studies in ethologically-relevant conditions, such as when animals are concealed within nests or in obscured environments. Moreover, current tools are laborious and time-consuming to employ, requiring extensive, setup-specific annotation and model training/validation procedures. To address these challenges, we introduce the 'Detect Any Mouse Model' (DAMM), a pretrained object detector for localizing mice in complex environments, capable of robust performance with zero to minimal additional training on new experimental setups. Our approach involves collecting and annotating a diverse dataset that encompasses single and multi-housed mice in various lighting conditions, experimental setups, and occlusion levels. We utilize the Mask R-CNN architecture for instance segmentation and validate DAMM's performance with no additional training data (zero-shot inference) and with few examples for fine-tuning (few-shot inference). DAMM excels in zero-shot inference, detecting mice, and even rats, in entirely unseen scenarios and further improves with minimal additional training. By integrating DAMM with the SORT algorithm, we demonstrate robust tracking, competitively performing with keypoint-estimation-based methods. Finally, to advance and simplify behavioral studies, we made DAMM accessible to the scientific community with a user-friendly Python API, shared model weights, and a Google Colab implementation.

SaLSa: A Combinatory Approach of Semi-Automatic Labeling and Long Short-Term Memory to Classify Behavioral Syllables

Accurately and quantitatively describing mouse behavior is an important area. Although advances in machine learning have made it possible to track their behaviors accurately, reliable classification of behavioral sequences or syllables remains a challenge. In this study, we present a novel machine learning approach, called SaLSa (a combination of semi-automatic labeling and long short-term memory-based classification), to classify behavioral syllables of mice exploring an open field. This approach consists of two major steps. First, after tracking multiple body parts, spatial and temporal features of their egocentric coordinates are extracted. A fully automated unsupervised process identifies candidates for behavioral syllables, followed by manual labeling of behavioral syllables using a graphical user interface (GUI). Second, a long short-term memory (LSTM) classifier is trained with the labeled data. We found that the classification performance was marked over 97%. It provides a performance equivalent to a state-of-the-art model while classifying some of the syllables. We applied this approach to examine how hyperactivity in a mouse model of Alzheimer's disease develops with age. When the proportion of each behavioral syllable was compared between genotypes and sexes, we found that the characteristic hyperlocomotion of female Alzheimer's disease mice emerges between four and eight months. In contrast, age-related reduction in rearing is common regardless of genotype and sex. Overall, SaLSa enables detailed characterization of mouse behavior.

Elephants and algorithms: a review of the current and future role of AI in elephant monitoring

Artificial intelligence (AI) and machine learning (ML) present revolutionary opportunities to enhance our understanding of animal behaviour and conservation strategies. Using elephants, a crucial species in Africa and Asia's protected areas, as our focal point, we delve into the role of AI and ML in their conservation. Given the increasing amounts of data gathered from a variety of sensors like cameras, microphones, geophones, drones and satellites, the challenge lies in managing and interpreting this vast data. New AI and ML techniques offer solutions to streamline this process, helping us extract vital information that might otherwise be overlooked. This paper focuses on the different AI-driven monitoring methods and their potential for improving elephant conservation. Collaborative efforts between AI experts and ecological researchers are essential in leveraging these innovative technologies for enhanced wildlife conservation, setting a precedent for numerous other species.

A developmentally defined population of neurons in the lateral septum controls responses to aversive stimuli

When interacting with their environment, animals must balance exploratory and defensive behavior to evaluate and respond to potential threats. The lateral septum (LS) is a structure in the ventral forebrain that calibrates the magnitude of behavioral responses to stress-related external stimuli, including the regulation of threat avoidance. The complex connectivity between the LS and other parts of the brain, together with its largely unexplored neuronal diversity, makes it difficult to understand how defined LS circuits control specific behaviors. Here, we describe a mouse model in which a population of neurons with a common developmental origin (Nkx2.1-lineage neurons) are absent from the LS. Using a combination of circuit tracing and behavioral analyses, we found that these neurons receive inputs from the perifornical area of the anterior hypothalamus (PeFAH) and are specifically activated in stressful contexts. Mice lacking Nkx2.1-lineage LS neurons display increased exploratory behavior even under stressful conditions. Our study extends the current knowledge about how defined neuronal populations within the LS can evaluate contextual information to select appropriate behavioral responses. This is a necessary step towards understanding the crucial role that the LS plays in neuropsychiatric conditions where defensive behavior is dysregulated, such as anxiety and aggression disorders.

Challenges and advanced concepts for the assessment of learning and memory function in mice

The mechanisms underlying the formation and retrieval of memories are still an active area of research and discussion. Manifold models have been proposed and refined over the years, with most assuming a dichotomy between memory processes involving non-conscious and conscious mechanisms. Despite our incomplete understanding of the underlying mechanisms, tests of memory and learning count among the most performed behavioral experiments. Here, we will discuss available protocols for testing learning and memory using the example of the most prevalent animal species in research, the laboratory mouse. A wide range of protocols has been developed in mice to test, e.g., object recognition, spatial learning, procedural memory, sequential problem solving, operant- and fear conditioning, and social recognition. Those assays are carried out with individual subjects in apparatuses such as arenas and mazes, which allow for a high degree of standardization across laboratories and straightforward data interpretation but are not without caveats and limitations. In animal research, there is growing concern about the translatability of study results and animal welfare, leading to novel approaches beyond established protocols. Here, we present some of the more recent developments and more advanced concepts in learning and memory testing, such as multi-step sequential lockboxes, assays involving groups of animals, as well as home cage-based assays supported by automated tracking solutions; and weight their potential and limitations against those of established paradigms. Shifting the focus of learning tests from the classical experimental chamber to settings which are more natural for rodents comes with a new set of challenges for behavioral researchers, but also offers the opportunity to understand memory formation and retrieval in a more conclusive way than has been attainable with conventional test protocols. We predict and embrace an increase in studies relying on methods involving a higher degree of automatization, more naturalistic- and home cage-based experimental setting as well as more integrated learning tasks in the future. We are confident these trends are suited to alleviate the burden on animal subjects and improve study designs in memory research.

Disentangling rodent behaviors to improve automated behavior recognition

Automated observation and analysis of behavior is important to facilitate progress in many fields of science. Recent developments in deep learning have enabled progress in object detection and tracking, but rodent behavior recognition struggles to exceed 75-80% accuracy for ethologically relevant behaviors. We investigate the main reasons why and distinguish three aspects of behavior dynamics that are difficult to automate. We isolate these aspects in an artificial dataset and reproduce effects with the state-of-the-art behavior recognition models. Having an endless amount of labeled training data with minimal input noise and representative dynamics will enable research to optimize behavior recognition architectures and get closer to human-like recognition performance for behaviors with challenging dynamics.