Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Mapping the landscape of social behavior

Social interaction is integral to animal behavior. However, we lack tools to describe it with quantitative rigor, limiting our understanding of its principles and neuropsychiatric disorders, like autism, that perturb it. Here, we present a technique for high-resolution 3D tracking of postural dynamics and social touch in freely interacting animals, solving the challenging subject occlusion and part assignment problems using 3D geometric reasoning, graph neural networks, and semi-supervised learning. We collected over 140 million 3D postures in interacting rodents, featuring new monogenic autism rat lines lacking reports of social behavioral phenotypes. Using a novel multi-scale embedding approach, we identified a rich landscape of stereotyped actions, interactions, synchrony, and body contact. This enhanced phenotyping revealed a spectrum of changes in autism models and in response to amphetamine that were inaccessible to conventional measurements. Our framework and large library of interactions will greatly facilitate studies of social behaviors and their neurobiological underpinnings.

A Novel Head-Fixed Assay for Social Touch in Mice Uncovers Aversive Responses in Two Autism Models

Social touch, an important aspect of social interaction and communication, is essential to kinship across animal species. How animals experience and respond to social touch has not been thoroughly investigated, in part because of the lack of appropriate assays. Previous studies that examined social touch in freely moving rodents lacked the necessary temporal and spatial control over individual touch interactions. We designed a novel head-fixed assay for social touch in mice, in which the experimenter has complete control to elicit highly stereotyped bouts of social touch between two animals. The user determines the number, duration, context, and type of social touch interactions, while monitoring an array of complex behavioral responses with high resolution cameras. We focused on social touch to the face because of its high translational relevance to humans. We validated this assay in two different models of autism spectrum disorder (ASD), the Fmr1 knock-out (KO) model of Fragile X syndrome (FXS) and maternal immune activation (MIA) mice. We observed higher rates of avoidance running, hyperarousal, and aversive facial expressions (AFEs) to social touch than to object touch, in both ASD models compared with controls. Fmr1 KO mice showed more AFEs to mice of the same sex but whether they were stranger or familiar mice mattered less. Because this new social touch assay for head-fixed mice can be used to record neural activity during repeated bouts of social touch it could be used to uncover underlying circuit differences.SIGNIFICANCE STATEMENT Social touch is important for communication in animals and humans. However, it has not been extensively studied and current assays to measure animals' responses to social touch have limitations. We present a novel head-fixed assay to quantify how mice respond to social facial touch with another mouse. We validated this assay in autism mouse models since autistic individuals exhibit differences in social interaction and touch sensitivity. We find that mouse models of autism exhibit more avoidance, hyperarousal, and aversive facial expressions (AFEs) to social touch compared with controls. Thus, this novel assay can be used to investigate behavioral responses to social touch and the underlying brain mechanisms in rodent models of neurodevelopmental conditions, and to evaluate therapeutic responses in preclinical studies.

3D mouse pose from single-view video and a new dataset

We present a method to infer the 3D pose of mice, including the limbs and feet, from monocular videos. Many human clinical conditions and their corresponding animal models result in abnormal motion, and accurately measuring 3D motion at scale offers insights into health. The 3D poses improve classification of health-related attributes over 2D representations. The inferred poses are accurate enough to estimate stride length even when the feet are mostly occluded. This method could be applied as part of a continuous monitoring system to non-invasively measure animal health, as demonstrated by its use in successfully classifying animals based on age and genotype. We introduce the Mouse Pose Analysis Dataset, the first large scale video dataset of lab mice in their home cage with ground truth keypoint and behavior labels. The dataset also contains high resolution mouse CT scans, which we use to build the shape models for 3D pose reconstruction.

A novel head-fixed assay for social touch in mice uncovers aversive responses in two autism models

Social touch, an important aspect of social interaction and communication, is essential to kinship across animal species. How animals experience and respond to social touch has not been thoroughly investigated, in part due to the lack of appropriate assays. Previous studies that examined social touch in freely moving rodents lacked the necessary temporal and spatial control over individual touch interactions. We designed a novel head-fixed assay for social touch in mice, in which the experimenter has complete control to elicit highly stereotyped bouts of social touch between two animals. The user determines the number, duration, context, and type of social touch interactions, while monitoring with high frame rate cameras an array of complex behavioral responses. We focused on social touch to the face because of their high translational relevance to humans. We validated this assay in two different models of autism spectrum disorder (ASD), the Fmr1 knockout model of Fragile X Syndrome and maternal immune activation mice. We observed increased avoidance, hyperarousal, and more aversive facial expressions to social touch, but not to object touch, in both ASD models compared to controls. Because this new social touch assay for head-fixed mice can be used to record neural activity during repeated bouts of social touch it should be of interest to neuroscientists interested in uncovering the underlying circuits.

Marker-less tracking system for multiple mice using Mask R-CNN

Although the appropriate evaluation of mouse behavior is crucial in pharmacological research, most current methods focus on single mouse behavior under light conditions, owing to the limitations of human observation and experimental tools. In this study, we aimed to develop a novel marker-less tracking method for multiple mice with top-view videos using deep-learning-based techniques. The following stepwise method was introduced: (i) detection of mouse contours, (ii) assignment of identifiers (IDs) to each mouse, and (iii) correction of mis-predictions. The behavior of C57BL/6 mice was recorded in an open-field arena, and the mouse contours were manually annotated for hundreds of frame images. Then, we trained the mask regional convolutional neural network (Mask R-CNN) with all annotated images. The mouse contours predicted by the trained model in each frame were assigned to IDs by calculating the similarities of every mouse pair between frames. After assigning IDs, correction steps were applied to remove the predictive errors semi-automatically. The established method could accurately predict two to four mice for first-look videos recorded under light conditions. The method could also be applied to videos recorded under dark conditions, extending our ability to accurately observe and analyze the sociality of nocturnal mice. This technology would enable a new approach to understand mouse sociality and advance the pharmacological research.