Quantifying behavior to understand the brain

Social experience alters oxytocinergic modulation in the nucleus accumbens of female prairie voles

Social relationships are dynamic and evolve with shared and personal experiences. Whether the functional role of social neuromodulators also evolves with experience to shape the trajectory of relationships is unknown. We utilized pair bonding in the socially monogamous prairie vole as an example of socio-sexual experience that dramatically alters behaviors displayed toward other individuals. We investigated oxytocin-dependent modulation of excitatory synaptic transmission in the nucleus accumbens as a function of pair-bonding status. We found that an oxytocin receptor agonist decreases the amplitude of spontaneous excitatory postsynaptic currents (sEPSCs) in sexually naive virgin, but not pair-bonded, female voles, while it increases the amplitude of electrically evoked EPSCs in paired voles, but not in virgins. This oxytocin-induced potentiation of synaptic transmission relies on the de novo coupling between oxytocin receptor signaling and endocannabinoid receptor type 1 (CB1) receptor signaling in pair-bonded voles. Blocking CB1 receptors after pair-bond formation increases the occurrence of a specific form of social rejection-defensive upright response-that is displayed toward the partner, but not toward a novel individual. Altogether, our results demonstrate that oxytocin's action in the nucleus accumbens is changed through social experience in a way that regulates the trajectory of social interactions as the relationship with the partner unfolds, potentially promoting the maintenance of a pair bond by inhibiting aggressive responses. These results provide a mechanism by which social experience and context shift oxytocinergic signaling to impact neural and behavioral responses to social cues.

Deep phenotyping reveals movement phenotypes in mouse neurodevelopmental models

BACKGROUND

Repetitive action, resistance to environmental change and fine motor disruptions are hallmarks of autism spectrum disorder (ASD) and other neurodevelopmental disorders, and vary considerably from individual to individual. In animal models, conventional behavioral phenotyping captures such fine-scale variations incompletely. Here we observed male and female C57BL/6J mice to methodically catalog adaptive movement over multiple days and examined two rodent models of developmental disorders against this dynamic baseline. We then investigated the behavioral consequences of a cerebellum-specific deletion in Tsc1 protein and a whole-brain knockout in Cntnap2 protein in mice. Both of these mutations are found in clinical conditions and have been associated with ASD.

METHODS

We used advances in computer vision and deep learning, namely a generalized form of high-dimensional statistical analysis, to develop a framework for characterizing mouse movement on multiple timescales using a single popular behavioral assay, the open-field test. The pipeline takes virtual markers from pose estimation to find behavior clusters and generate wavelet signatures of behavior classes. We measured spatial and temporal habituation to a new environment across minutes and days, different types of self-grooming, locomotion and gait.

RESULTS

Both Cntnap2 knockouts and L7-Tsc1 mutants showed forelimb lag during gait. L7-Tsc1 mutants and Cntnap2 knockouts showed complex defects in multi-day adaptation, lacking the tendency of wild-type mice to spend progressively more time in corners of the arena. In L7-Tsc1 mutant mice, failure to adapt took the form of maintained ambling, turning and locomotion, and an overall decrease in grooming. However, adaptation in these traits was similar between wild-type mice and Cntnap2 knockouts. L7-Tsc1 mutant and Cntnap2 knockout mouse models showed different patterns of behavioral state occupancy.

LIMITATIONS

Genetic risk factors for autism are numerous, and we tested only two. Our pipeline was only done under conditions of free behavior. Testing under task or social conditions would reveal more information about behavioral dynamics and variability.

CONCLUSIONS

Our automated pipeline for deep phenotyping successfully captures model-specific deviations in adaptation and movement as well as differences in the detailed structure of behavioral dynamics. The reported deficits indicate that deep phenotyping constitutes a robust set of ASD symptoms that may be considered for implementation in clinical settings as quantitative diagnosis criteria.

Siamese Network-Based All-Purpose-Tracker, a Model-Free Deep Learning Tool for Animal Behavioral Tracking

Accurate tracking is the basis of behavioral analysis, an important research method in neuroscience and many other fields. However, the currently available tracking methods have limitations. Traditional computer vision methods have problems in complex environments, and deep learning methods are hard to be applied universally due to the requirement of laborious annotations. To address the trade-off between accuracy and universality, we developed an easy-to-use tracking tool, Siamese Network-based All-Purpose Tracker (SNAP-Tracker), a model-free tracking software built on the Siamese network. The pretrained Siamese network offers SNAP-Tracker a remarkable feature extraction ability to keep tracking accuracy, and the model-free design makes it usable directly before laborious annotations and network refinement. SNAP-Tracker provides a “tracking with detection” mode to track longer videos with an additional detection module. We demonstrate the stability of SNAP-Tracker through different experimental conditions and different tracking tasks. In short, SNAP-Tracker provides a general solution to behavioral tracking without compromising accuracy. For the user’s convenience, we have integrated the tool into a tidy graphic user interface and opened the source code for downloading and using (https://github.com/slh0302/SNAP).

A dynamical systems view of neuroethology: Uncovering stateful computation in natural behaviors

State-dependent computation is key to cognition in both biological and artificial systems. Alan Turing recognized the power of stateful computation when he created the Turing machine with theoretically infinite computational capacity in 1936. Independently, by 1950, ethologists such as Tinbergen and Lorenz also began to implicitly embed rudimentary forms of state-dependent computation to create qualitative models of internal drives and naturally occurring animal behaviors. Here, we reformulate core ethological concepts in explicitly dynamical systems terms for stateful computation. We examine, based on a wealth of recent neural data collected during complex innate behaviors across species, the neural dynamics that determine the temporal structure of internal states. We will also discuss the degree to which the brain can be hierarchically partitioned into nested dynamical systems and the need for a multi-dimensional state-space model of the neuromodulatory system that underlies motivational and affective states.

Fast estimation of plant growth dynamics using deep neural networks

BACKGROUND

In recent years, there has been an increase of interest in plant behaviour as represented by growth-driven responses. These are generally classified into nastic (internally driven) and tropic (environmentally driven) movements. Nastic movements include circumnutations, a circular movement of plant organs commonly associated with search and exploration, while tropisms refer to the directed growth of plant organs toward or away from environmental stimuli, such as light and gravity. Tracking these movements is therefore fundamental for the study of plant behaviour. Convolutional neural networks, as used for human and animal pose estimation, offer an interesting avenue for plant tracking. Here we adopted the Social LEAP Estimates Animal Poses (SLEAP) framework for plant tracking. We evaluated it on time-lapse videos of cases spanning a variety of parameters, such as: (i) organ types and imaging angles (e.g., top-view crown leaves vs. side-view shoots and roots), (ii) lighting conditions (full spectrum vs. IR), (iii) plant morphologies and scales (100 μm-scale Arabidopsis seedlings vs. cm-scale sunflowers and beans), and (iv) movement types (circumnutations, tropisms and twining).

RESULTS

Overall, we found SLEAP to be accurate in tracking side views of shoots and roots, requiring only a low number of user-labelled frames for training. Top views of plant crowns made up of multiple leaves were found to be more challenging, due to the changing 2D morphology of leaves, and the occlusions of overlapping leaves. This required a larger number of labelled frames, and the choice of labelling "skeleton" had great impact on prediction accuracy, i.e., a more complex skeleton with fewer individuals (tracking individual plants) provided better results than a simpler skeleton with more individuals (tracking individual leaves).

CONCLUSIONS

In all, these results suggest SLEAP is a robust and versatile tool for high-throughput automated tracking of plants, presenting a new avenue for research focusing on plant dynamics.

Emerging Role of Translational Digital Biomarkers Within Home Cage Monitoring Technologies in Preclinical Drug Discovery and Development

In drug discovery and development, traditional assessment of human patients and preclinical subjects occurs at limited time points in potentially stressful surroundings (i.e., the clinic or a test arena), which can impact data quality and welfare. However, recent advances in remote digital monitoring technologies enable the assessment of human patients and preclinical subjects across multiple time points in familiar surroundings. The ability to monitor a patient throughout disease progression provides an opportunity for more relevant and efficient diagnosis as well as improved assessment of drug efficacy and safety. In preclinical in vivo animal models, these digital technologies allow for continuous, longitudinal, and non-invasive monitoring in the home environment. This manuscript provides an overview of digital monitoring technologies for use in preclinical studies including their history and evolution, current engagement through use cases, and impact of digital biomarkers (DBs) on drug discovery and the 3Rs. We also discuss barriers to implementation and strategies to overcome them. Finally, we address data consistency and technology standards from the perspective of technology providers, end-users, and subject matter experts. Overall, this review establishes an improved understanding of the value and implementation of digital biomarker (DB) technologies in preclinical research.

Neural Circuits Underlying Behavioral Flexibility: Insights From Drosophila

Behavioral flexibility is critical to survival. Animals must adapt their behavioral responses based on changes in the environmental context, internal state, or experience. Studies in Drosophila melanogaster have provided insight into the neural circuit mechanisms underlying behavioral flexibility. Here we discuss how Drosophila behavior is modulated by internal and behavioral state, environmental context, and learning. We describe general principles of neural circuit organization and modulation that underlie behavioral flexibility, principles that are likely to extend to other species.

A high-throughput method to deliver targeted optogenetic stimulation to moving C. elegans populations

We present a high-throughput optogenetic illumination system capable of simultaneous closed-loop light delivery to specified targets in populations of moving Caenorhabditis elegans. The instrument addresses three technical challenges: It delivers targeted illumination to specified regions of the animal's body such as its head or tail; it automatically delivers stimuli triggered upon the animal's behavior; and it achieves high throughput by targeting many animals simultaneously. The instrument was used to optogenetically probe the animal's behavioral response to competing mechanosensory stimuli in the the anterior and posterior gentle touch receptor neurons. Responses to more than 43,418 stimulus events from a range of anterior-posterior intensity combinations were measured. The animal's probability of sprinting forward in response to a mechanosensory stimulus depended on both the anterior and posterior stimulation intensity, while the probability of reversing depended primarily on the anterior stimulation intensity. We also probed the animal's response to mechanosensory stimulation during the onset of turning, a relatively rare behavioral event, by delivering stimuli automatically when the animal began to turn. Using this closed-loop approach, over 9,700 stimulus events were delivered during turning onset at a rate of 9.2 events per worm hour, a greater than 25-fold increase in throughput compared to previous investigations. These measurements validate with greater statistical power previous findings that turning acts to gate mechanosensory evoked reversals. Compared to previous approaches, the current system offers targeted optogenetic stimulation to specific body regions or behaviors with many fold increases in throughput to better constrain quantitative models of sensorimotor processing.

Looking for the neural basis of memory

Memory neuroscientists often measure neural activity during task trials designed to recruit specific memory processes. Behavior is championed as crucial for deciphering brain-memory linkages but is impoverished in typical experiments that rely on summary judgments. We criticize this approach as being blind to the multiple cognitive, neural, and behavioral processes that occur rapidly within a trial to support memory. Instead, time-resolved behaviors such as eye movements occur at the speed of cognition and neural activity. We highlight successes using eye-movement tracking with in vivo electrophysiology to link rapid hippocampal oscillations to encoding and retrieval processes that interact over hundreds of milliseconds. This approach will improve research on the neural basis of memory because it pinpoints discrete moments of brain-behavior-cognition correspondence.

Bubblewrap: Online tiling and real-time flow prediction on neural manifolds

While most classic studies of function in experimental neuroscience have focused on the coding properties of individual neurons, recent developments in recording technologies have resulted in an increasing emphasis on the dynamics of neural populations. This has given rise to a wide variety of models for analyzing population activity in relation to experimental variables, but direct testing of many neural population hypotheses requires intervening in the system based on current neural state, necessitating models capable of inferring neural state online. Existing approaches, primarily based on dynamical systems, require strong parametric assumptions that are easily violated in the noise-dominated regime and do not scale well to the thousands of data channels in modern experiments. To address this problem, we propose a method that combines fast, stable dimensionality reduction with a soft tiling of the resulting neural manifold, allowing dynamics to be approximated as a probability flow between tiles. This method can be fit efficiently using online expectation maximization, scales to tens of thousands of tiles, and outperforms existing methods when dynamics are noise-dominated or feature multi-modal transition probabilities. The resulting model can be trained at kiloHertz data rates, produces accurate approximations of neural dynamics within minutes, and generates predictions on submillisecond time scales. It retains predictive performance throughout many time steps into the future and is fast enough to serve as a component of closed-loop causal experiments.

The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

Multi-agent behavior modeling aims to understand the interactions that occur between agents. We present a multi-agent dataset from behavioral neuroscience, the Caltech Mouse Social Interactions (CalMS21) Dataset. Our dataset consists of trajectory data of social interactions, recorded from videos of freely behaving mice in a standard resident-intruder assay. To help accelerate behavioral studies, the CalMS21 dataset provides benchmarks to evaluate the performance of automated behavior classification methods in three settings: (1) for training on large behavioral datasets all annotated by a single annotator, (2) for style transfer to learn inter-annotator differences in behavior definitions, and (3) for learning of new behaviors of interest given limited training data. The dataset consists of 6 million frames of unlabeled tracked poses of interacting mice, as well as over 1 million frames with tracked poses and corresponding frame-level behavior annotations. The challenge of our dataset is to be able to classify behaviors accurately using both labeled and unlabeled tracking data, as well as being able to generalize to new settings.

The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice

The study of naturalistic social behavior requires quantification of animals' interactions. This is generally done through manual annotation-a highly time-consuming and tedious process. Recent advances in computer vision enable tracking the pose (posture) of freely behaving animals. However, automatically and accurately classifying complex social behaviors remains technically challenging. We introduce the Mouse Action Recognition System (MARS), an automated pipeline for pose estimation and behavior quantification in pairs of freely interacting mice. We compare MARS's annotations to human annotations and find that MARS's pose estimation and behavior classification achieve human-level performance. We also release the pose and annotation datasets used to train MARS to serve as community benchmarks and resources. Finally, we introduce the Behavior Ensemble and Neural Trajectory Observatory (BENTO), a graphical user interface for analysis of multimodal neuroscience datasets. Together, MARS and BENTO provide an end-to-end pipeline for behavior data extraction and analysis in a package that is user-friendly and easily modifiable.

Modelling behaviors relevant to brain disorders in the nonhuman primate: Are we there yet?

Recent years have seen a profound resurgence of activity with nonhuman primates (NHPs) to model human brain disorders. From marmosets to macaques, the study of NHP species offers a unique window into the function of primate-specific neural circuits that are impossible to examine in other models. Examining how these circuits manifest into the complex behaviors of primates, such as advanced cognitive and social functions, has provided enormous insights to date into the mechanisms underlying symptoms of numerous neurological and neuropsychiatric illnesses. With the recent optimization of modern techniques to manipulate and measure neural activity in vivo, such as optogenetics and calcium imaging, NHP research is more well-equipped than ever to probe the neural mechanisms underlying pathological behavior. However, methods for behavioral experimentation and analysis in NHPs have noticeably failed to keep pace with these advances. As behavior ultimately lies at the junction between preclinical findings and its translation to clinical outcomes for brain disorders, approaches to improve the integrity, reproducibility, and translatability of behavioral experiments in NHPs requires critical evaluation. In this review, we provide a unifying account of existing brain disorder models using NHPs, and provide insights into the present and emerging contributions of behavioral studies to the field.

Miniature microscopes for manipulating and recording in vivo brain activity

Here we describe the development and application of miniature integrated microscopes (miniscopes) paired with microendoscopes that allow for the visualization and manipulation of neural circuits in superficial and subcortical brain regions in freely behaving animals. Over the past decade the miniscope platform has expanded to include simultaneous optogenetic capabilities, electrically-tunable lenses that enable multi-plane imaging, color-corrected optics, and an integrated data acquisition platform that streamlines multimodal experiments. Miniscopes have given researchers an unprecedented ability to monitor hundreds to thousands of genetically-defined neurons from weeks to months in both healthy and diseased animal brains. Sophisticated algorithms that take advantage of constrained matrix factorization allow for background estimation and reliable cell identification, greatly improving the reliability and scalability of source extraction for large imaging datasets. Data generated from miniscopes have empowered researchers to investigate the neural circuit underpinnings of a wide array of behaviors that cannot be studied under head-fixed conditions, such as sleep, reward seeking, learning and memory, social behaviors, and feeding. Importantly, the miniscope has broadened our understanding of how neural circuits can go awry in animal models of progressive neurological disorders, such as Parkinson's disease. Continued miniscope development, including the ability to record from multiple populations of cells simultaneously, along with continued multimodal integration of techniques such as electrophysiology, will allow for deeper understanding into the neural circuits that underlie complex and naturalistic behavior.

An atlas for unveiling the functional organization of spinal circuits

Defining the positional organization of neurons in the spinal cord is critical for understanding their function. In this issue, Fiederling and colleagues present a method to accurately map position and connectivity of neurons in a universal three-dimensional spinal cord reference atlas.

Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain

Neural computation has evolved to optimize the behaviors that enable our survival. Although much previous work in neuroscience has focused on constrained task behaviors, recent advances in computer vision are fueling a trend toward the study of naturalistic behaviors. Automated tracking of fine-scale behaviors is generating rich datasets for animal models including rodents, fruit flies, zebrafish, and worms. However, extracting meaning from these large and complex data often requires sophisticated computational techniques. Here we review the latest methods and modeling approaches providing new insights into the brain from behavior. We focus on unsupervised methods for identifying stereotyped behaviors and for resolving details of the structure and dynamics of behavioral sequences.

Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders

Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful information from video data. Here we introduce a new video analysis tool that combines the output of supervised pose estimation algorithms (e.g. DeepLabCut) with unsupervised dimensionality reduction methods to produce interpretable, low-dimensional representations of behavioral videos that extract more information than pose estimates alone. We demonstrate this tool by extracting interpretable behavioral features from videos of three different head-fixed mouse preparations, as well as a freely moving mouse in an open field arena, and show how these interpretable features can facilitate downstream behavioral and neural analyses. We also show how the behavioral features produced by our model improve the precision and interpretation of these downstream analyses compared to using the outputs of either fully supervised or fully unsupervised methods alone.

Collective action or individual choice: Spontaneity and individuality contribute to decision-making in Drosophila

Our own unique character traits make our behavior consistent and define our individuality. Yet, this consistency does not entail that we behave repetitively like machines. Like humans, animals also combine personality traits with spontaneity to produce adaptive behavior: consistent, but not fully predictable. Here, we study an iconically rigid behavioral trait, insect phototaxis, that nevertheless also contains both components of individuality and spontaneity. In a light/dark T-maze, approximately 70% of a group of Drosophila fruit flies choose the bright arm of the T-Maze, while the remaining 30% walk into the dark. Taking the photopositive and the photonegative subgroups and re-testing them reveals the spontaneous component: a similar 70–30 distribution emerges in each of the two subgroups. Increasing the number of choices to ten choices, reveals the individuality component: flies with an extremely negative series of first choices were more likely to show photonegative behavior in subsequent choices and vice versa. General behavioral traits, independent of light/dark preference, contributed to the development of this individuality. The interaction of individuality and spontaneity together explains why group averages, even for such seemingly stereotypical behaviors, are poor predictors of individual choices.

The future of bone regeneration: integrating AI into tissue engineering

Tissue engineering is a branch of regenerative medicine that harnesses biomaterial and stem cell research to utilise the body's natural healing responses to regenerate tissue and organs. There remain many unanswered questions in tissue engineering, with optimal biomaterial designs still to be developed and a lack of adequate stem cell knowledge limiting successful application. Advances in artificial intelligence (AI), and deep learning specifically, offer the potential to improve both scientific understanding and clinical outcomes in regenerative medicine. With enhanced perception of how to integrate artificial intelligence into current research and clinical practice, AI offers an invaluable tool to improve patient outcome.

Systematic analysis of goal-related movement sequences during maternal behaviour in a female mouse model for Rett syndrome

Rodent dams seek and gather scattered pups back to the nest (pup retrieval), an essential aspect of maternal care. Systematic analysis of the dynamic sequences of goal-related movements that comprise the entire behavioural sequence, which would be ultimately essential for understanding the underlying neurobiology, is not well-characterized. Here, we present such analysis across 3 days in alloparental female mice (Surrogates or Sur) of two genotypes; Mecp2Heterozygotes (Het), a female mouse model for Rett syndrome and their wild type (WT) siblings. We analysed CBA/CaJ and C57BL/6J WT surrogates for within-strain comparisons. Frame-by-frame analysis over different phases was performed manually using DataVyu software. We previously showed that surrogate Het are inefficient at pup retrieval, by end-point analysis such as latency index and errors. Here, the sequence of searching, pup-approach and successful retrieval streamlines over days for WT, while Het exhibits variations in this pattern. Goal-related movements between Het and WT are similar in other phases, suggesting context-driven atypical patterns in Het during the pup retrieval phase. We identified proximal pup approach and pup grooming as atypical tactile interactions between pups and Het. Day-by-day analysis showed dynamic changes in goal-related movements in individual animals across genotypes and strains. Overall, our approach (1) highlights natural variation in individual mice on different days, (2) establishes a "gold-standard" manually curated dataset to help build behavioural repertoires using machine learning approaches, and (3) suggests atypical tactile sensory processing and possible regression in a female mouse model for Rett syndrome.

Expressions of emotions across species

What are emotions and how should we study them? These questions give rise to ongoing controversy amongst scientists in the fields of neuroscience, psychology and philosophy, and have resulted in different views on emotions [1-6]. In this review, we define emotions as functional states that bear essential roles in promoting survival and thus have emerged through evolution. Emotions trigger behavioral, somatic, hormonal, and neurochemical reactions, referred to as expressions of emotion. We discuss recent studies on emotion expression across species and highlight emerging common principles. We argue that detailed and multidimensional analyses of emotion expressions are key to develop biology-based definitions of emotions and to reveal their neuronal underpinnings.

Analysis of ultrasonic vocalizations from mice using computer vision and machine learning

Mice emit ultrasonic vocalizations (USVs) that communicate socially relevant information. To detect and classify these USVs, here we describe VocalMat. VocalMat is a software that uses image-processing and differential geometry approaches to detect USVs in audio files, eliminating the need for user-defined parameters. VocalMat also uses computational vision and machine learning methods to classify USVs into distinct categories. In a data set of >4000 USVs emitted by mice, VocalMat detected over 98% of manually labeled USVs and accurately classified ≈86% of the USVs out of 11 USV categories. We then used dimensionality reduction tools to analyze the probability distribution of USV classification among different experimental groups, providing a robust method to quantify and qualify the vocal repertoire of mice. Thus, VocalMat makes it possible to perform automated, accurate, and quantitative analysis of USVs without the need for user inputs, opening the opportunity for detailed and high-throughput analysis of this behavior.

Neuroscience needs behavior: inferring psychophysical strategy trial by trial

Tools quantifying dynamic behavior are important to understand brain function. In this issue of Neuron, Roy et al. (2021) extended the available repertoire with PsyTrack, which tracks, trial by trial, how subjects performing psychophysical tasks adjust the way they weigh stimuli and task covariates or change their biases.