Mapping the stereotyped behaviour of freely moving fruit flies

Mapping the developmental structure of stereotyped and individual-unique behavioral spaces in C. elegans

Developmental patterns of behavior are variably organized in time and among different individuals. However, long-term behavioral diversity was previously studied using pre-defined behavioral parameters, representing a limited fraction of the full individuality structure. Here, we continuously extract ∼1.2 billion body postures of ∼2,200 single C. elegans individuals throughout their full development time to create a complete developmental atlas of stereotyped and individual-unique behavioral spaces. Unsupervised inference of low-dimensional movement modes of each single individual identifies a dynamic developmental trajectory of stereotyped behavioral spaces and exposes unique behavioral trajectories of individuals that deviate from the stereotyped patterns. Moreover, classification of behavioral spaces within tens of neuromodulatory and environmentally perturbed populations shows plasticity in the temporal structures of stereotyped behavior and individuality. These results present a comprehensive atlas of continuous behavioral dynamics across development time and a general framework for unsupervised dissection of shared and unique developmental signatures of behavior.

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface

Objective

Creating an intracortical brain-computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue.

Approach

We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to.

Main results

We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network (RNN) decoder with 10-12 clusters.

Significance

This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

Unsupervised decomposition of natural monkey behavior into a sequence of motion motifs

Nonhuman primates (NHPs) exhibit complex and diverse behavior that typifies advanced cognitive function and social communication, but quantitative and systematical measure of this natural nonverbal processing has been a technical challenge. Specifically, a method is required to automatically segment time series of behavior into elemental motion motifs, much like finding meaningful words in character strings. Here, we propose a solution called SyntacticMotionParser (SMP), a general-purpose unsupervised behavior parsing algorithm using a nonparametric Bayesian model. Using three-dimensional posture-tracking data from NHPs, SMP automatically outputs an optimized sequence of latent motion motifs classified into the most likely number of states. When applied to behavioral datasets from common marmosets and rhesus monkeys, SMP outperformed conventional posture-clustering models and detected a set of behavioral ethograms from publicly available data. SMP also quantified and visualized the behavioral effects of chemogenetic neural manipulations. SMP thus has the potential to dramatically improve our understanding of natural NHP behavior in a variety of contexts.

A Markovian dynamics for Caenorhabditis elegans behavior across scales

How do we capture the breadth of behavior in animal movement, from rapid body twitches to aging? Using high-resolution videos of the nematode worm Caenorhabditis elegans, we show that a single dynamics connects posture-scale fluctuations with trajectory diffusion and longer-lived behavioral states. We take short posture sequences as an instantaneous behavioral measure, fixing the sequence length for maximal prediction. Within the space of posture sequences, we construct a fine-scale, maximum entropy partition so that transitions among microstates define a high-fidelity Markov model, which we also use as a means of principled coarse-graining. We translate these dynamics into movement using resistive force theory, capturing the statistical properties of foraging trajectories. Predictive across scales, we leverage the longest-lived eigenvectors of the inferred Markov chain to perform a top-down subdivision of the worm's foraging behavior, revealing both "runs-and-pirouettes" as well as previously uncharacterized finer-scale behaviors. We use our model to investigate the relevance of these fine-scale behaviors for foraging success, recovering a trade-off between local and global search strategies.

A virtual rodent predicts the structure of neural activity across behaviours

Animals have exquisite control of their bodies, allowing them to perform a diverse range of behaviours. How such control is implemented by the brain, however, remains unclear. Advancing our understanding requires models that can relate principles of control to the structure of neural activity in behaving animals. Here, to facilitate this, we built a 'virtual rodent', in which an artificial neural network actuates a biomechanically realistic model of the rat1 in a physics simulator2. We used deep reinforcement learning3-5 to train the virtual agent to imitate the behaviour of freely moving rats, thus allowing us to compare neural activity recorded in real rats to the network activity of a virtual rodent mimicking their behaviour. We found that neural activity in the sensorimotor striatum and motor cortex was better predicted by the virtual rodent's network activity than by any features of the real rat's movements, consistent with both regions implementing inverse dynamics6. Furthermore, the network's latent variability predicted the structure of neural variability across behaviours and afforded robustness in a way consistent with the minimal intervention principle of optimal feedback control7. These results demonstrate how physical simulation of biomechanically realistic virtual animals can help interpret the structure of neural activity across behaviour and relate it to theoretical principles of motor control.

A study of animal action segmentation algorithms across supervised, unsupervised, and semi-supervised learning paradigms

Action segmentation of behavioral videos is the process of labeling each frame as belonging to one or more discrete classes, and is a crucial component of many studies that investigate animal behavior. A wide range of algorithms exist to automatically parse discrete animal behavior, encompassing supervised, unsupervised, and semi-supervised learning paradigms. These algorithms - which include tree-based models, deep neural networks, and graphical models - differ widely in their structure and assumptions on the data. Using four datasets spanning multiple species - fly, mouse, and human - we systematically study how the outputs of these various algorithms align with manually annotated behaviors of interest. Along the way, we introduce a semi-supervised action segmentation model that bridges the gap between supervised deep neural networks and unsupervised graphical models. We find that fully supervised temporal convolutional networks with the addition of temporal information in the observations perform the best on our supervised metrics across all datasets.

Exploring motion using geometric morphometrics in microscopic aquatic invertebrates: 'modes' and movement patterns during feeding in a bdelloid rotifer model species

BACKGROUND

Movement is a defining aspect of animals, but it is rarely studied using quantitative methods in microscopic invertebrates. Bdelloid rotifers are a cosmopolitan class of aquatic invertebrates of great scientific interest because of their ability to survive in very harsh environment and also because they represent a rare example of an ancient lineage that only includes asexually reproducing species. In this class, Adineta ricciae has become a model species as it is unusually easy to culture. Yet, relatively little is known of its ethology and almost nothing on how it behaves during feeding.

METHODS

To explore feeding behaviour in A. ricciae, as well as to provide an example of application of computational ethology in a microscopic invertebrate, we apply Procrustes motion analysis in combination with ordination and clustering methods to a laboratory bred sample of individuals recorded during feeding.

RESULTS

We demonstrate that movement during feeding can be accurately described in a simple two-dimensional shape space with three main 'modes' of motion. Foot telescoping, with the body kept straight, is the most frequent 'mode', but it is accompanied by periodic rotations of the foot together with bending while the foot is mostly retracted.

CONCLUSIONS

Procrustes motion analysis is a relatively simple but effective tool for describing motion during feeding in A. ricciae. The application of this method generates quantitative data that could be analysed in relation to genetic and ecological differences in a variety of experimental settings. The study provides an example that is easy to replicate in other invertebrates, including other microscopic animals whose behavioural ecology is often poorly known.

Bridging model and experiment in systems neuroscience with Cleo: the Closed-Loop, Electrophysiology, and Optophysiology simulation testbed

Systems neuroscience has experienced an explosion of new tools for reading and writing neural activity, enabling exciting new experiments such as all-optical or closed-loop control that effect powerful causal interventions. At the same time, improved computational models are capable of reproducing behavior and neural activity with increasing fidelity. Unfortunately, these advances have drastically increased the complexity of integrating different lines of research, resulting in the missed opportunities and untapped potential of suboptimal experiments. Experiment simulation can help bridge this gap, allowing model and experiment to better inform each other by providing a low-cost testbed for experiment design, model validation, and methods engineering. Specifically, this can be achieved by incorporating the simulation of the experimental interface into our models, but no existing tool integrates optogenetics, two-photon calcium imaging, electrode recording, and flexible closed-loop processing with neural population simulations. To address this need, we have developed Cleo: the Closed-Loop, Electrophysiology, and Optophysiology experiment simulation testbed. Cleo is a Python package enabling injection of recording and stimulation devices as well as closed-loop control with realistic latency into a Brian spiking neural network model. It is the only publicly available tool currently supporting two-photon and multi-opsin/wavelength optogenetics. To facilitate adoption and extension by the community, Cleo is open-source, modular, tested, and documented, and can export results to various data formats. Here we describe the design and features of Cleo, validate output of individual components and integrated experiments, and demonstrate its utility for advancing optogenetic techniques in prospective experiments using previously published systems neuroscience models.

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices

Objective.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models.Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.

Heightened lateral habenula activity during stress produces brainwide and behavioral substrates of susceptibility

Some individuals are susceptible to the experience of chronic stress and others are more resilient. While many brain regions implicated in learning are dysregulated after stress, little is known about whether and how neural teaching signals during stress differ between susceptible and resilient individuals. Here, we seek to determine if activity in the lateral habenula (LHb), which encodes a negative teaching signal, differs between susceptible and resilient mice during stress to produce different outcomes. After, but not before, chronic social defeat stress (CSDS), the LHb is active when susceptible mice are in the proximity of the aggressor strain. During stress itself, LHb activity is higher in susceptible mice during aggressor proximity, and activation of the LHb during stress biases mice towards susceptibility. This manipulation generates a persistent and widespread increase in the balance of subcortical versus cortical activity in susceptible mice. Taken together, our results indicate that heightened activity in the LHb during stress produces lasting brainwide and behavioral substrates of susceptibility.

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ('syllables') from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce 'Lightning Pose', an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We released a cloud application that allows users to label data, train networks and process new videos directly from the browser.

Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience

The study of complex behaviors is often challenging when using manual annotation due to the absence of quantifiable behavioral definitions and the subjective nature of behavioral annotation. Integration of supervised machine learning approaches mitigates some of these issues through the inclusion of accessible and explainable model interpretation. To decrease barriers to access, and with an emphasis on accessible model explainability, we developed the open-source Simple Behavioral Analysis (SimBA) platform for behavioral neuroscientists. SimBA introduces several machine learning interpretability tools, including SHapley Additive exPlanation (SHAP) scores, that aid in creating explainable and transparent behavioral classifiers. Here we show how the addition of explainability metrics allows for quantifiable comparisons of aggressive social behavior across research groups and species, reconceptualizing behavior as a sharable reagent and providing an open-source framework. We provide an open-source, graphical user interface (GUI)-driven, well-documented package to facilitate the movement toward improved automation and sharing of behavioral classification tools across laboratories.

Temporally aligned segmentation and clustering (TASC) framework for behavior time series analysis

Behavior exhibits a complex spatiotemporal structure consisting of discrete sub-behaviors, or motifs. Continuous behavior data requires segmentation and clustering to reveal these embedded motifs. The popularity of automatic behavior quantification is growing, but existing solutions are often tailored to specific needs and are not designed for the time scale and precision required in many experimental and clinical settings. Here we propose a generalized framework with an iterative approach to refine both segmentation and clustering. Temporally aligned segmentation and clustering (TASC) uses temporal linear alignment to compute distances between and align the recurring behavior motifs in a multidimensional time series, enabling precise segmentation and clustering. We introduce an alternating-step process: evaluation of temporal neighbors against current cluster centroids using linear alignment, alternating with selecting the best non-overlapping segments and their subsequent re-clustering. The framework is evaluated on semi-synthetic and real-world experimental and clinical data, demonstrating enhanced segmentation and clustering, offering a better foundation for consequent research. The framework may be used to extend existing tools in the field of behavior research and may be applied to other domains requiring high precision of time series segmentation.

Characterizing the structure of mouse behavior using Motion Sequencing

Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.

FlyVISTA, an Integrated Machine Learning Platform for Deep Phenotyping of Sleep in Drosophila

Animal behavior depends on internal state. While subtle movements can signify significant changes in internal state, computational methods for analyzing these "microbehaviors" are lacking. Here, we present FlyVISTA, a machine-learning platform to characterize microbehaviors in freely-moving flies, which we use to perform deep phenotyping of sleep. This platform comprises a high-resolution closed-loop video imaging system, coupled with a deep-learning network to annotate 35 body parts, and a computational pipeline to extract behaviors from high-dimensional data. FlyVISTA reveals the distinct spatiotemporal dynamics of sleep-associated microbehaviors in flies. We further show that stimulation of dorsal fan-shaped body neurons induces micromovements, not sleep, whereas activating R5 ring neurons triggers rhythmic proboscis extension followed by persistent sleep. Importantly, we identify a novel microbehavior ("haltere switch") exclusively seen during quiescence that indicates a deeper sleep stage. These findings enable the rigorous analysis of sleep in Drosophila and set the stage for computational analyses of microbehaviors.

Reconfiguration of Behavioral Signals in the Anterior Cingulate Cortex Based on Emotional State

Behaviors and their execution depend on the context and emotional state in which they are performed. The contextual modulation of behavior likely relies on regions such as the anterior cingulate cortex (ACC) that multiplex information about emotional/autonomic states and behaviors. The objective of the present study was to understand how the representations of behaviors by ACC neurons become modified when performed in different emotional states. A pipeline of machine learning techniques was developed to categorize and classify complex, spontaneous behaviors in male rats from the video. This pipeline, termed Hierarchical Unsupervised Behavioural Discovery Tool (HUB-DT), discovered a range of statistically separable behaviors during a task in which motivationally significant outcomes were delivered in blocks of trials that created three unique "emotional contexts." HUB-DT was capable of detecting behaviors specific to each emotional context and was able to identify and segregate the portions of a neural signal related to a behavior and to emotional context. Overall, ∼10× as many neurons responded to behaviors in a contextually dependent versus a fixed manner, highlighting the extreme impact of emotional state on representations of behaviors that were precisely defined based on detailed analyses of limb kinematics. This type of modulation may be a key mechanism that allows the ACC to modify the behavioral output based on emotional states and contextual demands.

Visual detection of seizures in mice using supervised machine learning

Seizures are caused by abnormally synchronous brain activity that can result in changes in muscle tone, such as twitching, stiffness, limpness, or rhythmic jerking. These behavioral manifestations are clear on visual inspection and the most widely used seizure scoring systems in preclinical models, such as the Racine scale in rodents, use these behavioral patterns in semiquantitative seizure intensity scores. However, visual inspection is time-consuming, low-throughput, and partially subjective, and there is a need for rigorously quantitative approaches that are scalable. In this study, we used supervised machine learning approaches to develop automated classifiers to predict seizure severity directly from noninvasive video data. Using the PTZ-induced seizure model in mice, we trained video-only classifiers to predict ictal events, combined these events to predict an univariate seizure intensity for a recording session, as well as time-varying seizure intensity scores. Our results show, for the first time, that seizure events and overall intensity can be rigorously quantified directly from overhead video of mice in a standard open field using supervised approaches. These results enable high-throughput, noninvasive, and standardized seizure scoring for downstream applications such as neurogenetics and therapeutic discovery.

Exploratory Rearing Is Governed by Hypothalamic Melanin-Concentrating Hormone Neurons According to Locus Ceruleus

Information seeking, such as standing on tiptoes to look around in humans, is observed across animals and helps survival. Its rodent analog-unsupported rearing on hind legs-was a classic model in deciphering neural signals of cognition and is of intense renewed interest in preclinical modeling of neuropsychiatric states. Neural signals and circuits controlling this dedicated decision to seek information remain largely unknown. While studying subsecond timing of spontaneous behavioral acts and activity of melanin-concentrating hormone (MCH) neurons (MNs) in behaving male and female mice, we observed large MN activity spikes that aligned to unsupported rears. Complementary causal, loss and gain of function, analyses revealed specific control of rear frequency and duration by MNs and MCHR1 receptors. Activity in a key stress center of the brain-the locus ceruleus noradrenaline cells-rapidly inhibited MNs and required functional MCH receptors for its endogenous modulation of rearing. By defining a neural module that both tracks and controls rearing, these findings may facilitate further insights into biology of information seeking.

STPoseNet: A real-time spatiotemporal network model for robust mouse pose estimation

Animal behavior analysis plays a crucial role in contemporary neuroscience research. However, the performance of the frame-by-frame approach may degrade in scenarios with occlusions or motion blur. In this study, we propose a spatiotemporal network model based on YOLOv8 to enhance the accuracy of key-point detection in mouse behavioral experimental videos. This model integrates a time-domain tracking strategy comprising two components: the first part utilizes key-point detection results from the previous frame to detect potential target locations in the subsequent frame; the second part employs Kalman filtering to analyze key-point changes prior to detection, allowing for the estimation of missing key-points. In the comparison of pose estimation results between our approach, YOLOv8, DeepLabCut and SLEAP on videos of three mouse behavioral experiments, our approach demonstrated significantly superior performance. This suggests that our method offers a new and effective means of accurately tracking and estimating pose in mice through spatiotemporal processing.

SAUSI: a novel assay for measuring social anxiety and motivation

Social anxiety is one of the most prevalent mental health disorders, though the underlying neurobiology is poorly understood. Progress in understanding the etiology of social anxiety has been hindered by the lack of comprehensive tools to assess social anxiety in model systems. Here, we created a new behavioral task - Selective Access to Unrestricted Social Interaction (SAUSI), which combines elements of social motivation, hesitancy, decision-making, and free interaction to enable the wholistic assessment of social anxiety-like behaviors in mice. Using this novel assay, we found that social isolation-induced social anxiety-like behaviors in female mice are largely driven by increases in social fear, social hesitancy, and altered ultrasonic vocalizations. Deep learning analyses were able to computationally identify a unique behavioral footprint underlying the state produced by social isolation, demonstrating the compatibility of modern computational approaches with SAUSI. Finally, we compared the results of SAUSI to traditionally social assays including the 3-chamber sociability assay and the resident intruder task. This revealed that behavioral changes induced by isolation were highly context dependent, and that while fragments of social anxiety measured in SAUSI were replicable across other tasks, a wholistic assessment was not obtainable from these alternative assays. Our findings debut a novel task for the behavioral toolbox - one which overcomes limitations of previous assays, allowing for both social choice as well as free interaction, and offers a new approach for assessing social anxiety in rodents.

Learning dynamical models of single and collective cell migration: a review

Single and collective cell migration are fundamental processes critical for physiological phenomena ranging from embryonic development and immune response to wound healing and cancer metastasis. To understand cell migration from a physical perspective, a broad variety of models for the underlying physical mechanisms that govern cell motility have been developed. A key challenge in the development of such models is how to connect them to experimental observations, which often exhibit complex stochastic behaviours. In this review, we discuss recent advances in data-driven theoretical approaches that directly connect with experimental data to infer dynamical models of stochastic cell migration. Leveraging advances in nanofabrication, image analysis, and tracking technology, experimental studies now provide unprecedented large datasets on cellular dynamics. In parallel, theoretical efforts have been directed towards integrating such datasets into physical models from the single cell to the tissue scale with the aim of conceptualising the emergent behaviour of cells. We first review how this inference problem has been addressed in both freely migrating and confined cells. Next, we discuss why these dynamics typically take the form of underdamped stochastic equations of motion, and how such equations can be inferred from data. We then review applications of data-driven inference and machine learning approaches to heterogeneity in cell behaviour, subcellular degrees of freedom, and to the collective dynamics of multicellular systems. Across these applications, we emphasise how data-driven methods can be integrated with physical active matter models of migrating cells, and help reveal how underlying molecular mechanisms control cell behaviour. Together, these data-driven approaches are a promising avenue for building physical models of cell migration directly from experimental data, and for providing conceptual links between different length-scales of description.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce "Lightning Pose," an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry, and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post-hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We release a cloud application that allows users to label data, train networks, and predict new videos directly from the browser.

Characterization of Ultrasonic Vocalization-Modulated Neurons in Rat Motor Cortex Based on Their Activity Modulation and Axonal Projection to the Periaqueductal Gray

Vocalization, a means of social communication, is prevalent among many species, including humans. Both rats and mice use ultrasonic vocalizations (USVs) in various social contexts and affective states. The motor cortex is hypothesized to be involved in precisely controlling USVs through connections with critical regions of the brain for vocalization, such as the periaqueductal gray matter (PAG). However, it is unclear how neurons in the motor cortex are modulated during USVs. Moreover, the relationship between USV modulation of neurons and anatomical connections from the motor cortex to PAG is also not clearly understood. In this study, we first characterized the activity patterns of neurons in the primary and secondary motor cortices during emission of USVs in rats using large-scale electrophysiological recordings. We also examined the axonal projection of the motor cortex to PAG using retrograde labeling and identified two clusters of PAG-projecting neurons in the anterior and posterior parts of the motor cortex. The neural activity patterns around the emission of USVs differed between the anterior and posterior regions, which were divided based on the distribution of PAG-projecting neurons in the motor cortex. Furthermore, using optogenetic tagging, we recorded the USV modulation of PAG-projecting neurons in the posterior part of the motor cortex and found that they showed predominantly sustained excitatory responses during USVs. These results contribute to our understanding of the involvement of the motor cortex in the generation of USV at the neuronal and circuit levels.

A-SOiD, an active-learning platform for expert-guided, data-efficient discovery of behavior

To identify and extract naturalistic behavior, two methods have become popular: supervised and unsupervised. Each approach carries its own strengths and weaknesses (for example, user bias, training cost, complexity and action discovery), which the user must consider in their decision. Here, an active-learning platform, A-SOiD, blends these strengths, and in doing so, overcomes several of their inherent drawbacks. A-SOiD iteratively learns user-defined groups with a fraction of the usual training data, while attaining expansive classification through directed unsupervised classification. In socially interacting mice, A-SOiD outperformed standard methods despite requiring 85% less training data. Additionally, it isolated ethologically distinct mouse interactions via unsupervised classification. We observed similar performance and efficiency using nonhuman primate and human three-dimensional pose data. In both cases, the transparency in A-SOiD's cluster definitions revealed the defining features of the supervised classification through a game-theoretic approach. To facilitate use, A-SOiD comes as an intuitive, open-source interface for efficient segmentation of user-defined behaviors and discovered sub-actions.

Arbovirus impact on mosquito behavior: the jury is still out

Parasites can manipulate host behavior to enhance transmission, but our understanding of arbovirus-induced changes in mosquito behavior is limited. Here, we explore current knowledge on such behavioral alterations in mosquito vectors, focusing on host-seeking and blood-feeding behaviors. Reviewing studies on dengue, Zika, La Crosse, Sindbis, and West Nile viruses in Aedes or Culex mosquitoes reveals subtle yet potentially significant effects. However, assay heterogeneity and limited sample sizes challenge definitive conclusions. To enhance robustness, we propose using deep-learning tools for automated behavior quantification and stress the need for standardized assays. Additionally, conducting longitudinal studies across the extrinsic incubation period and integrating diverse traits into modeling frameworks are crucial for understanding the nuanced implications of arbovirus-induced behavioral changes for virus transmission dynamics.

An inhibitory acetylcholine receptor gates context dependent mechanosensory processing in C. elegans

An animal's current behavior influences its response to sensory stimuli, but the molecular and circuit-level mechanisms of this context-dependent decision-making is not well understood. In the nematode C. elegans, inhibitory feedback from turning associated neurons alter downstream mechanosensory processing to gate the animal's response to stimuli depending on whether the animal is turning or moving forward [1-3]. Until now, the specific neurons and receptors that mediate this inhibitory feedback were not known. We use genetic manipulations, single-cell rescue experiments and high-throughput closed-loop optogenetic perturbations during behavior to reveal the specific neuron and receptor responsible for receiving inhibition and altering sensorimotor processing. An inhibitory acetylcholine gated chloride channel comprised of lgc-47 and acc-1 expressed in neuron RIM receives inhibitory signals from turning neurons and performs the gating that disrupts the worm's mechanosensory evoked reversal response.

Multidimensional Analysis of a Social Behavior Identifies Regression and Phenotypic Heterogeneity in a Female Mouse Model for Rett Syndrome

Regression is a key feature of neurodevelopmental disorders such as autism spectrum disorder, Fragile X syndrome, and Rett syndrome (RTT). RTT is caused by mutations in the X-linked gene methyl-CpG-binding protein 2 (MECP2). It is characterized by an early period of typical development with subsequent regression of previously acquired motor and speech skills in girls. The syndromic phenotypes are individualistic and dynamic over time. Thus far, it has been difficult to capture these dynamics and syndromic heterogeneity in the preclinical Mecp2-heterozygous female mouse model (Het). The emergence of computational neuroethology tools allows for robust analysis of complex and dynamic behaviors to model endophenotypes in preclinical models. Toward this first step, we utilized DeepLabCut, a marker-less pose estimation software to quantify trajectory kinematics and multidimensional analysis to characterize behavioral heterogeneity in Het in the previously benchmarked, ethologically relevant social cognition task of pup retrieval. We report the identification of two distinct phenotypes of adult Het: Het that display a delay in efficiency in early days and then improve over days like wild-type mice and Het that regress and perform worse in later days. Furthermore, regression is dependent on age and behavioral context and can be detected in the initial days of retrieval. Together, the novel identification of two populations of Het suggests differential effects on neural circuitry, opens new avenues to investigate the underlying molecular and cellular mechanisms of heterogeneity, and designs better studies for stratifying therapeutics.

Mapping the neural dynamics of locomotion across the Drosophila brain

Locomotion engages widely distributed networks of neurons. However, our understanding of the spatial architecture and temporal dynamics of the networks that underpin walking remains incomplete. We use volumetric two-photon imaging to map neural activity associated with walking across the entire brain of Drosophila. We define spatially clustered neural signals selectively associated with changes in either forward or angular velocity, demonstrating that neurons with similar behavioral selectivity are clustered. These signals reveal distinct topographic maps in diverse brain regions involved in navigation, memory, sensory processing, and motor control, as well as regions not previously linked to locomotion. We identify temporal trajectories of neural activity that sweep across these maps, including signals that anticipate future movement, representing the sequential engagement of clusters with different behavioral specificities. Finally, we register these maps to a connectome and identify neural networks that we propose underlie the observed signals, setting a foundation for subsequent circuit dissection. Overall, our work suggests a spatiotemporal framework for the emergence and execution of complex walking maneuvers and links this brain-wide neural activity to single neurons and local circuits.

Automated 3D analysis of social head-gaze behaviors in freely moving marmosets

Social communication relies on the ability to perceive and interpret the direction of others' attention, which is commonly conveyed through head orientation and gaze direction in both humans and non-human primates. However, traditional social gaze experiments in non-human primates require restraining head movements, which significantly limit their natural behavioral repertoire. Here, we developed a novel framework for accurately tracking facial features and three-dimensional head gaze orientations of multiple freely moving common marmosets (Callithrix jacchus). To accurately track the facial features of marmoset dyads in an arena, we adapted computer vision tools using deep learning networks combined with triangulation algorithms applied to the detected facial features to generate dynamic geometric facial frames in 3D space, overcoming common occlusion challenges. Furthermore, we constructed a virtual cone, oriented perpendicular to the facial frame, to model the head gaze directions. Using this framework, we were able to detect different types of interactive social gaze events, including partner-directed gaze and jointly-directed gaze to a shared spatial location. We observed clear effects of sex and familiarity on both interpersonal distance and gaze dynamics in marmoset dyads. Unfamiliar pairs exhibited more stereotyped patterns of arena occupancy, more sustained levels of social gaze across inter-animal distance, and increased gaze monitoring. On the other hand, familiar pairs exhibited higher levels of joint gazes. Moreover, males displayed significantly elevated levels of gazes toward females' faces and the surrounding regions irrespective of familiarity. Our study lays the groundwork for a rigorous quantification of primate behaviors in naturalistic settings.

Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.

Sensory neurons couple arousal and foraging decisions in Caenorhabditis elegans

Foraging animals optimize feeding decisions by adjusting both common and rare behavioral patterns. Here, we characterize the relationship between an animal's arousal state and a rare decision to leave a patch of bacterial food. Using long-term tracking and behavioral state classification, we find that food leaving decisions in Caenorhabditis elegans are coupled to arousal states across multiple timescales. Leaving emerges probabilistically over minutes from the high arousal roaming state, but is suppressed during the low arousal dwelling state. Immediately before leaving, animals have a brief acceleration in speed that appears as a characteristic signature of this behavioral motif. Neuromodulatory mutants and optogenetic manipulations that increase roaming have a coupled increase in leaving rates, and similarly acute manipulations that inhibit feeding induce both roaming and leaving. By contrast, inactivating a set of chemosensory neurons that depend on the cGMP-gated transduction channel TAX-4 uncouples roaming and leaving dynamics. In addition, tax-4-expressing sensory neurons promote lawn-leaving behaviors that are elicited by feeding inhibition. Our results indicate that sensory neurons responsive to both internal and external cues play an integrative role in arousal and foraging decisions.

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms have revolutionized the analysis of animal behavior, enabling investigators to flexibly quantify behavioral dynamics from conventional video recordings obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into the modules out of which behavior is organized. This challenge is particularly acute because keypoint data is susceptible to high frequency jitter that clustering algorithms can mistake for transitions between behavioral modules. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ("syllables") from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to effectively identify syllables whose boundaries correspond to natural sub-second discontinuities inherent to mouse behavior. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior, and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq therefore renders behavioral syllables and grammar accessible to the many researchers who use standard video to capture animal behavior.

Myomatrix arrays for high-definition muscle recording

Neurons coordinate their activity to produce an astonishing variety of motor behaviors. Our present understanding of motor control has grown rapidly thanks to new methods for recording and analyzing populations of many individual neurons over time. In contrast, current methods for recording the nervous system's actual motor output - the activation of muscle fibers by motor neurons - typically cannot detect the individual electrical events produced by muscle fibers during natural behaviors and scale poorly across species and muscle groups. Here we present a novel class of electrode devices ('Myomatrix arrays') that record muscle activity at unprecedented resolution across muscles and behaviors. High-density, flexible electrode arrays allow for stable recordings from the muscle fibers activated by a single motor neuron, called a 'motor unit,' during natural behaviors in many species, including mice, rats, primates, songbirds, frogs, and insects. This technology therefore allows the nervous system's motor output to be monitored in unprecedented detail during complex behaviors across species and muscle morphologies. We anticipate that this technology will allow rapid advances in understanding the neural control of behavior and identifying pathologies of the motor system.

Artificial intelligence in epilepsy phenotyping

Artificial intelligence (AI) allows data analysis and integration at an unprecedented granularity and scale. Here we review the technological advances, challenges, and future perspectives of using AI for electro-clinical phenotyping of animal models and patients with epilepsy. In translational research, AI models accurately identify behavioral states in animal models of epilepsy, allowing identification of correlations between neural activity and interictal and ictal behavior. Clinical applications of AI-based automated and semi-automated analysis of audio and video recordings of people with epilepsy, allow significant data reduction and reliable detection and classification of major motor seizures. AI models can accurately identify electrographic biomarkers of epilepsy, such as spikes, high-frequency oscillations, and seizure patterns. Integrating AI analysis of electroencephalographic, clinical, and behavioral data will contribute to optimizing therapy for patients with epilepsy.

Everyone knows what behavior is but they just don't agree on it

Studying "behavior" lies at the heart of many disciplines. Nevertheless, academics rarely provide an explicit definition of what "behavior" actually is. What range of definitions do people use, and how does that vary across disciplines? To answer these questions, we have developed a survey to probe what constitutes "behavior." We find that academics adopt different definitions of behavior according to their academic discipline, animal model that they work with, and level of academic seniority. Using hierarchical clustering, we identify at least six distinct types of "behavior" which are used in seven distinct operational archetypes of "behavior." Individual respondents have clear consistent definitions of behavior, but these definitions are not consistent across the population. Our study is a call for academics to clarify what they mean by "behavior" wherever they study it, with the hope that this will foster interdisciplinary studies that will improve our understanding of behavioral phenomena.

Elephants and algorithms: a review of the current and future role of AI in elephant monitoring

Artificial intelligence (AI) and machine learning (ML) present revolutionary opportunities to enhance our understanding of animal behaviour and conservation strategies. Using elephants, a crucial species in Africa and Asia's protected areas, as our focal point, we delve into the role of AI and ML in their conservation. Given the increasing amounts of data gathered from a variety of sensors like cameras, microphones, geophones, drones and satellites, the challenge lies in managing and interpreting this vast data. New AI and ML techniques offer solutions to streamline this process, helping us extract vital information that might otherwise be overlooked. This paper focuses on the different AI-driven monitoring methods and their potential for improving elephant conservation. Collaborative efforts between AI experts and ecological researchers are essential in leveraging these innovative technologies for enhanced wildlife conservation, setting a precedent for numerous other species.

Automated maternal behavior during early life in rodents (AMBER) pipeline

Mother-infant interactions during the early postnatal period are critical for infant survival and the scaffolding of infant development. Rodent models are used extensively to understand how these early social experiences influence neurobiology across the lifespan. However, methods for measuring postnatal dam-pup interactions typically involve time-consuming manual scoring, vary widely between research groups, and produce low density data that limits downstream analytical applications. To address these methodological issues, we developed the Automated Maternal Behavior during Early life in Rodents (AMBER) pipeline for quantifying home-cage maternal and mother-pup interactions using open-source machine learning tools. DeepLabCut was used to track key points on rat dams (32 points) and individual pups (9 points per pup) in postnatal day 1-10 video recordings. Pose estimation models reached key point test errors of approximately 4.1-10 mm (14.39 pixels) and 3.44-7.87 mm (11.81 pixels) depending on depth of animal in the frame averaged across all key points for dam and pups respectively. Pose estimation data and human-annotated behavior labels from 38 videos were used with Simple Behavioral Analysis (SimBA) to generate behavior classifiers for dam active nursing, passive nursing, nest attendance, licking and grooming, self-directed grooming, eating, and drinking using random forest algorithms. All classifiers had excellent performance on test frames, with F1 scores above 0.886. Performance on hold-out videos remained high for nest attendance (F1 = 0.990), active nursing (F1 = 0.828), and licking and grooming (F1 = 0.766) but was lower for eating, drinking, and self-directed grooming (F1 = 0.534-0.554). A set of 242 videos was used with AMBER and produced behavior measures in the expected range from postnatal 1-10 home-cage videos. This pipeline is a major advancement in assessing home-cage dam-pup interactions in a way that reduces experimenter burden while increasing reproducibility, reliability, and detail of data for use in developmental studies without the need for special housing systems or proprietary software.

Ethanol Behavioral Responses in Drosophila

Drosophila melanogaster is a powerful genetic model for investigating the mechanisms underlying ethanol-induced behaviors, metabolism, and preference. Ethanol-induced locomotor activity is especially useful for understanding the mechanisms by which ethanol acutely affects the brain and behavior. Ethanol-induced locomotor activity is characterized by hyperlocomotion and subsequent sedation with increased exposure duration or concentration. Locomotor activity is an efficient, easy, robust, and reproducible behavioral screening tool for identifying underlying genes and neuronal circuits as well as investigating genetic and molecular pathways. We introduce a detailed protocol for performing experiments investigating how volatilized ethanol affects locomotor activity using the fly Group Activity Monitor (flyGrAM). We introduce installation, implementation, data collection, and subsequent data-analysis methods for investigating how volatilized stimuli affect activity. We also introduce a procedure for how to optogenetically probe neuronal activity to identify the neural mechanisms underlying locomotor activity.

Methods for Exploring the Circuit Basis of Ethanol-Induced Changes in Drosophila Group Locomotor Activity

Locomotion is a behavioral readout that can be used to understand responses to specific stimuli or perturbations. The fly Group Activity Monitor (flyGrAM) provides a high-throughput and high-content readout of the acute stimulatory and sedative effects of ethanol. The flyGrAM system is adaptable and seamlessly introduces thermogenetic or optogenetic stimulation to dissect neural circuits underlying behavior and tests responses to other volatilized stimuli (humidified air, odorants, anesthetics, vaporized drugs of abuse, etc.). The automated quantification and readout of activity provide users with a real-time representation of the group activity within each chamber throughout the experiment, helping users to quickly determine proper ethanol doses and duration, run behavioral screens, and plan follow-up experiments.

NAPS: Integrating pose estimation and tag-based tracking

1. Significant advances in computational ethology have allowed the quantification of behaviour in unprecedented detail. Tracking animals in social groups, however, remains challenging as most existing methods can either capture pose or robustly retain individual identity over time but not both. 2. To capture finely resolved behaviours while maintaining individual identity, we built NAPS (NAPS is ArUco Plus SLEAP), a hybrid tracking framework that combines state-of-the-art, deep learning-based methods for pose estimation (SLEAP) with unique markers for identity persistence (ArUco). We show that this framework allows the exploration of the social dynamics of the common eastern bumblebee (Bombus impatiens). 3. We provide a stand-alone Python package for implementing this framework along with detailed documentation to allow for easy utilization and expansion. We show that NAPS can scale to long timescale experiments at a high frame rate and that it enables the investigation of detailed behavioural variation within individuals in a group. 4. Expanding the toolkit for capturing the constituent behaviours of social groups is essential for understanding the structure and dynamics of social networks. NAPS provides a key tool for capturing these behaviours and can provide critical data for understanding how individual variation influences collective dynamics.

Rapid automated 3-D pose estimation of larval zebrafish using a physical model-trained neural network

Quantitative ethology requires an accurate estimation of an organism's postural dynamics in three dimensions plus time. Technological progress over the last decade has made animal pose estimation in challenging scenarios possible with unprecedented detail. Here, we present (i) a fast automated method to record and track the pose of individual larval zebrafish in a 3-D environment, applicable when accurate human labeling is not possible; (ii) a rich annotated dataset of 3-D larval poses for ethologists and the general zebrafish and machine learning community; and (iii) a technique to generate realistic, annotated larval images in different behavioral contexts. Using a three-camera system calibrated with refraction correction, we record diverse larval swims under free swimming conditions and in response to acoustic and optical stimuli. We then employ a convolutional neural network to estimate 3-D larval poses from video images. The network is trained against a set of synthetic larval images rendered using a 3-D physical model of larvae. This 3-D model samples from a distribution of realistic larval poses that we estimate a priori using a template-based pose estimation of a small number of swim bouts. Our network model, trained without any human annotation, performs larval pose estimation three orders of magnitude faster and with accuracy comparable to the template-based approach, capturing detailed kinematics of 3-D larval swims. It also applies accurately to other datasets collected under different imaging conditions and containing behavioral contexts not included in our training.

Discovering individual-specific gait signatures from data-driven models of neuromechanical dynamics

Locomotion results from the interactions of highly nonlinear neural and biomechanical dynamics. Accordingly, understanding gait dynamics across behavioral conditions and individuals based on detailed modeling of the underlying neuromechanical system has proven difficult. Here, we develop a data-driven and generative modeling approach that recapitulates the dynamical features of gait behaviors to enable more holistic and interpretable characterizations and comparisons of gait dynamics. Specifically, gait dynamics of multiple individuals are predicted by a dynamical model that defines a common, low-dimensional, latent space to compare group and individual differences. We find that highly individualized dynamics-i.e., gait signatures-for healthy older adults and stroke survivors during treadmill walking are conserved across gait speed. Gait signatures further reveal individual differences in gait dynamics, even in individuals with similar functional deficits. Moreover, components of gait signatures can be biomechanically interpreted and manipulated to reveal their relationships to observed spatiotemporal joint coordination patterns. Lastly, the gait dynamics model can predict the time evolution of joint coordination based on an initial static posture. Our gait signatures framework thus provides a generalizable, holistic method for characterizing and predicting cyclic, dynamical motor behavior that may generalize across species, pathologies, and gait perturbations.

Drosophila genotypes can be predicted from their exploration locomotive trajectories using supervised machine learning

This study employs supervised machine learning algorithms to test whether locomotive features during exploratory activity in open field arenas can serve as predictors for the genotype of fruit flies. Because of the nonlinearity in locomotive trajectories, traditional statistical methods that are used to compare exploratory activity between genotypes of fruit flies may not reveal all insights. 10-minute-long trajectories of four different genotypes of fruit flies in an open-field arena environment were captured. Turn angles and step size features extracted from the trajectories were used for training supervised learning models to predict the genotype of the fruit flies. Using the first five minute locomotive trajectories, an accuracy of 83% was achieved in differentiating wild-type flies from three other mutant genotypes. Using the final 5 min and the entire ten minute duration decreased the performance indicating that the most variations between the genotypes in their exploratory activity are exhibited in the first few minutes. Feature importance analysis revealed that turn angle is a better predictor than step size in predicting fruit fly genotype. Overall, this study demonstrates that features of trajectories can be used to predict the genotype of fruit flies through supervised machine learning methods.

Hierarchical action encoding in prefrontal cortex of freely moving macaques

Our natural behavioral repertoires include coordinated actions of characteristic types. To better understand how neural activity relates to the expression of actions and action switches, we studied macaques performing a freely moving foraging task in an open environment. We developed a novel analysis pipeline that can identify meaningful units of behavior, corresponding to recognizable actions such as sitting, walking, jumping, and climbing. On the basis of transition probabilities between these actions, we found that behavior is organized in a modular and hierarchical fashion. We found that, after regressing out many potential confounders, actions are associated with specific patterns of firing in each of six prefrontal brain regions and that, overall, encoding of action category is progressively stronger in more dorsal and more caudal prefrontal regions. Together, these results establish a link between selection of units of primate behavior on one hand and neuronal activity in prefrontal regions on the other.

Myomatrix arrays for high-definition muscle recording

Neurons coordinate their activity to produce an astonishing variety of motor behaviors. Our present understanding of motor control has grown rapidly thanks to new methods for recording and analyzing populations of many individual neurons over time. In contrast, current methods for recording the nervous system's actual motor output - the activation of muscle fibers by motor neurons - typically cannot detect the individual electrical events produced by muscle fibers during natural behaviors and scale poorly across species and muscle groups. Here we present a novel class of electrode devices ("Myomatrix arrays") that record muscle activity at unprecedented resolution across muscles and behaviors. High-density, flexible electrode arrays allow for stable recordings from the muscle fibers activated by a single motor neuron, called a "motor unit", during natural behaviors in many species, including mice, rats, primates, songbirds, frogs, and insects. This technology therefore allows the nervous system's motor output to be monitored in unprecedented detail during complex behaviors across species and muscle morphologies. We anticipate that this technology will allow rapid advances in understanding the neural control of behavior and in identifying pathologies of the motor system.

Capturing continuous, long timescale behavioral changes in Drosophila melanogaster postural data

Animal behavior spans many timescales, from short, seconds-scale actions to circadian rhythms over many hours to life-long changes during aging. Most quantitative behavior studies have focused on short-timescale behaviors such as locomotion and grooming. Analysis of these data suggests there exists a hierarchy of timescales; however, the limited duration of these experiments prevents the investigation of the full temporal structure. To access longer timescales of behavior, we continuously recorded individual Drosophila melanogaster at 100 frames per second for up to 7 days at a time in featureless arenas on sucrose-agarose media. We use the deep learning framework SLEAP to produce a full-body postural data set for 47 individuals resulting in nearly 2 billion pose instances. We identify stereotyped behaviors such as grooming, proboscis extension, and locomotion and use the resulting ethograms to explore how the flies' behavior varies across time of day and days in the experiment. We find distinct circadian patterns in all of our stereotyped behavior and also see changes in behavior over the course of the experiment as the flies weaken and die.

A unified open-source platform for multimodal neural recording and perturbation during naturalistic behavior

Behavioral neuroscience faces two conflicting demands: long-duration recordings from large neural populations and unimpeded animal behavior. To meet this challenge, we developed ONIX, an open-source data acquisition system with high data throughput (2GB/sec) and low closed-loop latencies (<1ms) that uses a novel 0.3 mm thin tether to minimize behavioral impact. Head position and rotation are tracked in 3D and used to drive active commutation without torque measurements. ONIX can acquire from combinations of passive electrodes, Neuropixels probes, head-mounted microscopes, cameras, 3D-trackers, and other data sources. We used ONIX to perform uninterrupted, long (~7 hours) neural recordings in mice as they traversed complex 3-dimensional terrain. ONIX allowed exploration with similar mobility as non-implanted animals, in contrast to conventional tethered systems which restricted movement. By combining long recordings with full mobility, our technology will enable new progress on questions that require high-quality neural recordings during ethologically grounded behaviors.

Inhibitory feedback from the motor circuit gates mechanosensory processing in Caenorhabditis elegans

Animals must integrate sensory cues with their current behavioral context to generate a suitable response. How this integration occurs is poorly understood. Previously, we developed high-throughput methods to probe neural activity in populations of Caenorhabditis elegans and discovered that the animal's mechanosensory processing is rapidly modulated by the animal's locomotion. Specifically, we found that when the worm turns it suppresses its mechanosensory-evoked reversal response. Here, we report that C. elegans use inhibitory feedback from turning-associated neurons to provide this rapid modulation of mechanosensory processing. By performing high-throughput optogenetic perturbations triggered on behavior, we show that turning-associated neurons SAA, RIV, and/or SMB suppress mechanosensory-evoked reversals during turns. We find that activation of the gentle-touch mechanosensory neurons or of any of the interneurons AIZ, RIM, AIB, and AVE during a turn is less likely to evoke a reversal than activation during forward movement. Inhibiting neurons SAA, RIV, and SMB during a turn restores the likelihood with which mechanosensory activation evokes reversals. Separately, activation of premotor interneuron AVA evokes reversals regardless of whether the animal is turning or moving forward. We therefore propose that inhibitory signals from SAA, RIV, and/or SMB gate mechanosensory signals upstream of neuron AVA. We conclude that C. elegans rely on inhibitory feedback from the motor circuit to modulate its response to sensory stimuli on fast timescales. This need for motor signals in sensory processing may explain the ubiquity in many organisms of motor-related neural activity patterns seen across the brain, including in sensory processing areas.