Predictability and hierarchy in Drosophila behavior

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ('syllables') from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.

Characterizing the structure of mouse behavior using Motion Sequencing

Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, exhibiting variability across scales that reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build Markov models of movement sequences that bridges across time scales and enables a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish responding to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising versus wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive stimuli, or in search for appetitive prey. As our method encodes the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies, we use it to uncover a hierarchical structure in the phenotypic variability that reflects exploration-exploitation trade-offs. Across a wide range of sensory cues, a major source of variation among fish is driven by prior and/or immediate exposure to prey that induces exploitation phenotypes. A large degree of variability that is not explained by environmental cues unravels motivational states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, by extracting the timescales of motor strategies deployed during navigation, our approach exposes structure among individuals and reveals internal states tuned by prior experience.

STPoseNet: A real-time spatiotemporal network model for robust mouse pose estimation

Animal behavior analysis plays a crucial role in contemporary neuroscience research. However, the performance of the frame-by-frame approach may degrade in scenarios with occlusions or motion blur. In this study, we propose a spatiotemporal network model based on YOLOv8 to enhance the accuracy of key-point detection in mouse behavioral experimental videos. This model integrates a time-domain tracking strategy comprising two components: the first part utilizes key-point detection results from the previous frame to detect potential target locations in the subsequent frame; the second part employs Kalman filtering to analyze key-point changes prior to detection, allowing for the estimation of missing key-points. In the comparison of pose estimation results between our approach, YOLOv8, DeepLabCut and SLEAP on videos of three mouse behavioral experiments, our approach demonstrated significantly superior performance. This suggests that our method offers a new and effective means of accurately tracking and estimating pose in mice through spatiotemporal processing.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, resulting in variability that ultimately reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build models that bridges across time scales that enable a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish exposed to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising and wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive (dark) stimuli or in search for prey. Our method enables us to encode the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies. By doing so, we uncover a hierarchical structure to the phenotypic variability that corresponds to exploration-exploitation trade-offs. Within a wide range of sensory cues, a major source of variation among fish is driven by prior and immediate exposure to prey that induces exploitation phenotypes. However, a large degree of variability is unexplained by environmental cues, pointing to hidden states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, our approach extracts the timescales of motor strategies deployed during navigation, exposing undiscovered structure among individuals and pointing to internal states tuned by prior experience.

Inhibitory control of locomotor statistics in walking Drosophila

In order to forage for food, many animals regulate not only specific limb movements but the statistics of locomotor behavior over time, for example switching between long-range dispersal behaviors and more localized search depending on the availability of resources. How pre-motor circuits regulate such locomotor statistics is not clear. Here we took advantage of the robust changes in locomotor statistics evoked by attractive odors in walking Drosophila to investigate their neural control. We began by analyzing the statistics of ground speed and angular velocity during three well-defined motor regimes: baseline walking, upwind running during odor, and search behavior following odor offset. We find that during search behavior, flies adopt higher angular velocities and slower ground speeds, and tend to turn for longer periods of time in one direction. We further find that flies spontaneously adopt periods of different mean ground speed, and that these changes in state influence the length of odor-evoked runs. We next developed a simple physiologically-inspired computational model of locomotor control that can recapitulate these statistical features of fly locomotion. Our model suggests that contralateral inhibition plays a key role both in regulating the difference between baseline and search behavior, and in modulating the response to odor with ground speed. As the fly connectome predicts decussating inhibitory neurons in the lateral accessory lobe (LAL), a pre-motor structure, we generated genetic tools to target these neurons and test their role in behavior. Consistent with our model, we found that activation of neurons labeled in one line increased curvature. In a second line labeling distinct neurons, activation and inactivation strongly and reciprocally regulated ground speed and altered the length of the odor-evoked run. Additional targeted light activation experiments argue that these effects arise from the brain rather than from neurons in the ventral nerve cord, while sparse activation experiments argue that speed control in the second line arises from both LAL neurons and a population of neurons in the dorsal superior medial protocerebrum (SMP). Together, our work develops a biologically plausible computational architecture that captures the statistical features of fly locomotion across behavioral states and identifies potential neural substrates of these computations.

Glial expression of Drosophila UBE3A causes spontaneous seizures modulated by 5-HT signaling

Misexpression of the E3 ubiquitin ligase UBE3A is thought to contribute to a range of neurological disorders. In the context of Dup15q syndrome, excess genomic copies of UBE3A is thought to contribute to the autism, muscle tone and spontaneous seizures characteristic of the disorder. In a Drosophila model of Dup 15q syndrome, it was recently shown glial-driven expression of the UBE3A ortholog dube3a led to a "bang-sensitive" phenotype, where mechanical shock triggers convulsions, suggesting glial dube3a expression contributes to hyperexcitability in flies. Here we directly compare the consequences of glial- and neuronal-driven dube3a expression on motor coordination and neuronal excitability in Drosophila. We utilized IowaFLI tracker and developed a hidden Markov Model to classify seizure-related immobilization. Both glial and neuronal driven dube3a expression led to clear motor phenotypes. However, only glial-driven dube3a expression displayed spontaneous immobilization events, that were exacerbated at high-temperature (38 °C). Using a tethered fly preparation we monitored flight muscle activity, we found glial-driven dube3a flies display spontaneous spike discharges which were bilaterally synchronized indicative of seizure activity. Neither control flies, nor neuronal- dube3a overexpressing flies display such firing patterns. Prior drug screen indicated bang-sensitivity in glial-driven dube3a expressing flies could be suppressed by certain 5-HT modulators. Consistent with this report, we found glial-driven dube3a flies fed the serotonin reuptake inhibitor vortioxetine and the 5HT 2A antagonist ketanserin displayed reduced immobilization and spike bursting. Together these findings highlight the potential for glial pathophysiology to drive Dup15q syndrome-related seizure activity.

Inferring neural dynamics of memory during naturalistic social communication

Memory processes in complex behaviors like social communication require forming representations of the past that grow with time. The neural mechanisms that support such continually growing memory remain unknown. We address this gap in the context of fly courtship, a natural social behavior involving the production and perception of long, complex song sequences. To study female memory for male song history in unrestrained courtship, we present 'Natural Continuation' (NC)-a general, simulation-based model comparison procedure to evaluate candidate neural codes for complex stimuli using naturalistic behavioral data. Applying NC to fly courtship revealed strong evidence for an adaptive population mechanism for how female auditory neural dynamics could convert long song histories into a rich mnemonic format. Song temporal patterning is continually transformed by heterogeneous nonlinear adaptation dynamics, then integrated into persistent activity, enabling common neural mechanisms to retain continuously unfolding information over long periods and yielding state-of-the-art predictions of female courtship behavior. At a population level this coding model produces multi-dimensional advection-diffusion-like responses that separate songs over a continuum of timescales and can be linearly transformed into flexible output signals, illustrating its potential to create a generic, scalable mnemonic format for extended input signals poised to drive complex behavioral responses. This work thus shows how naturalistic behavior can directly inform neural population coding models, revealing here a novel process for memory formation.

Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms have revolutionized the analysis of animal behavior, enabling investigators to flexibly quantify behavioral dynamics from conventional video recordings obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into the modules out of which behavior is organized. This challenge is particularly acute because keypoint data is susceptible to high frequency jitter that clustering algorithms can mistake for transitions between behavioral modules. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ("syllables") from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to effectively identify syllables whose boundaries correspond to natural sub-second discontinuities inherent to mouse behavior. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior, and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq therefore renders behavioral syllables and grammar accessible to the many researchers who use standard video to capture animal behavior.

NAPS: Integrating pose estimation and tag-based tracking

1. Significant advances in computational ethology have allowed the quantification of behaviour in unprecedented detail. Tracking animals in social groups, however, remains challenging as most existing methods can either capture pose or robustly retain individual identity over time but not both. 2. To capture finely resolved behaviours while maintaining individual identity, we built NAPS (NAPS is ArUco Plus SLEAP), a hybrid tracking framework that combines state-of-the-art, deep learning-based methods for pose estimation (SLEAP) with unique markers for identity persistence (ArUco). We show that this framework allows the exploration of the social dynamics of the common eastern bumblebee (Bombus impatiens). 3. We provide a stand-alone Python package for implementing this framework along with detailed documentation to allow for easy utilization and expansion. We show that NAPS can scale to long timescale experiments at a high frame rate and that it enables the investigation of detailed behavioural variation within individuals in a group. 4. Expanding the toolkit for capturing the constituent behaviours of social groups is essential for understanding the structure and dynamics of social networks. NAPS provides a key tool for capturing these behaviours and can provide critical data for understanding how individual variation influences collective dynamics.

Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

Deep learning-based markerless tracking has revolutionized studies of animal behavior. Yet the generalizability of trained models tends to be limited, as new training data typically needs to be generated manually for each setup or visual environment. With each model trained from scratch, researchers track distinct landmarks and analyze the resulting kinematic data in idiosyncratic ways. Moreover, due to inherent limitations in manual annotation, only a sparse set of landmarks are typically labeled. To address these issues, we developed an approach, which we term GlowTrack, for generating orders of magnitude more training data, enabling models that generalize across experimental contexts. We describe: a) a high-throughput approach for producing hidden labels using fluorescent markers; b) a multi-camera, multi-light setup for simulating diverse visual conditions; and c) a technique for labeling many landmarks in parallel, enabling dense tracking. These advances lay a foundation for standardized behavioral pipelines and more complete scrutiny of movement.

Capturing continuous, long timescale behavioral changes in Drosophila melanogaster postural data

Animal behavior spans many timescales, from short, seconds-scale actions to circadian rhythms over many hours to life-long changes during aging. Most quantitative behavior studies have focused on short-timescale behaviors such as locomotion and grooming. Analysis of these data suggests there exists a hierarchy of timescales; however, the limited duration of these experiments prevents the investigation of the full temporal structure. To access longer timescales of behavior, we continuously recorded individual Drosophila melanogaster at 100 frames per second for up to 7 days at a time in featureless arenas on sucrose-agarose media. We use the deep learning framework SLEAP to produce a full-body postural data set for 47 individuals resulting in nearly 2 billion pose instances. We identify stereotyped behaviors such as grooming, proboscis extension, and locomotion and use the resulting ethograms to explore how the flies' behavior varies across time of day and days in the experiment. We find distinct circadian patterns in all of our stereotyped behavior and also see changes in behavior over the course of the experiment as the flies weaken and die.

Emergent programmable behavior and chaos in dynamically driven active filaments

How the behavior of cells emerges from their constituent subcellular biochemical and physical parts is an outstanding challenge at the intersection of biology and physics. A remarkable example of single-cell behavior occurs in the ciliate Lacrymaria olor, which hunts for its prey via rapid movements and protrusions of a slender neck, many times the size of the original cell body. The dynamics of this cell neck is powered by a coat of cilia across its length and tip. How a cell can program this active filamentous structure to produce desirable behaviors like search and homing to a target remains unknown. Here, we present an active filament model that allows us to uncover how a "program" (time sequence of active forcing) leads to "behavior" (filament shape dynamics). Our model captures two key features of this system-time-varying activity patterns (extension and compression cycles) and active stresses that are uniquely aligned with the filament geometry-a "follower force" constraint. We show that active filaments under deterministic, time-varying follower forces display rich behaviors including periodic and aperiodic dynamics over long times. We further show that aperiodicity occurs due to a transition to chaos in regions of a biologically accessible parameter space. We also identify a simple nonlinear iterated map of filament shape that approximately predicts long-term behavior suggesting simple, artificial "programs" for filament functions such as homing and searching space. Last, we directly measure the statistical properties of biological programs in L. olor, enabling comparisons between model predictions and experiments.

SAMPL is a high-throughput solution to study unconstrained vertical behavior in small animals

Balance and movement are impaired in many neurological disorders. Recent advances in behavioral monitoring provide unprecedented access to posture and locomotor kinematics but without the throughput and scalability necessary to screen candidate genes/potential therapeutics. Here, we present a scalable apparatus to measure posture and locomotion (SAMPL). SAMPL includes extensible hardware and open-source software with real-time processing and can acquire data from D. melanogaster, C. elegans, and D. rerio as they move vertically. Using SAMPL, we define how zebrafish balance as they navigate vertically and discover small but systematic variations among kinematic parameters between genetic backgrounds. We demonstrate SAMPL's ability to resolve differences in posture and navigation as a function of effect size and data gathered, providing key data for screens. SAMPL is therefore both a tool to model balance and locomotor disorders and an exemplar of how to scale apparatus to support screens.

Hierarchical organization of rhesus macaque behavior

Primatologists, psychologists and neuroscientists have long hypothesized that primate behavior is highly structured. However, delineating that structure has been impossible due to the difficulties of precision behavioral tracking. Here we analyzed a dataset consisting of continuous measures of the 3D position of two male rhesus macaques (Macaca mulatta) performing three different tasks in a large unrestrained environment over several hours. Using an unsupervised embedding approach on the tracked joints, we identified commonly repeated pose patterns, which we call postures. We found that macaques' behavior is characterized by 49 distinct postures, lasting an average of 0.6 seconds. We found evidence that behavior is hierarchically organized, in that transitions between poses tend to occur within larger modules, which correspond to identifiable actions; these actions are further organized hierarchically. Our behavioral decomposition allows us to identify universal (cross-individual and cross-task) and unique (specific to each individual and task) principles of behavior. These results demonstrate the hierarchical nature of primate behavior, provide a method for the automated ethogramming of primate behavior, and provide important constraints on neural models of pose generation.

Arc regulates a second-guessing cognitive bias during naturalistic foraging through effects on discrete behavior modules

Foraging in animals relies on innate decision-making heuristics that can result in suboptimal cognitive biases in some contexts. The mechanisms underlying these biases are not well understood, but likely involve strong genetic effects. To explore this, we studied fasted mice using a naturalistic foraging paradigm and discovered an innate cognitive bias called "second-guessing." This involves repeatedly investigating an empty former food patch instead of consuming available food, which hinders the mice from maximizing feeding benefits. The synaptic plasticity gene Arc is revealed to play a role in this bias, as Arc-deficient mice did not exhibit second-guessing and consumed more food. In addition, unsupervised machine learning decompositions of foraging identified specific behavior sequences, or "modules", that are affected by Arc. These findings highlight the genetic basis of cognitive biases in decision making, show links between behavior modules and cognitive bias, and provide insight into the ethological roles of Arc in naturalistic foraging.

Intrinsic timescales in the visual cortex change with selective attention and reflect spatial connectivity

Intrinsic timescales characterize dynamics of endogenous fluctuations in neural activity. Variation of intrinsic timescales across the neocortex reflects functional specialization of cortical areas, but less is known about how intrinsic timescales change during cognitive tasks. We measured intrinsic timescales of local spiking activity within columns of area V4 in male monkeys performing spatial attention tasks. The ongoing spiking activity unfolded across at least two distinct timescales, fast and slow. The slow timescale increased when monkeys attended to the receptive fields location and correlated with reaction times. By evaluating predictions of several network models, we found that spatiotemporal correlations in V4 activity were best explained by the model in which multiple timescales arise from recurrent interactions shaped by spatially arranged connectivity, and attentional modulation of timescales results from an increase in the efficacy of recurrent interactions. Our results suggest that multiple timescales may arise from the spatial connectivity in the visual cortex and flexibly change with the cognitive state due to dynamic effective interactions between neurons.

Emerging technologies for behavioral research in changing environments

The first response exhibited by animals to changing environments is typically behavioral. Behavior is thus central to predicting, and mitigating, the impacts that natural and anthropogenic environmental changes will have on populations and, consequently, ecosystems. Yet the inherently multiscale nature of behavior, as well as the complexities associated with inferring how animals perceive their world, and make decisions, has constrained the scope of behavioral research. Major technological advances in electronics and in machine learning, however, provide increasingly powerful means to see, analyze, and interpret behavior in its natural complexity. We argue that these disruptive technologies will foster new approaches that will allow us to move beyond quantitative descriptions and reveal the underlying generative processes that give rise to behavior.

Scalable Apparatus to Measure Posture and Locomotion (SAMPL): a high-throughput solution to study unconstrained vertical behavior in small animals

Balance and movement are impaired in a wide variety of neurological disorders. Recent advances in behavioral monitoring provide unprecedented access to posture and locomotor kinematics, but without the throughput and scalability necessary to screen candidate genes / potential therapeutics. We present a powerful solution: a Scalable Apparatus to Measure Posture and Locomotion (SAMPL). SAMPL includes extensible imaging hardware and low-cost open-source acquisition software with real-time processing. We first demonstrate that SAMPL's hardware and acquisition software can acquire data from from D. melanogaster, C. elegans, and D. rerio as they move vertically. Next, we leverage SAMPL's throughput to rapidly (two weeks) gather a new zebrafish dataset. We use SAMPL's analysis and visualization tools to replicate and extend our current understanding of how zebrafish balance as they navigate through a vertical environment. Next, we discover (1) that key kinematic parameters vary systematically with genetic background, and (2) that such background variation is small relative to the changes that accompany early development. Finally, we simulate SAMPL's ability to resolve differences in posture or vertical navigation as a function of affect size and data gathered -- key data for screens. Taken together, our apparatus, data, and analysis provide a powerful solution for labs using small animals to investigate balance and locomotor disorders at scale. More broadly, SAMPL is both an adaptable resource for labs looking process videographic measures of behavior in real-time, and an exemplar of how to scale hardware to enable the throughput necessary for screening.

Leveraging big data to uncover the eco-evolutionary factors shaping behavioural development

Mapping the eco-evolutionary factors shaping the development of animals' behavioural phenotypes remains a great challenge. Recent advances in 'big behavioural data' research-the high-resolution tracking of individuals and the harnessing of that data with powerful analytical tools-have vastly improved our ability to measure and model developing behavioural phenotypes. Applied to the study of behavioural ontogeny, the unfolding of whole behavioural repertoires can be mapped in unprecedented detail with relative ease. This overcomes long-standing experimental bottlenecks and heralds a surge of studies that more finely define and explore behavioural-experiential trajectories across development. In this review, we first provide a brief guide to state-of-the-art approaches that allow the collection and analysis of high-resolution behavioural data across development. We then outline how such approaches can be used to address key issues regarding the ecological and evolutionary factors shaping behavioural development: developmental feedbacks between behaviour and underlying states, early life effects and behavioural transitions, and information integration across development.

Social behavioral profiling by unsupervised deep learning reveals a stimulative effect of dopamine D3 agonists on zebrafish sociality

It has been a major challenge to systematically evaluate and compare how pharmacological perturbations influence social behavioral outcomes. Although some pharmacological agents are known to alter social behavior, precise description and quantification of such effects have proven difficult. We developed a scalable social behavioral assay for zebrafish named ZeChat based on unsupervised deep learning to characterize sociality at high resolution. High-dimensional and dynamic social behavioral phenotypes are automatically classified using this method. By screening a neuroactive compound library, we found that different classes of chemicals evoke distinct patterns of social behavioral fingerprints. By examining these patterns, we discovered that dopamine D3 agonists possess a social stimulative effect on zebrafish. The D3 agonists pramipexole, piribedil, and 7-hydroxy-DPAT-HBr rescued social deficits in a valproic-acid-induced zebrafish autism model. The ZeChat platform provides a promising approach for dissecting the pharmacology of social behavior and discovering novel social-modulatory compounds.

Emergent behaviour and neural dynamics in artificial agents tracking odour plumes

Tracking an odour plume to locate its source under variable wind and plume statistics is a complex task. Flying insects routinely accomplish such tracking, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behaviour and its underlying neural circuitry have been studied experimentally. Here we take a complementary in silico approach to develop an integrated understanding of their behaviour and neural computations. Specifically, we train artificial recurrent neural network agents using deep reinforcement learning to locate the source of simulated odour plumes that mimic features of plumes in a turbulent flow. Interestingly, the agents' emergent behaviours resemble those of flying insects, and the recurrent neural networks learn to compute task-relevant variables with distinct dynamic structures in population activity. Our analyses put forward a testable behavioural hypothesis for tracking plumes in changing wind direction, and we provide key intuitions for memory requirements and neural dynamics in odour plume tracking.

Structural analyses in the study of behavior: From rodents to non-human primates

The term "structure" indicates a set of components that, in relation to each other, shape an organic complex. Such a complex takes on essential connotations of functionally unitary entity resulting from the mutual relationships of its constituent elements. In a broader sense, we can use the word "structure" to define the set of relationships among the elements of an emergent system that is not determined by the mere algebraic sum of these elements, but by the interdependence relationships of these components from which the function of the entire structure itself derives. The behavior of an integrated living being can be described in structural terms via an ethogram, defined as an itemized list of behavioral units. Akin to an architectural structure, a behavioral structure arises from the reciprocal relationships that the individual units of behavior establish. Like an architectural structure, the function of the resulting behaving complex emerges from the relationships of the parts. Hence, studying behavior in its wholeness necessitates not only the identification of its constitutive units in their autarchic individuality, but also, and importantly, some understanding of their relationships. This paper aimed to critically review different methods to study behavior in structural terms. First, we emphasized the utilization of T-pattern analysis, i.e., one of the most effective and reliable tools to provide structural information on behavior. Second, we discussed the application of other methodological approaches that are based on the analysis of transition matrices, such as hierarchical clustering, stochastic analyses, and adjusted residuals. Unlike T-pattern analysis, these methods allow researchers to explore behavioral structure beyond its temporal characteristics and through other relational constraints. After an overview of how these methods are used in the study of animal behavior, from rodents to non-human primates, we discussed the specificities, advantages and challenges of each approach. This paper could represent a useful background for all scientists who intend to study behavior both quantitatively and structurally, that is in terms of the reciprocal relationships that the various units of a given behavioral repertoire normally weave together.

Inference, Prediction, & Entropy-Rate Estimation of Continuous-Time, Discrete-Event Processes

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.

A unicellular walker controlled by a microtubule-based finite-state machine

Cells are complex biochemical systems whose behaviors emerge from interactions among myriad molecular components. Computation is often invoked as a general framework for navigating this cellular complexity. However, it is unclear how cells might embody computational processes such that the theories of computation, including finite-state machine models, could be productively applied. Here, we demonstrate finite-state-machine-like processing embodied in cells using the walking behavior of Euplotes eurystomus, a ciliate that walks across surfaces using fourteen motile appendages (cirri). We found that cellular walking entails regulated transitions among a discrete set of gait states. The set of observed transitions decomposes into a small group of high-probability, temporally irreversible transitions and a large group of low-probability, time-symmetric transitions, thus revealing stereotypy in the sequential patterns of state transitions. Simulations and experiments suggest that the sequential logic of the gait is functionally important. Taken together, these findings implicate a finite-state-machine-like process. Cirri are connected by microtubule bundles (fibers), and we found that the dynamics of cirri involved in different state transitions are associated with the structure of the fiber system. Perturbative experiments revealed that the fibers mediate gait coordination, suggesting a mechanical basis of gait control.

The emergence and influence of internal states

Animal behavior is shaped by a variety of "internal states"-partially hidden variables that profoundly shape perception, cognition, and action. The neural basis of internal states, such as fear, arousal, hunger, motivation, aggression, and many others, is a prominent focus of research efforts across animal phyla. Internal states can be inferred from changes in behavior, physiology, and neural dynamics and are characterized by properties such as pleiotropy, persistence, scalability, generalizability, and valence. To date, it remains unclear how internal states and their properties are generated by nervous systems. Here, we review recent progress, which has been driven by advances in behavioral quantification, cellular manipulations, and neural population recordings. We synthesize research implicating defined subsets of state-inducing cell types, widespread changes in neural activity, and neuromodulation in the formation and updating of internal states. In addition to highlighting the significance of these findings, our review advocates for new approaches to clarify the underpinnings of internal brain states across the animal kingdom.

The vertical light-gradient and its potential impact on animal distribution and behavior

The visual environment provides vital cues allowing animals to assess habitat quality, weather conditions or measure time of day. Together with other sensory cues and physiological conditions, the visual environment sets behavioral states that make the animal more prone to engage in some behaviors, and less in others. This master-control of behavior serves a fundamental and essential role in determining the distribution and behavior of all animals. Although it is obvious that visual information contains vital input for setting behavioral states, the precise nature of these visual cues remains unknown. Here we use a recently described method to quantify the distribution of light reaching animals’ eyes in different environments. The method records the vertical gradient (as a function of elevation angle) of intensity, spatial structure and spectral balance. Comparison of measurements from different types of environments, weather conditions, times of day, and seasons reveal that these aspects can be readily discriminated from one another. The vertical gradients of radiance, spatial structure (contrast) and color are thus reliable indicators that are likely to have a strong impact on animal behavior and spatial distribution.

Neural mechanisms underlying the temporal organization of naturalistic animal behavior

Naturalistic animal behavior exhibits a strikingly complex organization in the temporal domain, with variability arising from at least three sources: hierarchical, contextual, and stochastic. What neural mechanisms and computational principles underlie such intricate temporal features? In this review, we provide a critical assessment of the existing behavioral and neurophysiological evidence for these sources of temporal variability in naturalistic behavior. Recent research converges on an emergent mechanistic theory of temporal variability based on attractor neural networks and metastable dynamics, arising via coordinated interactions between mesoscopic neural circuits. We highlight the crucial role played by structural heterogeneities as well as noise from mesoscopic feedback loops in regulating flexible behavior. We assess the shortcomings and missing links in the current theoretical and experimental literature and propose new directions of investigation to fill these gaps.

T cell morphodynamics reveal periodic shape oscillations in three-dimensional migration

T cells use sophisticated shape dynamics (morphodynamics) to migrate towards and neutralize infected and cancerous cells. However, there is limited quantitative understanding of the migration process in three-dimensional extracellular matrices (ECMs) and across timescales. Here, we leveraged recent advances in lattice light-sheet microscopy to quantitatively explore the three-dimensional morphodynamics of migrating T cells at high spatio-temporal resolution. We first developed a new shape descriptor based on spherical harmonics, incorporating key polarization information of the uropod. We found that the shape space of T cells is low-dimensional. At the behavioural level, run-and-stop migration modes emerge at approximately 150 s, and we mapped the morphodynamic composition of each mode using multiscale wavelet analysis, finding 'stereotyped' motifs. Focusing on the run mode, we found morphodynamics oscillating periodically (every approx. 100 s) that can be broken down into a biphasic process: front-widening with retraction of the uropod, followed by a rearward surface motion and forward extension, where intercalation with the ECM in both of these steps likely facilitates forward motion. Further application of these methods may enable the comparison of T cell migration across different conditions (e.g. differentiation, activation, tissues and drug treatments) and improve the precision of immunotherapeutic development.

Deep phenotyping reveals movement phenotypes in mouse neurodevelopmental models

BACKGROUND

Repetitive action, resistance to environmental change and fine motor disruptions are hallmarks of autism spectrum disorder (ASD) and other neurodevelopmental disorders, and vary considerably from individual to individual. In animal models, conventional behavioral phenotyping captures such fine-scale variations incompletely. Here we observed male and female C57BL/6J mice to methodically catalog adaptive movement over multiple days and examined two rodent models of developmental disorders against this dynamic baseline. We then investigated the behavioral consequences of a cerebellum-specific deletion in Tsc1 protein and a whole-brain knockout in Cntnap2 protein in mice. Both of these mutations are found in clinical conditions and have been associated with ASD.

METHODS

We used advances in computer vision and deep learning, namely a generalized form of high-dimensional statistical analysis, to develop a framework for characterizing mouse movement on multiple timescales using a single popular behavioral assay, the open-field test. The pipeline takes virtual markers from pose estimation to find behavior clusters and generate wavelet signatures of behavior classes. We measured spatial and temporal habituation to a new environment across minutes and days, different types of self-grooming, locomotion and gait.

RESULTS

Both Cntnap2 knockouts and L7-Tsc1 mutants showed forelimb lag during gait. L7-Tsc1 mutants and Cntnap2 knockouts showed complex defects in multi-day adaptation, lacking the tendency of wild-type mice to spend progressively more time in corners of the arena. In L7-Tsc1 mutant mice, failure to adapt took the form of maintained ambling, turning and locomotion, and an overall decrease in grooming. However, adaptation in these traits was similar between wild-type mice and Cntnap2 knockouts. L7-Tsc1 mutant and Cntnap2 knockout mouse models showed different patterns of behavioral state occupancy.

LIMITATIONS

Genetic risk factors for autism are numerous, and we tested only two. Our pipeline was only done under conditions of free behavior. Testing under task or social conditions would reveal more information about behavioral dynamics and variability.

CONCLUSIONS

Our automated pipeline for deep phenotyping successfully captures model-specific deviations in adaptation and movement as well as differences in the detailed structure of behavioral dynamics. The reported deficits indicate that deep phenotyping constitutes a robust set of ASD symptoms that may be considered for implementation in clinical settings as quantitative diagnosis criteria.

Long-range sequential dependencies precede complex syntactic production in language acquisition

To convey meaning, human language relies on hierarchically organized, long-range relationships spanning words, phrases, sentences and discourse. As the distances between elements (e.g. phonemes, characters, words) in human language sequences increase, the strength of the long-range relationships between those elements decays following a power law. This power-law relationship has been attributed variously to long-range sequential organization present in human language syntax, semantics and discourse structure. However, non-linguistic behaviours in numerous phylogenetically distant species, ranging from humpback whale song to fruit fly motility, also demonstrate similar long-range statistical dependencies. Therefore, we hypothesized that long-range statistical dependencies in human speech may occur independently of linguistic structure. To test this hypothesis, we measured long-range dependencies in several speech corpora from children (aged 6 months-12 years). We find that adult-like power-law statistical dependencies are present in human vocalizations at the earliest detectable ages, prior to the production of complex linguistic structure. These linguistic structures cannot, therefore, be the sole cause of long-range statistical dependencies in language.

Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group

The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimensionality-reduction framework already in use is information bottleneck (IB), in which the goal is to compress an "input" signal X while maximizing its mutual information with some stochastic "relevance" variable Y. IB has been applied in the vertebrate and invertebrate processing systems to characterize optimal encoding of the future motion of the external world. Other recent work has shown that the RG scheme for the dimer model could be "discovered" by a neural network attempting to solve an IB-like problem. This manuscript explores whether IB and any existing formulation of RG are formally equivalent. A class of soft-cutoff non-perturbative RG techniques are defined by families of non-deterministic coarsening maps, and hence can be formally mapped onto IB, and vice versa. For concreteness, this discussion is limited entirely to Gaussian statistics (GIB), for which IB has exact, closed-form solutions. Under this constraint, GIB has a semigroup structure, in which successive transformations remain IB-optimal. Further, the RG cutoff scheme associated with GIB can be identified. Our results suggest that IB can be used to impose a notion of "large scale" structure, such as biological function, on an RG procedure.

Measuring the repertoire of age-related behavioral changes in Drosophila melanogaster

Aging affects almost all aspects of an organism—its morphology, its physiology, its behavior. Isolating which biological mechanisms are regulating these changes, however, has proven difficult, potentially due to our inability to characterize the full repertoire of an animal’s behavior across the lifespan. Using data from fruit flies (D. melanogaster) we measure the full repertoire of behaviors as a function of age. We observe a sexually dimorphic pattern of changes in the behavioral repertoire during aging. Although the stereotypy of the behaviors and the complexity of the repertoire overall remains relatively unchanged, we find evidence that the observed alterations in behavior can be explained by changing the fly’s overall energy budget, suggesting potential connections between metabolism, aging, and behavior.

Aging is a ubiquitous biological phenomenon that affects many aspects of an animal’s appearance, physiology, and behavior. Our understanding of how changes in physiology lead to behavioral changes, however, has been partially limited by our ability to robustly quantify how behavior alters over timescales of days and weeks. In this study, we measure a large repertoire of behaviors of fruit flies at various ages, finding how the actions the animals perform shift with age. We observe a difference between the aging dynamics of male and female flies, and we show that many of these changes can be explained with a model of energy consumption, leading us to make predictions as to the role of metabolism in changes in aging behavior.

The evolution of brain architectures for predictive coding and active inference

This article considers the evolution of brain architectures for predictive processing. We argue that brain mechanisms for predictive perception and action are not late evolutionary additions of advanced creatures like us. Rather, they emerged gradually from simpler predictive loops (e.g. autonomic and motor reflexes) that were a legacy from our earlier evolutionary ancestors-and were key to solving their fundamental problems of adaptive regulation. We characterize simpler-to-more-complex brains formally, in terms of generative models that include predictive loops of increasing hierarchical breadth and depth. These may start from a simple homeostatic motif and be elaborated during evolution in four main ways: these include the multimodal expansion of predictive control into an allostatic loop; its duplication to form multiple sensorimotor loops that expand an animal's behavioural repertoire; and the gradual endowment of generative models with hierarchical depth (to deal with aspects of the world that unfold at different spatial scales) and temporal depth (to select plans in a future-oriented manner). In turn, these elaborations underwrite the solution to biological regulation problems faced by increasingly sophisticated animals. Our proposal aligns neuroscientific theorising-about predictive processing-with evolutionary and comparative data on brain architectures in different animal species. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

The Evolution of Visual Roles – Ancient Vision Versus Object Vision

Just like other complex biological features, image vision (multi-pixel light sensing) did not evolve suddenly. Animal visual systems have a long prehistory of non-imaging light sensitivity. The first spatial vision was likely very crude with only few pixels, and evolved to improve orientation behaviors previously supported by single-channel directional photoreception. The origin of image vision was simply a switch from single to multiple spatial channels, which improved the behaviors for finding a suitable habitat and position itself within it. Orientation based on spatial vision obviously involves active guidance of behaviors but, by necessity, also assessment of habitat suitability and environmental conditions. These conditions are crucial for deciding when to forage, reproduce, seek shelter, rest, etc. When spatial resolution became good enough to see other animals and interact with them, a whole range of new visual roles emerged: pursuit, escape, communication and other interactions. All these new visual roles require entirely new types of visual processing. Objects needed to be separated from the background, identified and classified to make the correct choice of interaction. Object detection and identification can be used actively to guide behaviors but of course also to assess the over-all situation. Visual roles can thus be classified as either ancient non-object-based tasks, or object vision. Each of these two categories can also be further divided into active visual tasks and visual assessment tasks. This generates four major categories of vision into which I propose that all visual roles can be categorized.

Automatic mapping of multiplexed social receptive fields by deep learning and GPU-accelerated 3D videography

Social interactions powerfully impact the brain and the body, but high-resolution descriptions of these important physical interactions and their neural correlates are lacking. Currently, most studies rely on labor-intensive methods such as manual annotation. Scalable and objective tracking methods are required to understand the neural circuits underlying social behavior. Here we describe a hardware/software system and analysis pipeline that combines 3D videography, deep learning, physical modeling, and GPU-accelerated robust optimization, with automatic analysis of neuronal receptive fields recorded in interacting mice. Our system ("3DDD Social Mouse Tracker") is capable of fully automatic multi-animal tracking with minimal errors (including in complete darkness) during complex, spontaneous social encounters, together with simultaneous electrophysiological recordings. We capture posture dynamics of multiple unmarked mice with high spatiotemporal precision (~2 mm, 60 frames/s). A statistical model that relates 3D behavior and neural activity reveals multiplexed 'social receptive fields' of neurons in barrel cortex. Our approach could be broadly useful for neurobehavioral studies of multiple animals interacting in complex low-light environments.

Emergence of behaviour in a self-organized living matter network

What is the origin of behaviour? Although typically associated with a nervous system, simple organisms also show complex behaviours. Among them, the slime mold Physarum polycephalum, a giant single cell, is ideally suited to study emergence of behaviour. Here, we show how locomotion and morphological adaptation behaviour emerge from self-organized patterns of rhythmic contractions of the actomyosin lining of the tubes making up the network-shaped organism. We quantify the spatio-temporal contraction dynamics by decomposing experimentally recorded contraction patterns into spatial contraction modes. Notably, we find a continuous spectrum of modes, as opposed to a few dominant modes. Our data suggests that the continuous spectrum of modes allows for dynamic transitions between a plethora of specific behaviours with transitions marked by highly irregular contraction states. By mapping specific behaviours to states of active contractions, we provide the basis to understand behaviour’s complexity as a function of biomechanical dynamics.

Metastable attractors explain the variable timing of stable behavioral action sequences

The timing of self-initiated actions shows large variability even when they are executed in stable, well-learned sequences. Could this mix of reliability and stochasticity arise within the same neural circuit? We trained rats to perform a stereotyped sequence of self-initiated actions and recorded neural ensemble activity in secondary motor cortex (M2), which is known to reflect trial-by-trial action-timing fluctuations. Using hidden Markov models, we established a dictionary between activity patterns and actions. We then showed that metastable attractors, representing activity patterns with a reliable sequential structure and large transition timing variability, could be produced by reciprocally coupling a high-dimensional recurrent network and a low-dimensional feedforward one. Transitions between attractors relied on correlated variability in this mesoscale feedback loop, predicting a specific structure of low-dimensional correlations that were empirically verified in M2 recordings. Our results suggest a novel mesoscale network motif based on correlated variability supporting naturalistic animal behavior.

A lexical approach for identifying behavioural action sequences

Animals display characteristic behavioural patterns when performing a task, such as the spiraling of a soaring bird or the surge-and-cast of a male moth searching for a female. Identifying such recurring sequences occurring rarely in noisy behavioural data is key to understanding the behavioural response to a distributed stimulus in unrestrained animals. Existing models seek to describe the dynamics of behaviour or segment individual locomotor episodes rather than to identify the rare and transient sequences of locomotor episodes that make up the behavioural response. To fill this gap, we develop a lexical, hierarchical model of behaviour. We designed an unsupervised algorithm called "BASS" to efficiently identify and segment recurring behavioural action sequences transiently occurring in long behavioural recordings. When applied to navigating larval zebrafish, BASS extracts a dictionary of remarkably long, non-Markovian sequences consisting of repeats and mixtures of slow forward and turn bouts. Applied to a novel chemotaxis assay, BASS uncovers chemotactic strategies deployed by zebrafish to avoid aversive cues consisting of sequences of fast large-angle turns and burst swims. In a simulated dataset of soaring gliders climbing thermals, BASS finds the spiraling patterns characteristic of soaring behaviour. In both cases, BASS succeeds in identifying rare action sequences in the behaviour deployed by freely moving animals. BASS can be easily incorporated into the pipelines of existing behavioural analyses across diverse species, and even more broadly used as a generic algorithm for pattern recognition in low-dimensional sequential data.

Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions

Recently developed methods in computational neuroethology have enabled increasingly detailed and comprehensive quantification of animal movements and behavioral kinematics. Vocal communication behavior is well poised for application of similar large-scale quantification methods in the service of physiological and ethological studies. This review describes emerging techniques that can be applied to acoustic and vocal communication signals with the goal of enabling study beyond a small number of model species. We review a range of modern computational methods for bioacoustics, signal processing, and brain-behavior mapping. Along with a discussion of recent advances and techniques, we include challenges and broader goals in establishing a framework for the computational neuroethology of vocal communication.

Bubblewrap: Online tiling and real-time flow prediction on neural manifolds

While most classic studies of function in experimental neuroscience have focused on the coding properties of individual neurons, recent developments in recording technologies have resulted in an increasing emphasis on the dynamics of neural populations. This has given rise to a wide variety of models for analyzing population activity in relation to experimental variables, but direct testing of many neural population hypotheses requires intervening in the system based on current neural state, necessitating models capable of inferring neural state online. Existing approaches, primarily based on dynamical systems, require strong parametric assumptions that are easily violated in the noise-dominated regime and do not scale well to the thousands of data channels in modern experiments. To address this problem, we propose a method that combines fast, stable dimensionality reduction with a soft tiling of the resulting neural manifold, allowing dynamics to be approximated as a probability flow between tiles. This method can be fit efficiently using online expectation maximization, scales to tens of thousands of tiles, and outperforms existing methods when dynamics are noise-dominated or feature multi-modal transition probabilities. The resulting model can be trained at kiloHertz data rates, produces accurate approximations of neural dynamics within minutes, and generates predictions on submillisecond time scales. It retains predictive performance throughout many time steps into the future and is fast enough to serve as a component of closed-loop causal experiments.

Recent approaches to study the neural bases of complex insect behavior

Recent advances in biocompatible materials, miniaturized instrumentation, advanced computational algorithms, and genetic tools have enabled the development of novel methods and approaches to quantify the behavior of individuals or groups of animals. In conjunction with technologies that allow simultaneous monitoring of neural responses, quantitative studies of complex behaviors can reveal tighter links between the external sensory cues in the vicinity of the organism and neural responses they elicit, and how internal neural representations finally get mapped onto the behavior generated. In this review, we examine a few approaches that are beginning to be widely exploited for understanding neural-behavioral response transformations.

Distinct dynamics of neuronal activity during concurrent motor planning and execution

The smooth conduct of movements requires simultaneous motor planning and execution according to internal goals. So far it remains unknown how such movement plans are modified without interfering with ongoing movements. Previous studies have isolated planning and execution-related neuronal activity by separating behavioral planning and movement periods in time by sensory cues. Here, we separate continuous self-paced motor planning from motor execution statistically, by experimentally minimizing the repetitiveness of the movements. This approach shows that, in the rat sensorimotor cortex, neuronal motor planning processes evolve with slower dynamics than movement-related responses. Fast-evolving neuronal activity precees skilled forelimb movements and is nested within slower dynamics. We capture this effect via high-pass filtering and confirm the results with optogenetic stimulations. The various dynamics combined with adaptation-based high-pass filtering provide a simple principle for separating concurrent motor planning and execution.

Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain

Neural computation has evolved to optimize the behaviors that enable our survival. Although much previous work in neuroscience has focused on constrained task behaviors, recent advances in computer vision are fueling a trend toward the study of naturalistic behaviors. Automated tracking of fine-scale behaviors is generating rich datasets for animal models including rodents, fruit flies, zebrafish, and worms. However, extracting meaning from these large and complex data often requires sophisticated computational techniques. Here we review the latest methods and modeling approaches providing new insights into the brain from behavior. We focus on unsupervised methods for identifying stereotyped behaviors and for resolving details of the structure and dynamics of behavioral sequences.

A framework for studying behavioral evolution by reconstructing ancestral repertoires

Although different animal species often exhibit extensive variation in many behaviors, typically scientists examine one or a small number of behaviors in any single study. Here, we propose a new framework to simultaneously study the evolution of many behaviors. We measured the behavioral repertoire of individuals from six species of fruit flies using unsupervised techniques and identified all stereotyped movements exhibited by each species. We then fit a Generalized Linear Mixed Model to estimate the intra- and inter-species behavioral covariances, and, by using the known phylogenetic relationships among species, we estimated the (unobserved) behaviors exhibited by ancestral species. We found that much of intra-specific behavioral variation has a similar covariance structure to previously described long-time scale variation in an individual's behavior, suggesting that much of the measured variation between individuals of a single species in our assay reflects differences in the status of neural networks, rather than genetic or developmental differences between individuals. We then propose a method to identify groups of behaviors that appear to have evolved in a correlated manner, illustrating how sets of behaviors, rather than individual behaviors, likely evolved. Our approach provides a new framework for identifying co-evolving behaviors and may provide new opportunities to study the mechanistic basis of behavioral evolution.

B-SOiD, an open-source unsupervised algorithm for identification and fast prediction of behaviors

Studying naturalistic animal behavior remains a difficult objective. Recent machine learning advances have enabled limb localization; however, extracting behaviors requires ascertaining the spatiotemporal patterns of these positions. To provide a link from poses to actions and their kinematics, we developed B-SOiD - an open-source, unsupervised algorithm that identifies behavior without user bias. By training a machine classifier on pose pattern statistics clustered using new methods, our approach achieves greatly improved processing speed and the ability to generalize across subjects or labs. Using a frameshift alignment paradigm, B-SOiD overcomes previous temporal resolution barriers. Using only a single, off-the-shelf camera, B-SOiD provides categories of sub-action for trained behaviors and kinematic measures of individual limb trajectories in any animal model. These behavioral and kinematic measures are difficult but critical to obtain, particularly in the study of rodent and other models of pain, OCD, and movement disorders.

Decoding locomotion from population neural activity in moving C. elegans

We investigated the neural representation of locomotion in the nematode C. elegans by recording population calcium activity during movement. We report that population activity more accurately decodes locomotion than any single neuron. Relevant signals are distributed across neurons with diverse tunings to locomotion. Two largely distinct subpopulations are informative for decoding velocity and curvature, and different neurons’ activities contribute features relevant for different aspects of a behavior or different instances of a behavioral motif. To validate our measurements, we labeled neurons AVAL and AVAR and found that their activity exhibited expected transients during backward locomotion. Finally, we compared population activity during movement and immobilization. Immobilization alters the correlation structure of neural activity and its dynamics. Some neurons positively correlated with AVA during movement become negatively correlated during immobilization and vice versa. This work provides needed experimental measurements that inform and constrain ongoing efforts to understand population dynamics underlying locomotion in C. elegans.

Topological Data Analysis of C. elegans Locomotion and Behavior

We apply topological data analysis to the behavior of C. elegans, a widely studied model organism in biology. In particular, we use topology to produce a quantitative summary of complex behavior which may be applied to high-throughput data. Our methods allow us to distinguish and classify videos from various environmental conditions and we analyze the trade-off between accuracy and interpretability. Furthermore, we present a novel technique for visualizing the outputs of our analysis in terms of the input. Specifically, we use representative cycles of persistent homology to produce synthetic videos of stereotypical behaviors.

Body language signals for rodent social communication

Integration of social cues to initiate adaptive emotional and behavioral responses is a fundamental aspect of animal and human behavior. In humans, social communication includes prominent nonverbal components, such as social touch, gestures and facial expressions. Comparative studies investigating the neural basis of social communication in rodents has historically been centered on olfactory signals and vocalizations, with relatively less focus on non-verbal social cues. Here, we outline two exciting research directions: First, we will review recent observations pointing to a role of social facial expressions in rodents. Second, we will review observations that point to a role of 'non-canonical' rodent body language: body posture signals beyond stereotyped displays in aggressive and sexual behavior. In both sections, we will outline how social neuroscience can build on recent advances in machine learning, robotics and micro-engineering to push these research directions forward towards a holistic systems neurobiology of rodent body language.