Quantifying behavior to understand the brain

A Markovian dynamics for Caenorhabditis elegans behavior across scales

How do we capture the breadth of behavior in animal movement, from rapid body twitches to aging? Using high-resolution videos of the nematode worm Caenorhabditis elegans, we show that a single dynamics connects posture-scale fluctuations with trajectory diffusion and longer-lived behavioral states. We take short posture sequences as an instantaneous behavioral measure, fixing the sequence length for maximal prediction. Within the space of posture sequences, we construct a fine-scale, maximum entropy partition so that transitions among microstates define a high-fidelity Markov model, which we also use as a means of principled coarse-graining. We translate these dynamics into movement using resistive force theory, capturing the statistical properties of foraging trajectories. Predictive across scales, we leverage the longest-lived eigenvectors of the inferred Markov chain to perform a top-down subdivision of the worm's foraging behavior, revealing both "runs-and-pirouettes" as well as previously uncharacterized finer-scale behaviors. We use our model to investigate the relevance of these fine-scale behaviors for foraging success, recovering a trade-off between local and global search strategies.

Exploring motion using geometric morphometrics in microscopic aquatic invertebrates: 'modes' and movement patterns during feeding in a bdelloid rotifer model species

BACKGROUND

Movement is a defining aspect of animals, but it is rarely studied using quantitative methods in microscopic invertebrates. Bdelloid rotifers are a cosmopolitan class of aquatic invertebrates of great scientific interest because of their ability to survive in very harsh environment and also because they represent a rare example of an ancient lineage that only includes asexually reproducing species. In this class, Adineta ricciae has become a model species as it is unusually easy to culture. Yet, relatively little is known of its ethology and almost nothing on how it behaves during feeding.

METHODS

To explore feeding behaviour in A. ricciae, as well as to provide an example of application of computational ethology in a microscopic invertebrate, we apply Procrustes motion analysis in combination with ordination and clustering methods to a laboratory bred sample of individuals recorded during feeding.

RESULTS

We demonstrate that movement during feeding can be accurately described in a simple two-dimensional shape space with three main 'modes' of motion. Foot telescoping, with the body kept straight, is the most frequent 'mode', but it is accompanied by periodic rotations of the foot together with bending while the foot is mostly retracted.

CONCLUSIONS

Procrustes motion analysis is a relatively simple but effective tool for describing motion during feeding in A. ricciae. The application of this method generates quantitative data that could be analysed in relation to genetic and ecological differences in a variety of experimental settings. The study provides an example that is easy to replicate in other invertebrates, including other microscopic animals whose behavioural ecology is often poorly known.

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms can flexibly quantify animal movement from videos obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into discrete actions. This challenge is particularly acute because keypoint data are susceptible to high-frequency jitter that clustering algorithms can mistake for transitions between actions. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ('syllables') from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to identify syllables whose boundaries correspond to natural sub-second discontinuities in pose dynamics. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq also works in multiple species and generalizes beyond the syllable timescale, identifying fast sniff-aligned movements in mice and a spectrum of oscillatory behaviors in fruit flies. Keypoint-MoSeq, therefore, renders accessible the modular structure of behavior through standard video recordings.

Computer vision analysis of mother-infant interaction identified efficient pup retrieval in V1b receptor knockout mice

Close contact between lactating rodent mothers and their infants is essential for effective nursing. Whether the mother's effort to retrieve the infants to their nest requires the vasopressin-signaling via V1b receptor has not been fully defined. To address this question, V1b receptor knockout (V1bKO) and control mice were analyzed in pup retrieval test. Because an exploring mother in a new test cage randomly accessed to multiple infants in changing backgrounds over time, a computer vision-based deep learning analysis was applied to continuously calculate the distances between the mother and the infants as a parameter of their relationship. In an open-field, a virgin female V1bKO mice entered fewer times into the center area and moved shorter distances than wild-type (WT). While this behavioral pattern persisted in V1bKO mother, the pup retrieval test demonstrated that total distances between a V1bKO mother and infants came closer in a shorter time than with a WT mother. Moreover, in the medial preoptic area, parts of the V1b receptor transcripts were detected in galanin- and c-fos-positive neurons following maternal stimulation by infants. This research highlights the effectiveness of deep learning analysis in evaluating the mother-infant relationship and the critical role of V1b receptor in pup retrieval during the early lactation phase.

Temporally aligned segmentation and clustering (TASC) framework for behavior time series analysis

Behavior exhibits a complex spatiotemporal structure consisting of discrete sub-behaviors, or motifs. Continuous behavior data requires segmentation and clustering to reveal these embedded motifs. The popularity of automatic behavior quantification is growing, but existing solutions are often tailored to specific needs and are not designed for the time scale and precision required in many experimental and clinical settings. Here we propose a generalized framework with an iterative approach to refine both segmentation and clustering. Temporally aligned segmentation and clustering (TASC) uses temporal linear alignment to compute distances between and align the recurring behavior motifs in a multidimensional time series, enabling precise segmentation and clustering. We introduce an alternating-step process: evaluation of temporal neighbors against current cluster centroids using linear alignment, alternating with selecting the best non-overlapping segments and their subsequent re-clustering. The framework is evaluated on semi-synthetic and real-world experimental and clinical data, demonstrating enhanced segmentation and clustering, offering a better foundation for consequent research. The framework may be used to extend existing tools in the field of behavior research and may be applied to other domains requiring high precision of time series segmentation.

Characterizing the structure of mouse behavior using Motion Sequencing

Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.

Identifying behavioral links to neural dynamics of multifiber photometry recordings in a mouse social behavior network

Objective.Distributed hypothalamic-midbrain neural circuits help orchestrate complex behavioral responses during social interactions. Given rapid advances in optical imaging, it is a fundamental question how population-averaged neural activity measured by multi-fiber photometry (MFP) for calcium fluorescence signals correlates with social behaviors is a fundamental question. This paper aims to investigate the correspondence between MFP data and social behaviors.Approach:We propose a state-space analysis framework to characterize mouse MFP data based on dynamic latent variable models, which include a continuous-state linear dynamical system and a discrete-state hidden semi-Markov model. We validate these models on extensive MFP recordings during aggressive and mating behaviors in male-male and male-female interactions, respectively.Main results:Our results show that these models are capable of capturing both temporal behavioral structure and associated neural states, and produce interpretable latent states. Our approach is also validated in computer simulations in the presence of known ground truth.Significance:Overall, these analysis approaches provide a state-space framework to examine neural dynamics underlying social behaviors and reveals mechanistic insights into the relevant networks.

SuperAnimal pretrained pose estimation models for behavioral analysis

Quantification of behavior is critical in diverse applications from neuroscience, veterinary medicine to animal conservation. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present SuperAnimal, a method to develop unified foundation models that can be used on over 45 species, without additional manual labels. These models show excellent performance across six pose estimation benchmarks. We demonstrate how to fine-tune the models (if needed) on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If fine-tuned, SuperAnimal models are 10-100× more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification and kinematic analysis. Collectively, we present a data-efficient solution for animal pose estimation.

Large-scale foundation models and generative AI for BigData neuroscience

Recent advances in machine learning have led to revolutionary breakthroughs in computer games, image and natural language understanding, and scientific discovery. Foundation models and large-scale language models (LLMs) have recently achieved human-like intelligence thanks to BigData. With the help of self-supervised learning (SSL) and transfer learning, these models may potentially reshape the landscapes of neuroscience research and make a significant impact on the future. Here we present a mini-review on recent advances in foundation models and generative AI models as well as their applications in neuroscience, including natural language and speech, semantic memory, brain-machine interfaces (BMIs), and data augmentation. We argue that this paradigm-shift framework will open new avenues for many neuroscience research directions and discuss the accompanying challenges and opportunities.

Formalising the role of behaviour in neuroscience

We develop a mathematical approach to formally proving that certain neural computations and representations exist based on patterns observed in an organism's behaviour. To illustrate, we provide a simple set of conditions under which an ant's ability to determine how far it is from its nest would logically imply neural structures isomorphic to the natural numbersℕ $$ \mathrm{\mathbb{N}} $$ . We generalise these results to arbitrary behaviours and representations and show what mathematical characterisation of neural computation and representation is simplest while being maximally predictive of behaviour. We develop this framework in detail using a path integration example, where an organism's ability to search for its nest in the correct location implies representational structures isomorphic to two-dimensional coordinates under addition. We also study a system for processinga n b n $$ {a}^n{b}^n $$ strings common in comparative work. Our approach provides an objective way to determine what theory of a physical system is best, addressing a fundamental challenge in neuroscientific inference. These results motivate considering which neurobiological structures have the requisite formal structure and are otherwise physically plausible given relevant physical considerations such as generalisability, information density, thermodynamic stability and energetic cost.

Toward a neuroscience of natural behavior

One of the most exciting new developments in systems neuroscience is the progress being made toward neurophysiological experiments that move beyond simplified laboratory settings and address the richness of natural behavior. This is enabled by technological advances such as wireless recording in freely moving animals, automated quantification of behavior, and new methods for analyzing large data sets. Beyond new empirical methods and data, however, there is also a need for new theories and concepts to interpret that data. Such theories need to address the particular challenges of natural behavior, which often differ significantly from the scenarios studied in traditional laboratory settings. Here, we discuss some strategies for developing such novel theories and concepts and some example hypotheses being proposed.

An ethologically motivated neurobiology of primate visually-guided reach-to-grasp behavior

The precision of primate visually guided reaching likely evolved to meet the many challenges faced by living in arboreal environments, yet much of what we know about the underlying primate brain organization derives from a set of highly constrained experimental paradigms. Here we review the role of vision to guide natural reach-to-grasp movements in marmoset monkey prey capture to illustrate the breadth and diversity of these behaviors in ethological contexts, the fast predictive nature of these movements [1,2], and the advantages of this particular primate model to investigate the underlying neural mechanisms in more naturalistic contexts [3]. In addition to their amenability to freely-moving neural recording methods for investigating the neural basis of dynamic ethological behaviors [4,5], marmosets have a smooth neocortical surface that facilitates imaging and array recordings [6,7] in all areas in the primate fronto-parietal network [8,9]. Together, this model organism offers novel opportunities to study the real-world interplay between primate vision and reach-to-grasp dynamics using ethologically motivated neuroscientific experimental designs.

Descending networks transform command signals into population motor control

To convert intentions into actions, movement instructions must pass from the brain to downstream motor circuits through descending neurons (DNs). These include small sets of command-like neurons that are sufficient to drive behaviours1-the circuit mechanisms for which remain unclear. Here we show that command-like DNs in Drosophila directly recruit networks of additional DNs to orchestrate behaviours that require the active control of numerous body parts. Specifically, we found that command-like DNs previously thought to drive behaviours alone2-4 in fact co-activate larger populations of DNs. Connectome analyses and experimental manipulations revealed that this functional recruitment can be explained by direct excitatory connections between command-like DNs and networks of interconnected DNs in the brain. Descending population recruitment is necessary for behavioural control: DNs with many downstream descending partners require network co-activation to drive complete behaviours and drive only simple stereotyped movements in their absence. These DN networks reside within behaviour-specific clusters that inhibit one another. These results support a mechanism for command-like descending control in which behaviours are generated through the recruitment of increasingly large DN networks that compose behaviours by combining multiple motor subroutines.

Visual detection of seizures in mice using supervised machine learning

Seizures are caused by abnormally synchronous brain activity that can result in changes in muscle tone, such as twitching, stiffness, limpness, or rhythmic jerking. These behavioral manifestations are clear on visual inspection and the most widely used seizure scoring systems in preclinical models, such as the Racine scale in rodents, use these behavioral patterns in semiquantitative seizure intensity scores. However, visual inspection is time-consuming, low-throughput, and partially subjective, and there is a need for rigorously quantitative approaches that are scalable. In this study, we used supervised machine learning approaches to develop automated classifiers to predict seizure severity directly from noninvasive video data. Using the PTZ-induced seizure model in mice, we trained video-only classifiers to predict ictal events, combined these events to predict an univariate seizure intensity for a recording session, as well as time-varying seizure intensity scores. Our results show, for the first time, that seizure events and overall intensity can be rigorously quantified directly from overhead video of mice in a standard open field using supervised approaches. These results enable high-throughput, noninvasive, and standardized seizure scoring for downstream applications such as neurogenetics and therapeutic discovery.

Uncovering multiscale structure in the variability of larval zebrafish navigation

Animals chain movements into long-lived motor strategies, exhibiting variability across scales that reflects the interplay between internal states and environmental cues. To reveal structure in such variability, we build Markov models of movement sequences that bridges across time scales and enables a quantitative comparison of behavioral phenotypes among individuals. Applied to larval zebrafish responding to diverse sensory cues, we uncover a hierarchy of long-lived motor strategies, dominated by changes in orientation distinguishing cruising versus wandering strategies. Environmental cues induce preferences along these modes at the population level: while fish cruise in the light, they wander in response to aversive stimuli, or in search for appetitive prey. As our method encodes the behavioral dynamics of each individual fish in the transitions among coarse-grained motor strategies, we use it to uncover a hierarchical structure in the phenotypic variability that reflects exploration-exploitation trade-offs. Across a wide range of sensory cues, a major source of variation among fish is driven by prior and/or immediate exposure to prey that induces exploitation phenotypes. A large degree of variability that is not explained by environmental cues unravels motivational states that override the sensory context to induce contrasting exploration-exploitation phenotypes. Altogether, by extracting the timescales of motor strategies deployed during navigation, our approach exposes structure among individuals and reveals internal states tuned by prior experience.

SAUSI: a novel assay for measuring social anxiety and motivation

Social anxiety is one of the most prevalent mental health disorders, though the underlying neurobiology is poorly understood. Progress in understanding the etiology of social anxiety has been hindered by the lack of comprehensive tools to assess social anxiety in model systems. Here, we created a new behavioral task - Selective Access to Unrestricted Social Interaction (SAUSI), which combines elements of social motivation, hesitancy, decision-making, and free interaction to enable the wholistic assessment of social anxiety-like behaviors in mice. Using this novel assay, we found that social isolation-induced social anxiety-like behaviors in female mice are largely driven by increases in social fear, social hesitancy, and altered ultrasonic vocalizations. Deep learning analyses were able to computationally identify a unique behavioral footprint underlying the state produced by social isolation, demonstrating the compatibility of modern computational approaches with SAUSI. Finally, we compared the results of SAUSI to traditionally social assays including the 3-chamber sociability assay and the resident intruder task. This revealed that behavioral changes induced by isolation were highly context dependent, and that while fragments of social anxiety measured in SAUSI were replicable across other tasks, a wholistic assessment was not obtainable from these alternative assays. Our findings debut a novel task for the behavioral toolbox - one which overcomes limitations of previous assays, allowing for both social choice as well as free interaction, and offers a new approach for assessing social anxiety in rodents.

Control over a mixture of policies determines change of mind topology during continuous choice

Behavior is naturally organized into categorically distinct states with corresponding patterns of neural activity; how does the brain control those states? We propose that states are regulated by specific neural processes that implement meta-control that can blend simpler control processes. To test this hypothesis, we recorded from neurons in the dorsal anterior cingulate cortex (dACC) and dorsal premotor cortex (PMd) while macaques performed a continuous pursuit task with two moving prey that followed evasive strategies. We used a novel control theoretic approach to infer subjects' moment-to-moment latent control variables, which in turn dictated their blend of distinct identifiable control processes. We identified low-dimensional subspaces in neuronal responses that reflected the current strategy, the value of the pursued target, and the relative value of the two targets. The top two principal components of activity tracked changes of mind in abstract and change-type-specific formats, respectively. These results indicate that control of behavioral state reflects the interaction of brain processes found in dorsal prefrontal regions that implement a mixture over low-level control policies.

Fluctuating landscapes and heavy tails in animal behavior

Animal behavior is shaped by a myriad of mechanisms acting on a wide range of scales, which hampers quantitative reasoning and the identification of general principles. Here, we combine data analysis and theory to investigate the relationship between behavioral plasticity and heavy-tailed statistics often observed in animal behavior. Specifically, we first leverage high-resolution recordings of C. elegans locomotion to show that stochastic transitions among long-lived behaviors exhibit heavy-tailed first passage time distributions and correlation functions. Such heavy tails can be explained by slow adaptation of behavior over time. This particular result motivates our second step of introducing a general model where we separate fast dynamics on a quasi-stationary multi-well potential, from non-ergodic, slowly varying modes. We then show that heavy tails generically emerge in such a model, and we provide a theoretical derivation of the resulting functional form, which can become a power law with exponents that depend on the strength of the fluctuations. Finally, we provide direct support for the generality of our findings by testing them in a C. elegans mutant where adaptation is suppressed and heavy tails thus disappear, and recordings of larval zebrafish swimming behavior where heavy tails are again prevalent.

A Method for Assessing Dogs in a Test Evaluating Dogs' Suitability for Animal-Assisted Education

In objective behavioral rating (BR), dogs are rated for behavior based on the observations made during a behavioral test. However, behavioral definitions can differ between raters, and the definitions proposed by artificial intelligence (AI) can help standardize the applied measures. The aim of this study was to propose a new approach for assessing dogs' suitability for animal-assisted education (AAE) with the use of the BR method and the definitions of dog behaviors proposed by AI. The analysis relied on video footage of 25 dogs that participated in tests assessing the animals' suitability for AAE. The dogs were rated by five independent observers. Inter-rater reliability was satisfactory in 7 out of 9 subtests (0.715-0.856) and low in the remaining 2 subtests (0.667 in Subtest 2 and 0.617 in Subtest 5). The results indicate that the BR method based on AI definitions produced satisfactory inter-rater reliability in a test evaluating dog behavior. A comparison of subtest scores in the BR method and the subjective rating method did not reveal significant differences. Thus, it can be concluded that the proposed approach gives promising outcomes and can be used to develop new tests for assessing dogs' suitability for AAE and, perhaps, other types of work.

Separating cognitive and motor processes in the behaving mouse

The cognitive processes supporting complex animal behavior are closely associated with ubiquitous movements responsible for our posture, facial expressions, ability to actively sample our sensory environments, and other critical processes. These movements are strongly related to neural activity across much of the brain and are often highly correlated with ongoing cognitive processes, making it challenging to dissociate the neural dynamics that support cognitive processes from those supporting related movements. In such cases, a critical issue is whether cognitive processes are separable from related movements, or if they are driven by common neural mechanisms. Here, we demonstrate how the separability of cognitive and motor processes can be assessed, and, when separable, how the neural dynamics associated with each component can be isolated. We establish a novel two-context behavioral task in mice that involves multiple cognitive processes and show that commonly observed dynamics taken to support cognitive processes are strongly contaminated by movements. When cognitive and motor components are isolated using a novel approach for subspace decomposition, we find that they exhibit distinct dynamical trajectories. Further, properly accounting for movement revealed that largely separate populations of cells encode cognitive and motor variables, in contrast to the 'mixed selectivity' often reported. Accurately isolating the dynamics associated with particular cognitive and motor processes will be essential for developing conceptual and computational models of neural circuit function and evaluating the function of the cell types of which neural circuits are composed.

Representing the dynamics of natural marmoset vocal behaviors in frontal cortex

Here we tested the respective contributions of primate premotor and prefrontal cortex to support vocal behavior. We applied a model-based GLM analysis that better accounts for the inherent variance in natural, continuous behaviors to characterize the activity of neurons throughout frontal cortex as freely-moving marmosets engaged in conversational exchanges. While analyses revealed functional clusters of neural activity related to the different processes involved in the vocal behavior, these clusters did not map to subfields of prefrontal or premotor cortex, as has been observed in more conventional task-based paradigms. Our results suggest a distributed functional organization for the myriad neural mechanisms underlying natural social interactions and has implications for our concepts of the role that frontal cortex plays in governing ethological behaviors in primates.

Behavioral effects of visual stimuli in adult zebrafish using a novel eight-tank imaging system

Introduction

Animals respond to various environmental cues. Animal behavior is complex, and behavior analysis can greatly help to understand brain function. Most of the available behavioral imaging setups are expensive, provide limited options for customization, and allow for behavioral imaging of one animal at a time.

Methods

The current study takes advantage of adult zebrafish as a model organism to study behavior in a novel behavioral setup allowing one to concurrently image 8 adult zebrafish.

Results

Our results indicate that adult zebrafish show a unique behavioral profile in response to visual stimuli such as moving lines. In the presence of moving lines, the fish spent more time exploring the tank and spent more time toward the edges of the tanks. In addition, the fish moved and oriented themselves against the direction of the moving lines, indicating a negative optomotor response (OMR). With repeated exposure to moving lines, we observed a reduced optomotor response in adult zebrafish.

Discussion

Our behavioral setup is relatively inexpensive, provides flexibility in the presentation of various animated visual stimuli, and offers improved throughput for analyzing behavior in adult zebrafish. This behavioral setup shows promising potential to quantify various behavioral measures and opens new avenues to understand complex behaviors.

Tracking subjects' strategies in behavioural choice experiments at trial resolution

Investigating how, when, and what subjects learn during decision-making tasks requires tracking their choice strategies on a trial-by-trial basis. Here, we present a simple but effective probabilistic approach to tracking choice strategies at trial resolution using Bayesian evidence accumulation. We show this approach identifies both successful learning and the exploratory strategies used in decision tasks performed by humans, non-human primates, rats, and synthetic agents. Both when subjects learn and when rules change the exploratory strategies of win-stay and lose-shift, often considered complementary, are consistently used independently. Indeed, we find the use of lose-shift is strong evidence that subjects have latently learnt the salient features of a new rewarded rule. Our approach can be extended to any discrete choice strategy, and its low computational cost is ideally suited for real-time analysis and closed-loop control.

ReptiLearn: An automated home cage system for behavioral experiments in reptiles without human intervention

Understanding behavior and its evolutionary underpinnings is crucial for unraveling the complexities of brain function. Traditional approaches strive to reduce behavioral complexity by designing short-term, highly constrained behavioral tasks with dichotomous choices in which animals respond to defined external perturbation. In contrast, natural behaviors evolve over multiple time scales during which actions are selected through bidirectional interactions with the environment and without human intervention. Recent technological advancements have opened up new possibilities for experimental designs that more closely mirror natural behaviors by replacing stringent experimental control with accurate multidimensional behavioral analysis. However, these approaches have been tailored to fit only a small number of species. This specificity limits the experimental opportunities offered by species diversity. Further, it hampers comparative analyses that are essential for extracting overarching behavioral principles and for examining behavior from an evolutionary perspective. To address this limitation, we developed ReptiLearn-a versatile, low-cost, Python-based solution, optimized for conducting automated long-term experiments in the home cage of reptiles, without human intervention. In addition, this system offers unique features such as precise temperature measurement and control, live prey reward dispensers, engagement with touch screens, and remote control through a user-friendly web interface. Finally, ReptiLearn incorporates low-latency closed-loop feedback allowing bidirectional interactions between animals and their environments. Thus, ReptiLearn provides a comprehensive solution for researchers studying behavior in ectotherms and beyond, bridging the gap between constrained laboratory settings and natural behavior in nonconventional model systems. We demonstrate the capabilities of ReptiLearn by automatically training the lizard Pogona vitticeps on a complex spatial learning task requiring association learning, displaced reward learning, and reversal learning.

Grounding neuroscience in behavioral changes using artificial neural networks

Connecting neural activity to function is a common aim in neuroscience. How to define and conceptualize function, however, can vary. Here I focus on grounding this goal in the specific question of how a given change in behavior is produced by a change in neural circuits or activity. Artificial neural network models offer a particularly fruitful format for tackling such questions because they use neural mechanisms to perform complex transformations and produce appropriate behavior. Therefore, they can be a means of causally testing the extent to which a neural change can be responsible for an experimentally observed behavioral change. Furthermore, because the field of interpretability in artificial intelligence has similar aims, neuroscientists can look to interpretability methods for new ways of identifying neural features that drive performance and behaviors.

Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings

Accurate detection and tracking of animals across diverse environments are crucial for behavioral studies in various disciplines, including neuroscience. Recently, machine learning and computer vision techniques have become integral to the neuroscientist's toolkit, enabling high-throughput behavioral studies. Despite advancements in localizing individual animals in simple environments, the task remains challenging in complex conditions due to intra-class visual variability and environmental diversity. These limitations hinder studies in ethologically-relevant conditions, such as when animals are concealed within nests or in obscured environments. Moreover, current tools are laborious and time-consuming to employ, requiring extensive, setup-specific annotation and model training/validation procedures. To address these challenges, we introduce the 'Detect Any Mouse Model' (DAMM), a pretrained object detector for localizing mice in complex environments, capable of robust performance with zero to minimal additional training on new experimental setups. Our approach involves collecting and annotating a diverse dataset that encompasses single and multi-housed mice in various lighting conditions, experimental setups, and occlusion levels. We utilize the Mask R-CNN architecture for instance segmentation and validate DAMM's performance with no additional training data (zero-shot inference) and with few examples for fine-tuning (few-shot inference). DAMM excels in zero-shot inference, detecting mice, and even rats, in entirely unseen scenarios and further improves with minimal additional training. By integrating DAMM with the SORT algorithm, we demonstrate robust tracking, competitively performing with keypoint-estimation-based methods. Finally, to advance and simplify behavioral studies, we made DAMM accessible to the scientific community with a user-friendly Python API, shared model weights, and a Google Colab implementation.

Identifying behavioral links to neural dynamics of multifiber photometry recordings in a mouse social behavior network

Distributed hypothalamic-midbrain neural circuits orchestrate complex behavioral responses during social interactions. How population-averaged neural activity measured by multi-fiber photometry (MFP) for calcium fluorescence signals correlates with social behaviors is a fundamental question. We propose a state-space analysis framework to characterize mouse MFP data based on dynamic latent variable models, which include continuous-state linear dynamical system (LDS) and discrete-state hidden semi-Markov model (HSMM). We validate these models on extensive MFP recordings during aggressive and mating behaviors in male-male and male-female interactions, respectively. Our results show that these models are capable of capturing both temporal behavioral structure and associated neural states. Overall, these analysis approaches provide an unbiased strategy to examine neural dynamics underlying social behaviors and reveals mechanistic insights into the relevant networks.

Non-consummatory behavior signals predict aversion-resistant alcohol drinking in head-fixed mice

A key facet of alcohol use disorder is continuing to drink alcohol despite negative consequences (so called "aversion-resistant drinking"). In this study, we sought to assess the degree to which head-fixed mice exhibit aversion-resistant drinking and to leverage behavioral analysis techniques available in head-fixture to relate non-consummatory behaviors to aversion-resistant drinking. We assessed aversion-resistant drinking in head-fixed female and male C57BL/6 J mice. We adulterated 20% (v/v) alcohol with varying concentrations of the bitter tastant quinine to measure the degree to which mice would continue to drink despite this aversive stimulus. We recorded high-resolution video of the mice during head-fixed drinking, tracked body parts with machine vision tools, and analyzed body movements in relation to consumption. Female and male head-fixed mice exhibited heterogenous levels of aversion-resistant drinking. Additionally, non-consummatory behaviors, such as paw movement and snout movement, were related to the intensity of aversion-resistant drinking. These studies demonstrate that head-fixed mice exhibit aversion-resistant drinking and that non-consummatory behaviors can be used to assess perceived aversiveness in this paradigm. Furthermore, these studies lay the groundwork for future experiments that will utilize advanced electrophysiological techniques to record from large populations of neurons during aversion-resistant drinking to understand the neurocomputational processes that drive this clinically relevant behavior. This article is part of the Special Issue on "PFC circuit function in psychiatric disease and relevant models".

Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

Keypoint tracking algorithms have revolutionized the analysis of animal behavior, enabling investigators to flexibly quantify behavioral dynamics from conventional video recordings obtained in a wide variety of settings. However, it remains unclear how to parse continuous keypoint data into the modules out of which behavior is organized. This challenge is particularly acute because keypoint data is susceptible to high frequency jitter that clustering algorithms can mistake for transitions between behavioral modules. Here we present keypoint-MoSeq, a machine learning-based platform for identifying behavioral modules ("syllables") from keypoint data without human supervision. Keypoint-MoSeq uses a generative model to distinguish keypoint noise from behavior, enabling it to effectively identify syllables whose boundaries correspond to natural sub-second discontinuities inherent to mouse behavior. Keypoint-MoSeq outperforms commonly used alternative clustering methods at identifying these transitions, at capturing correlations between neural activity and behavior, and at classifying either solitary or social behaviors in accordance with human annotations. Keypoint-MoSeq therefore renders behavioral syllables and grammar accessible to the many researchers who use standard video to capture animal behavior.

OpenApePose, a database of annotated ape photographs for pose estimation

Because of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based pose tracking. Tracking, however, requires high-quality annotated datasets of ape photographs. Here we present OpenApePose, a new public dataset of 71,868 photographs, annotated with 16 body landmarks of six ape species in naturalistic contexts. We show that a standard deep net (HRNet-W48) trained on ape photos can reliably track out-of-sample ape photos better than networks trained on monkeys (specifically, the OpenMonkeyPose dataset) and on humans (COCO) can. This trained network can track apes almost as well as the other networks can track their respective taxa, and models trained without one of the six ape species can track the held-out species better than the monkey and human models can. Ultimately, the results of our analyses highlight the importance of large, specialized databases for animal tracking systems and confirm the utility of our new ape database.

SaLSa: A Combinatory Approach of Semi-Automatic Labeling and Long Short-Term Memory to Classify Behavioral Syllables

Accurately and quantitatively describing mouse behavior is an important area. Although advances in machine learning have made it possible to track their behaviors accurately, reliable classification of behavioral sequences or syllables remains a challenge. In this study, we present a novel machine learning approach, called SaLSa (a combination of semi-automatic labeling and long short-term memory-based classification), to classify behavioral syllables of mice exploring an open field. This approach consists of two major steps. First, after tracking multiple body parts, spatial and temporal features of their egocentric coordinates are extracted. A fully automated unsupervised process identifies candidates for behavioral syllables, followed by manual labeling of behavioral syllables using a graphical user interface (GUI). Second, a long short-term memory (LSTM) classifier is trained with the labeled data. We found that the classification performance was marked over 97%. It provides a performance equivalent to a state-of-the-art model while classifying some of the syllables. We applied this approach to examine how hyperactivity in a mouse model of Alzheimer's disease develops with age. When the proportion of each behavioral syllable was compared between genotypes and sexes, we found that the characteristic hyperlocomotion of female Alzheimer's disease mice emerges between four and eight months. In contrast, age-related reduction in rearing is common regardless of genotype and sex. Overall, SaLSa enables detailed characterization of mouse behavior.

The enigmatic HCN channels: A cellular neurophysiology perspective

What physiological role does a slow hyperpolarization-activated ion channel with mixed cation selectivity play in the fast world of neuronal action potentials that are driven by depolarization? That puzzling question has piqued the curiosity of physiology enthusiasts about the hyperpolarization-activated cyclic nucleotide-gated (HCN) channels, which are widely expressed across the body and especially in neurons. In this review, we emphasize the need to assess HCN channels from the perspective of how they respond to time-varying signals, while also accounting for their interactions with other co-expressing channels and receptors. First, we illustrate how the unique structural and functional characteristics of HCN channels allow them to mediate a slow negative feedback loop in the neurons that they express in. We present the several physiological implications of this negative feedback loop to neuronal response characteristics including neuronal gain, voltage sag and rebound, temporal summation, membrane potential resonance, inductive phase lead, spike triggered average, and coincidence detection. Next, we argue that the overall impact of HCN channels on neuronal physiology critically relies on their interactions with other co-expressing channels and receptors. Interactions with other channels allow HCN channels to mediate intrinsic oscillations, earning them the "pacemaker channel" moniker, and to regulate spike frequency adaptation, plateau potentials, neurotransmitter release from presynaptic terminals, and spike initiation at the axonal initial segment. We also explore the impact of spatially non-homogeneous subcellular distributions of HCN channels in different neuronal subtypes and their interactions with other channels and receptors. Finally, we discuss how plasticity in HCN channels is widely prevalent and can mediate different encoding, homeostatic, and neuroprotective functions in a neuron. In summary, we argue that HCN channels form an important class of channels that mediate a diversity of neuronal functions owing to their unique gating kinetics that made them a puzzle in the first place.

Fluctuating landscapes and heavy tails in animal behavior

Animal behavior is shaped by a myriad of mechanisms acting on a wide range of scales. This immense variability hampers quantitative reasoning and renders the identification of universal principles elusive. Through data analysis and theory, we here show that slow non-ergodic drives generally give rise to heavy-tailed statistics in behaving animals. We leverage high-resolution recordings of C. elegans locomotion to extract a self-consistent reduced order model for an inferred reaction coordinate, bridging from sub-second chaotic dynamics to long-lived stochastic transitions among metastable states. The slow mode dynamics exhibits heavy-tailed first passage time distributions and correlation functions, and we show that such heavy tails can be explained by dynamics on a time-dependent potential landscape. Inspired by these results, we introduce a generic model in which we separate faster mixing modes that evolve on a quasi-stationary potential, from slower non-ergodic modes that drive the potential landscape, and reflect slowly varying internal states. We show that, even for simple potential landscapes, heavy tails emerge when barrier heights fluctuate slowly and strongly enough. In particular, the distribution of first passage times and the correlation function can asymptote to a power law, with related exponents that depend on the strength and nature of the fluctuations. We support our theoretical findings through direct numerical simulations.

Non-Consummatory Behavior Signals Predict Aversion-Resistant Alcohol Drinking in Head-Fixed Mice

A key facet of alcohol use disorder is continuing to drink alcohol despite negative consequences (so called "aversion-resistant drinking"). In this study, we sought to assess the degree to which head-fixed mice exhibit aversion-resistant drinking and to leverage behavioral analysis techniques available in head-fixture to relate non-consummatory behaviors to aversion-resistant drinking. We assessed aversion-resistant drinking in head-fixed female and male C57BL/6J mice. We adulterated 20% (v/v) alcohol with varying concentrations of the bitter tastant quinine to measure the degree to which mice would continue to drink despite this aversive stimulus. We recorded high-resolution video of the mice during head-fixed drinking, tracked body parts with machine vision tools, and analyzed body movements in relation to consumption. Female and male head-fixed mice exhibited heterogenous levels of aversion-resistant drinking. Additionally, non-consummatory behaviors, such as paw movement and snout movement, were related to the intensity of aversion-resistant drinking. These studies demonstrate that head-fixed mice exhibit aversion-resistant drinking and that non-consummatory behaviors can be used to assess perceived aversiveness in this paradigm. Furthermore, these studies lay the groundwork for future experiments that will utilize advanced electrophysiological techniques to record from large populations of neurons during aversion-resistant drinking to understand the neurocomputational processes that drive this clinically relevant behavior.

Rapid automated 3-D pose estimation of larval zebrafish using a physical model-trained neural network

Quantitative ethology requires an accurate estimation of an organism's postural dynamics in three dimensions plus time. Technological progress over the last decade has made animal pose estimation in challenging scenarios possible with unprecedented detail. Here, we present (i) a fast automated method to record and track the pose of individual larval zebrafish in a 3-D environment, applicable when accurate human labeling is not possible; (ii) a rich annotated dataset of 3-D larval poses for ethologists and the general zebrafish and machine learning community; and (iii) a technique to generate realistic, annotated larval images in different behavioral contexts. Using a three-camera system calibrated with refraction correction, we record diverse larval swims under free swimming conditions and in response to acoustic and optical stimuli. We then employ a convolutional neural network to estimate 3-D larval poses from video images. The network is trained against a set of synthetic larval images rendered using a 3-D physical model of larvae. This 3-D model samples from a distribution of realistic larval poses that we estimate a priori using a template-based pose estimation of a small number of swim bouts. Our network model, trained without any human annotation, performs larval pose estimation three orders of magnitude faster and with accuracy comparable to the template-based approach, capturing detailed kinematics of 3-D larval swims. It also applies accurately to other datasets collected under different imaging conditions and containing behavioral contexts not included in our training.

Hierarchical action encoding in prefrontal cortex of freely moving macaques

Our natural behavioral repertoires include coordinated actions of characteristic types. To better understand how neural activity relates to the expression of actions and action switches, we studied macaques performing a freely moving foraging task in an open environment. We developed a novel analysis pipeline that can identify meaningful units of behavior, corresponding to recognizable actions such as sitting, walking, jumping, and climbing. On the basis of transition probabilities between these actions, we found that behavior is organized in a modular and hierarchical fashion. We found that, after regressing out many potential confounders, actions are associated with specific patterns of firing in each of six prefrontal brain regions and that, overall, encoding of action category is progressively stronger in more dorsal and more caudal prefrontal regions. Together, these results establish a link between selection of units of primate behavior on one hand and neuronal activity in prefrontal regions on the other.

Deep autoencoder-based behavioral pattern recognition outperforms standard statistical methods in high-dimensional zebrafish studies

Zebrafish have become an essential tool in screening for developmental neurotoxic chemicals and their molecular targets. The success of zebrafish as a screening model is partially due to their physical characteristics including their relatively simple nervous system, rapid development, experimental tractability, and genetic diversity combined with technical advantages that allow for the generation of large amounts of high-dimensional behavioral data. These data are complex and require advanced machine learning and statistical techniques to comprehensively analyze and capture spatiotemporal responses. To accomplish this goal, we have trained semi-supervised deep autoencoders using behavior data from unexposed larval zebrafish to extract quintessential "normal" behavior. Following training, our network was evaluated using data from larvae shown to have significant changes in behavior (using a traditional statistical framework) following exposure to toxicants that include nanomaterials, aromatics, per- and polyfluoroalkyl substances (PFAS), and other environmental contaminants. Further, our model identified new chemicals (Perfluoro-n-octadecanoic acid, 8-Chloroperfluorooctylphosphonic acid, and Nonafluoropentanamide) as capable of inducing abnormal behavior at multiple chemical-concentrations pairs not captured using distance moved alone. Leveraging this deep learning model will allow for better characterization of the different exposure-induced behavioral phenotypes, facilitate improved genetic and neurobehavioral analysis in mechanistic determination studies and provide a robust framework for analyzing complex behaviors found in higher-order model systems.

Identification of interacting neural populations: methods and statistical considerations

As improved recording technologies have created new opportunities for neurophysiological investigation, emphasis has shifted from individual neurons to multiple populations that form circuits, and it has become important to provide evidence of cross-population coordinated activity. We review various methods for doing so, placing them in six major categories while avoiding technical descriptions and instead focusing on high-level motivations and concerns. Our aim is to indicate what the methods can achieve and the circumstances under which they are likely to succeed. Toward this end, we include a discussion of four cross-cutting issues: the definition of neural populations, trial-to-trial variability and Poisson-like noise, time-varying dynamics, and causality.

A unified open-source platform for multimodal neural recording and perturbation during naturalistic behavior

Behavioral neuroscience faces two conflicting demands: long-duration recordings from large neural populations and unimpeded animal behavior. To meet this challenge, we developed ONIX, an open-source data acquisition system with high data throughput (2GB/sec) and low closed-loop latencies (<1ms) that uses a novel 0.3 mm thin tether to minimize behavioral impact. Head position and rotation are tracked in 3D and used to drive active commutation without torque measurements. ONIX can acquire from combinations of passive electrodes, Neuropixels probes, head-mounted microscopes, cameras, 3D-trackers, and other data sources. We used ONIX to perform uninterrupted, long (~7 hours) neural recordings in mice as they traversed complex 3-dimensional terrain. ONIX allowed exploration with similar mobility as non-implanted animals, in contrast to conventional tethered systems which restricted movement. By combining long recordings with full mobility, our technology will enable new progress on questions that require high-quality neural recordings during ethologically grounded behaviors.

Beyond simple laboratory studies: Developing sophisticated models to study rich behavior

Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.

Classifying interpersonal synchronization states using a data-driven approach: implications for social interaction understanding

This study presents a data-driven approach to identifying interpersonal motor synchrony states by analyzing hand movements captured from a 3D depth camera. Utilizing a single frame from the experiment, an XGBoost machine learning model was employed to differentiate between spontaneous and intentional synchrony modes with nearly [Formula: see text] accuracy. Our findings demonstrate a consistent pattern across subjects, revealing that movement velocity tends to be slower in synchrony modes. These insights support the notion that the relationship between velocity and synchrony is influenced by the cognitive load required for the task, with slower movements leading to higher synchrony in tasks demanding higher cognitive load. This work not only contributes to the limited literature on algorithms for identifying interpersonal synchrony but also has potential implications for developing new metrics to assess real-time human social interactions, understanding social interaction, and diagnosing and developing treatment strategies for social deficits associated with conditions such as Autism Spectrum Disorder.

Prediction of cognitive scores by joint use of movie-watching fMRI connectivity and eye tracking via Attention-CensNet

Background

Brain functional connectivity under the naturalistic paradigm has been shown to be better at predicting individual behaviors than other brain states, such as rest and doing tasks. Nevertheless, the state-of-the-art methods have found it difficult to achieve desirable results from movie-watching paradigm functional magnetic resonance imaging (mfMRI) -induced brain functional connectivity, especially when there are fewer datasets. Incorporating other physical measurements into the prediction method may enhance accuracy. Eye tracking, becoming popular due to its portability and lower expense, can provide abundant behavioral features related to the output of human's cognition, and thus might supplement the mfMRI in observing participants' subconscious behaviors. However, there are very few studies on how to effectively integrate the multimodal information to strengthen the performance by a unified framework.

Objective

A fusion approach with mfMRI and eye tracking, based on convolution with edge-node switching in graph neural networks (CensNet), is proposed in this article.

Methods

In this graph model, participants are designated as nodes, mfMRI derived functional connectivity as node features, and different eye-tracking features are used to compute similarity between participants to construct heterogeneous graph edges. By taking multiple graphs as different channels, we introduce squeeze-and-excitation attention module to CensNet (A-CensNet) to integrate graph embeddings from multiple channels into one.

Results

The proposed model outperforms those using a single modality and single channel, and state-of-the-art methods.

Conclusions

The results indicate that brain functional activities and eye behaviors might complement each other in interpreting trait-like phenotypes.

Early-life sleep disruption impairs subtle social behaviours in prairie voles: a pose-estimation study

Early-life sleep disruption (ELSD) has been shown to have long-lasting effects on social behaviour in adult prairie voles (Microtus ochrogaster), including impaired expression of pair bonding during partner preference testing. However, due to the limitations of manual behaviour tracking, the effects of ELSD across the time course of pair bonding have not yet been described, hindering our ability to trace mechanisms. Here, we used pose estimation to track prairie voles during opposite-sex cohabitation, the process leading to pair bonding. Male-female pairs were allowed to interact through a mesh divider in the home cage for 72 h, providing variables of body direction, distance-to-divider and locomotion speed. We found that control males displayed periodic patterns of body orientation towards females during cohabitation. In contrast, ELSD males showed reduced duration and ultradian periodicity of these body orientation behaviours towards females. Furthermore, in both sexes, ELSD altered spatial and temporal patterns of locomotion across the light/dark cycles of the 72 h recordings. This study allows a comprehensive behavioural assessment of the effects of ELSD on later life sociality and highlights subtle prairie vole behaviours. Our findings may shed light on neurodevelopmental disorders featuring sleep disruption and social deficits, such as autism spectrum disorders.

Hierarchical organization of rhesus macaque behavior

Primatologists, psychologists and neuroscientists have long hypothesized that primate behavior is highly structured. However, delineating that structure has been impossible due to the difficulties of precision behavioral tracking. Here we analyzed a dataset consisting of continuous measures of the 3D position of two male rhesus macaques (Macaca mulatta) performing three different tasks in a large unrestrained environment over several hours. Using an unsupervised embedding approach on the tracked joints, we identified commonly repeated pose patterns, which we call postures. We found that macaques' behavior is characterized by 49 distinct postures, lasting an average of 0.6 seconds. We found evidence that behavior is hierarchically organized, in that transitions between poses tend to occur within larger modules, which correspond to identifiable actions; these actions are further organized hierarchically. Our behavioral decomposition allows us to identify universal (cross-individual and cross-task) and unique (specific to each individual and task) principles of behavior. These results demonstrate the hierarchical nature of primate behavior, provide a method for the automated ethogramming of primate behavior, and provide important constraints on neural models of pose generation.

AlphaTracker: a multi-animal tracking and behavioral analysis tool

Computer vision has emerged as a powerful tool to elevate behavioral research. This protocol describes a computer vision machine learning pipeline called AlphaTracker, which has minimal hardware requirements and produces reliable tracking of multiple unmarked animals, as well as behavioral clustering. AlphaTracker pairs a top-down pose-estimation software combined with unsupervised clustering to facilitate behavioral motif discovery that will accelerate behavioral research. All steps of the protocol are provided as open-source software with graphic user interfaces or implementable with command-line prompts. Users with a graphical processing unit (GPU) can model and analyze animal behaviors of interest in less than a day. AlphaTracker greatly facilitates the analysis of the mechanism of individual/social behavior and group dynamics.

The lost art of mathematical modelling

We provide a critique of mathematical biology in light of rapid developments in modern machine learning. We argue that out of the three modelling activities - (1) formulating models; (2) analysing models; and (3) fitting or comparing models to data - inherent to mathematical biology, researchers currently focus too much on activity (2) at the cost of (1). This trend, we propose, can be reversed by realising that any given biological phenomenon can be modelled in an infinite number of different ways, through the adoption of an pluralistic approach, where we view a system from multiple, different points of view. We explain this pluralistic approach using fish locomotion as a case study and illustrate some of the pitfalls - universalism, creating models of models, etc. - that hinder mathematical biology. We then ask how we might rediscover a lost art: that of creative mathematical modelling.

CATER: Combined Animal Tracking & Environment Reconstruction

Quantifying the behavior of small animals traversing long distances in complex environments is one of the most difficult tracking scenarios for computer vision. Tiny and low-contrast foreground objects have to be localized in cluttered and dynamic scenes as well as trajectories compensated for camera motion and drift in multiple lengthy recordings. We introduce CATER, a novel methodology combining an unsupervised probabilistic detection mechanism with a globally optimized environment reconstruction pipeline enabling precision behavioral quantification in natural environments. Implemented as an easy to use and highly parallelized tool, we show its application to recover fine-scale motion trajectories, registered to a high-resolution image mosaic reconstruction, of naturally foraging desert ants from unconstrained field recordings. By bridging the gap between laboratory and field experiments, we gain previously unknown insights into ant navigation with respect to motivational states, previous experience, and current environments and provide an appearance-agnostic method applicable to study the behavior of a wide range of terrestrial species under realistic conditions.

Emerging technologies for behavioral research in changing environments

The first response exhibited by animals to changing environments is typically behavioral. Behavior is thus central to predicting, and mitigating, the impacts that natural and anthropogenic environmental changes will have on populations and, consequently, ecosystems. Yet the inherently multiscale nature of behavior, as well as the complexities associated with inferring how animals perceive their world, and make decisions, has constrained the scope of behavioral research. Major technological advances in electronics and in machine learning, however, provide increasingly powerful means to see, analyze, and interpret behavior in its natural complexity. We argue that these disruptive technologies will foster new approaches that will allow us to move beyond quantitative descriptions and reveal the underlying generative processes that give rise to behavior.