A rapid whisker-based decision underlying skilled locomotion in mice

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce 'Lightning Pose', an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We released a cloud application that allows users to label data, train networks and process new videos directly from the browser.

From innate to instructed: A new look at perceptual decision-making

Understanding how subjects perceive sensory stimuli in their environment and use this information to guide appropriate actions is a major challenge in neuroscience. To study perceptual decision-making in animals, researchers use tasks that either probe spontaneous responses to stimuli (often described as "naturalistic") or train animals to associate stimuli with experimenter-defined responses. Spontaneous decisions rely on animals' pre-existing knowledge, while trained tasks offer greater versatility, albeit often at the cost of extensive training. Here, we review emerging approaches to investigate perceptual decision-making using both spontaneous and trained behaviors, highlighting their strengths and limitations. Additionally, we propose how trained decision-making tasks could be improved to achieve faster learning and a more generalizable understanding of task rules.

STPoseNet: A real-time spatiotemporal network model for robust mouse pose estimation

Animal behavior analysis plays a crucial role in contemporary neuroscience research. However, the performance of the frame-by-frame approach may degrade in scenarios with occlusions or motion blur. In this study, we propose a spatiotemporal network model based on YOLOv8 to enhance the accuracy of key-point detection in mouse behavioral experimental videos. This model integrates a time-domain tracking strategy comprising two components: the first part utilizes key-point detection results from the previous frame to detect potential target locations in the subsequent frame; the second part employs Kalman filtering to analyze key-point changes prior to detection, allowing for the estimation of missing key-points. In the comparison of pose estimation results between our approach, YOLOv8, DeepLabCut and SLEAP on videos of three mouse behavioral experiments, our approach demonstrated significantly superior performance. This suggests that our method offers a new and effective means of accurately tracking and estimating pose in mice through spatiotemporal processing.

Simultaneous, cortex-wide dynamics of up to 1 million neurons reveal unbounded scaling of dimensionality with neuron number

The brain's remarkable properties arise from the collective activity of millions of neurons. Widespread application of dimensionality reduction to multi-neuron recordings implies that neural dynamics can be approximated by low-dimensional "latent" signals reflecting neural computations. However, can such low-dimensional representations truly explain the vast range of brain activity, and if not, what is the appropriate resolution and scale of recording to capture them? Imaging neural activity at cellular resolution and near-simultaneously across the mouse cortex, we demonstrate an unbounded scaling of dimensionality with neuron number in populations up to 1 million neurons. Although half of the neural variance is contained within sixteen dimensions correlated with behavior, our discovered scaling of dimensionality corresponds to an ever-increasing number of neuronal ensembles without immediate behavioral or sensory correlates. The activity patterns underlying these higher dimensions are fine grained and cortex wide, highlighting that large-scale, cellular-resolution recording is required to uncover the full substrates of neuronal computations.

behaviorMate: An Intranet of Things Approach for Adaptable Control of Behavioral and Navigation-Based Experiments

Investigators conducting behavioral experiments often need precise control over the timing of the delivery of stimuli to subjects and to collect the precise times of the subsequent behavioral responses. Furthermore, investigators want fine-tuned control over how various multi-modal cues are presented. behaviorMate takes an "Intranet of Things" approach, using a networked system of hardware and software components for achieving these goals. The system outputs a file with integrated timestamp-event pairs that investigators can then format and process using their own analysis pipelines. We present an overview of the electronic components and GUI application that make up behaviorMate as well as mechanical designs for compatible experimental rigs to provide the reader with the ability to set up their own system. A wide variety of paradigms are supported, including goal-oriented learning, random foraging, and context switching. We demonstrate behaviorMate's utility and reliability with a range of use cases from several published studies and benchmark tests. Finally, we present experimental validation demonstrating different modalities of hippocampal place field studies. Both treadmill with burlap belt and virtual reality with running wheel paradigms were performed to confirm the efficacy and flexibility of the approach. Previous solutions rely on proprietary systems that may have large upfront costs or present frameworks that require customized software to be developed. behaviorMate uses open-source software and a flexible configuration system to mitigate both concerns. behaviorMate has a proven record for head-fixed imaging experiments and could be easily adopted for task control in a variety of experimental situations.

Attentional modulation of secondary somatosensory and visual thalamus of mice

Each sensory modality has its own primary and secondary thalamic nuclei. While the primary thalamic nuclei are well understood to relay sensory information from the periphery to the cortex, the role of secondary sensory nuclei is elusive. One hypothesis has been that secondary nuclei may support feature-based attention. If this is true, one would also expect the activity in different nuclei to reflect the degree to which modalities are or are not behaviorally relevant in a task. We trained head-fixed mice to attend to one sensory modality while ignoring a second modality, namely to attend to touch and ignore vision, or vice versa. Arrays were used to record simultaneously from secondary somatosensory thalamus (POm) and secondary visual thalamus (LP). In mice trained to respond to tactile stimuli and ignore visual stimuli, POm was robustly activated by touch and largely unresponsive to visual stimuli. A different pattern was observed when mice were trained to respond to visual stimuli and ignore touch, with POm now more robustly activated during visual trials. This POm activity was not explained by differences in movements (i.e., whisking, licking, pupil dilation) resulting from the two tasks. Post hoc histological reconstruction of array tracks through POm revealed that subregions varied in their degree of plasticity. LP exhibited similar phenomena. We conclude that behavioral training reshapes activity in secondary thalamic nuclei. Secondary nuclei may respond to behaviorally relevant, reward-predicting stimuli regardless of stimulus modality.

Direct cortical inputs to hippocampal area CA1 transmit complementary signals for goal-directed navigation

The subregions of the entorhinal cortex (EC) are conventionally thought to compute dichotomous representations for spatial processing, with the medial EC (MEC) providing a global spatial map and the lateral EC (LEC) encoding specific sensory details of experience. Yet, little is known about the specific types of information EC transmits downstream to the hippocampus. Here, we exploit in vivo sub-cellular imaging to record from EC axons in CA1 while mice perform navigational tasks in virtual reality (VR). We uncover distinct yet overlapping representations of task, location, and context in both MEC and LEC axons. MEC transmitted highly location- and context-specific codes; LEC inputs were biased by ongoing navigational goals. However, during tasks with reliable reward locations, the animals' position could be accurately decoded from either subregion. Our results revise the prevailing dogma about EC information processing, revealing novel ways spatial and non-spatial information is routed and combined upstream of the hippocampus.

An output-null signature of inertial load in motor cortex

Coordinated movement requires the nervous system to continuously compensate for changes in mechanical load across different contexts. For voluntary movements like reaching, the motor cortex is a critical hub that generates commands to move the limbs and counteract loads. How does cortex contribute to load compensation when rhythmic movements are clocked by a spinal pattern generator? Here, we address this question by manipulating the mass of the forelimb in unrestrained mice during locomotion. While load produces changes in motor output that are robust to inactivation of motor cortex, it also induces a profound shift in cortical dynamics, which is minimally affected by cerebellar perturbation and significantly larger than the response in the spinal motoneuron population. This latent representation may enable motor cortex to generate appropriate commands when a voluntary movement must be integrated with an ongoing, spinally-generated rhythm.

Acetylcholine waves and dopamine release in the striatum

Striatal dopamine encodes reward, with recent work showing that dopamine release occurs in spatiotemporal waves. However, the mechanism of dopamine waves is unknown. Here we report that acetylcholine release in mouse striatum also exhibits wave activity, and that the spatial scale of striatal dopamine release is extended by nicotinic acetylcholine receptors. Based on these findings, and on our demonstration that single cholinergic interneurons can induce dopamine release, we hypothesized that the local reciprocal interaction between cholinergic interneurons and dopamine axons suffices to drive endogenous traveling waves. We show that the morphological and physiological properties of cholinergic interneuron - dopamine axon interactions can be modeled as a reaction-diffusion system that gives rise to traveling waves. Analytically-tractable versions of the model show that the structure and the nature of propagation of acetylcholine and dopamine traveling waves depend on their coupling, and that traveling waves can give rise to empirically observed correlations between these signals. Thus, our study provides evidence for striatal acetylcholine waves in vivo, and proposes a testable theoretical framework that predicts that the observed dopamine and acetylcholine waves are strongly coupled phenomena.

Intrinsic dopamine and acetylcholine dynamics in the striatum of mice

External rewards such as food and money are potent modifiers of behaviour1,2. Pioneering studies established that these salient sensory stimuli briefly interrupt the tonic discharge of neurons that produce the neuromodulators dopamine (DA) and acetylcholine (ACh): midbrain DA neurons (DANs) fire a burst of action potentials that broadly elevates DA in the striatum3,4 at the same time that striatal cholinergic interneurons (CINs) produce a characteristic pause in firing5,6. These phasic responses are thought to create unique, temporally limited conditions that motivate action and promote learning7-11. However, the dynamics of DA and ACh outside explicitly rewarded situations remain poorly understood. Here we show that extracellular DA and ACh levels fluctuate spontaneously and periodically at a frequency of approximately 2 Hz in the dorsal striatum of mice and maintain the same temporal relationship relative to one another as that evoked by reward. We show that this neuromodulatory coordination does not arise from direct interactions between DA and ACh within the striatum. Instead, we provide evidence that periodic fluctuations in striatal DA are inherited from midbrain DANs, while striatal ACh transients are driven by glutamatergic inputs, which act to locally synchronize the spiking of CINs. Together, our findings show that striatal neuromodulatory dynamics are autonomously organized by distributed extra-striatal afferents. The dominance of intrinsic rhythms in DA and ACh offers new insights for explaining how reward-associated neural dynamics emerge and how the brain motivates action and promotes learning from within.

The Novel Somatosensory Nose-Poke Adapted Paradigm (SNAP) Is an Effective Tool to Assess Differences in Tactile Sensory Preferences in Autistic-Like Mice

One of the most prevalent deficits in autism spectrum disorder (ASD) are sensitivities to sensory stimuli. Despite the prevalence of sensory deficits in autism, there are few paradigms capable of easily assessing sensory behaviors in ASD-like mouse models. We addressed this need by creating the Somatosensory Nose-poke Adapted Paradigm (SNAP), which consists of an elevated platform with 6 holes in the center, half of which are lined with sandpaper and half are smooth, requiring mice to use their whiskers to sense the texture. The SNAP paradigm assesses tactile sensory preferences as well as stereotypy, anxiety, and locomotion. We used two wild-type (neurotypical) mouse strains, C57BL/6J (C57) inbred and CD-1 outbred mice, and two ASD mouse models, BTBR (a model of idiopathic ASD) and Cntnap2 -/- mice (a model of syndromic ASD). We found that both ASD models produced more nose pokes into the rough condition than the smooth condition, suggesting an increased preference for complex tactile stimulation when compared with the neurotypical groups, wherein no differences were observed. Furthermore, we found increased stereotypy and time spent in the center, suggestive of decreased anxiety, only for BTBR mice compared with the other mouse strains. Overall, SNAP is an easy to implement task to assess the degree of preference for complex tactile stimulation in ASD mouse models that can be further modified to exclude possible confounding effects of novelty or anxiety on the sensory preferences.

Peripheral and central sensation: multisensory orienting and recognition across species

Attentional bottlenecks force animals to deeply process only a selected fraction of sensory inputs. This motivates a unifying central-peripheral dichotomy (CPD), which separates multisensory processing into functionally defined central and peripheral senses. Peripheral senses (e.g., human audition and peripheral vision) select a fraction of the sensory inputs by orienting animals' attention; central senses (e.g., human foveal vision) allow animals to recognize the selected inputs. Originally used to understand human vision, CPD can be applied to multisensory processes across species. I first describe key characteristics of central and peripheral senses, such as the degree of top-down feedback and density of sensory receptors, and then show CPD as a framework to link ecological, behavioral, neurophysiological, and anatomical data and produce falsifiable predictions.

Open-source tools for behavioral video analysis: Setup, methods, and best practices

Recently developed methods for video analysis, especially models for pose estimation and behavior classification, are transforming behavioral quantification to be more precise, scalable, and reproducible in fields such as neuroscience and ethology. These tools overcome long-standing limitations of manual scoring of video frames and traditional 'center of mass' tracking algorithms to enable video analysis at scale. The expansion of open-source tools for video acquisition and analysis has led to new experimental approaches to understand behavior. Here, we review currently available open-source tools for video analysis and discuss how to set up these methods for labs new to video recording. We also discuss best practices for developing and using video analysis methods, including community-wide standards and critical needs for the open sharing of datasets and code, more widespread comparisons of video analysis methods, and better documentation for these methods especially for new users. We encourage broader adoption and continued development of these tools, which have tremendous potential for accelerating scientific progress in understanding the brain and behavior.

A robust role for motor cortex

The role of motor cortex in non-primate mammals remains unclear. More than a century of stimulation, anatomical and electrophysiological studies has implicated neural activity in this region with all kinds of movement. However, following the removal of motor cortex, rats retain most of their adaptive behaviors, including previously learned skilled movements. Here we revisit these two conflicting views of motor cortex and present a new behavior assay, challenging animals to respond to unexpected situations while navigating a dynamic obstacle course. Surprisingly, rats with motor cortical lesions show clear impairments facing an unexpected collapse of the obstacles, while showing no impairment with repeated trials in many motor and cognitive metrics of performance. We propose a new role for motor cortex: extending the robustness of sub-cortical movement systems, specifically to unexpected situations demanding rapid motor responses adapted to environmental context. The implications of this idea for current and future research are discussed.

Behavior is movement only but how to interpret it? Problems and pitfalls in translational neuroscience-a 40-year experience

Translational research in behavioral neuroscience seeks causes and remedies for human mental health problems in animals, following leads imposed by clinical research in psychiatry. This endeavor faces several problems because scientists must read and interpret animal movements to represent human perceptions, mood, and memory processes. Yet, it is still not known how mammalian brains bundle all these processes into a highly compressed motor output in the brain stem and spinal cord, but without that knowledge, translational research remains aimless. Based on some four decades of experience in the field, the article identifies sources of interpretation problems and illustrates typical translational pitfalls. (1) The sensory world of mice is different. Smell, hearing, and tactile whisker sensations dominate in rodents, while visual input is comparatively small. In humans, the relations are reversed. (2) Mouse and human brains are equated inappropriately: the association cortex makes up a large portion of the human neocortex, while it is relatively small in rodents. The predominant associative cortex in rodents is the hippocampus itself, orchestrating chiefly inputs from secondary sensorimotor areas and generating species-typical motor patterns that are not easily reconciled with putative human hippocampal functions. (3) Translational interpretation of studies of memory or emotionality often neglects the ecology of mice, an extremely small species surviving by freezing or flight reactions that do not need much cognitive processing. (4) Further misinterpretations arise from confounding neuronal properties with system properties, and from rigid mechanistic thinking unaware that many experimentally induced changes in the brain do partially reflect unpredictable compensatory plasticity. (5) Based on observing hippocampal lesion effects in mice indoors and outdoors, the article offers a simplistic general model of hippocampal functions in relation to hypothalamic input and output, placing hypothalamus and the supraspinal motor system at the top of a cerebral hierarchy. (6) Many translational problems could be avoided by inclusion of simple species-typical behaviors as end-points comparable to human cognitive or executive processing, and to rely more on artificial intelligence for recognizing patterns not classifiable by traditional psychological concepts.

Signatures of rapid plasticity in hippocampal CA1 representations during novel experiences

Neurons in the hippocampus exhibit a striking selectivity for specific combinations of sensory features, forming representations that are thought to subserve episodic memory. Even during completely novel experiences, hippocampal "place cells" are rapidly configured such that the population sparsely encodes visited locations, stabilizing within minutes of the first exposure to a new environment. What mechanisms enable this fast encoding of experience? Using virtual reality and neural population recordings in mice, we dissected the effects of novelty and experience on the dynamics of place field formation. During place field formation, many CA1 neurons immediately modulated the amplitude of their activity and shifted the location of their field, rapid changes in tuning predicted by behavioral timescale synaptic plasticity (BTSP). Signatures of BTSP were particularly enriched during the exploration of a novel context and decayed with experience. Our data suggest that novelty modulates the effective learning rate in CA1, favoring rapid mechanisms of field formation to encode a new experience.

Behavioral and physiological monitoring for awake neurovascular coupling experiments: a how-to guide

Significance: Functional brain imaging in awake animal models is a popular and powerful technique that allows the investigation of neurovascular coupling (NVC) under physiological conditions. However, ubiquitous facial and body motions (fidgeting) are prime drivers of spontaneous fluctuations in neural and hemodynamic signals. During periods without movement, animals can rapidly transition into sleep, and the hemodynamic signals tied to arousal state changes can be several times larger than sensory-evoked responses. Given the outsized influence of facial and body motions and arousal signals in neural and hemodynamic signals, it is imperative to detect and monitor these events in experiments with un-anesthetized animals. Aim: To cover the importance of monitoring behavioral state in imaging experiments using un-anesthetized rodents, and describe how to incorporate detailed behavioral and physiological measurements in imaging experiments. Approach: We review the effects of movements and sleep-related signals (heart rate, respiration rate, electromyography, intracranial pressure, whisking, and other body movements) on brain hemodynamics and electrophysiological signals, with a focus on head-fixed experimental setup. We summarize the measurement methods currently used in animal models for detection of those behaviors and arousal changes. We then provide a guide on how to incorporate this measurements with functional brain imaging and electrophysiology measurements. Results: We provide a how-to guide on monitoring and interpreting a variety of physiological signals and their applications to NVC experiments in awake behaving mice. Conclusion: This guide facilitates the application of neuroimaging in awake animal models and provides neuroscientists with a standard approach for monitoring behavior and other associated physiological parameters in head-fixed animals.

Brainstem Circuits for Locomotion

Locomotion is a universal motor behavior that is expressed as the output of many integrated brain functions. Locomotion is organized at several levels of the nervous system, with brainstem circuits acting as the gate between brain areas regulating innate, emotional, or motivational locomotion and executive spinal circuits. Here we review recent advances on brainstem circuits involved in controlling locomotion. We describe how delineated command circuits govern the start, speed, stop, and steering of locomotion. We also discuss how these pathways interface between executive circuits in the spinal cord and diverse brain areas important for context-specific selection of locomotion. A recurrent theme is the need to establish a functional connectome to and from brainstem command circuits. Finally, we point to unresolved issues concerning the integrated function of locomotor control.

Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.