Phenotypic analysis of ataxia in spinocerebellar ataxia type 6 mice using DeepLabCut

This study emphasizes the benefits of open-source software such as DeepLabCut (DLC) and R to automate, customize and enhance data analysis of motor behavior. We recorded 2 different spinocerebellar ataxia type 6 mouse models while performing the classic beamwalk test, tracked multiple body parts using the markerless pose-estimation software DLC and analyzed the tracked data using self-written scripts in the programming language R. The beamwalk analysis script (BAS) counts and classifies minor and major hindpaw slips with an 83% accuracy compared to manual scoring. Nose, belly and tail positions relative to the beam, as well as the angle at the tail base relative to the nose and tail tip were determined to characterize motor deficits in greater detail. Our results found distinct ataxic abnormalities such as an increase in major left hindpaw slips and a lower belly and tail position in both SCA6 ataxic mouse models compared to control mice at 18 months of age. Furthermore, a more detailed analysis of various body parts relative to the beam revealed an overall lower body position in the SCA684Q compared to the CT-longQ27PC mouse line at 18 months of age, indicating a more severe ataxic deficit in the SCA684Q group.

Forebrain EAAT3 Overexpression Increases Susceptibility to Amphetamine-Induced Repetitive Behaviors

Obsessive-compulsive disorder (OCD) is a debilitating psychiatric disorder characterized by intrusive obsessive thoughts and compulsive behaviors. Multiple studies have shown the association of polymorphisms in the SLC1A1 gene with OCD. The most common of these OCD-associated polymorphisms increases the expression of the encoded protein, excitatory amino acid transporter 3 (EAAT3), a neuronal glutamate transporter. Previous work has shown that increased EAAT3 expression results in OCD-relevant behavioral phenotypes in rodent models. In this study, we created a novel mouse model with targeted, reversible overexpression of Slc1a1 in forebrain neurons. The mice do not have a baseline difference in repetitive behavior but show increased hyperlocomotion following a low dose of amphetamine (3 mg/kg) and increased stereotypy following a high dose of amphetamine (8 mg/kg). We next characterized the effect of amphetamine on striatal cFos response and found that amphetamine increased cFos throughout the striatum in both control and Slc1a1-overexpressing (OE) mice, but Slc1a1-OE mice had increased cFos expression in the ventral striatum relative to controls. We used an unbiased machine classifier to robustly characterize the behavioral response to different doses of amphetamine and found a unique response to amphetamine in Slc1a1-OE mice, relative to controls. Lastly, we found that the differences in striatal cFos expression in Slc1a1-OE mice were driven by cFos expression specifically in D1 neurons, as Slc1a1-OE mice had increased cFos in D1 ventral medial striatal neurons, implicating this region in the exaggerated behavioral response to amphetamine in Slc1a1-OE mice.

Transcranial direct current stimulation improves motor function in rats with 6-hydroxydopamine-induced Parkinsonism

Transcranial direct current stimulation (tDCS) is increasingly being used for Parkinson's disease (PD); however, the evaluation of its clinical impact remains complex owing to the heterogeneity of patients and treatments. Therefore, we used a unilateral 6-hydroxydopamine-induced PD rat model to investigate whether anodal tDCS of the primary motor cortex (M1) alleviates PD motor deficits. Before tDCS treatment, unilateral PD rats preferentially used the forelimb ipsilateral to the lesion in the exploratory cylinder test and showed reduced locomotor activity in the open field test. In addition, PD-related clumsy forelimb movements during treadmill walking were detected using deep learning-based video analysis (DeepLabCut). When the 5-day tDCS treatment began, the forelimb-use asymmetry was ameliorated gradually, and locomotor activity increased to pre-lesion levels. tDCS treatment also normalized unnatural forelimb movement during walking and restored a balanced gait. However, these therapeutic effects were rapidly lost or gradually disappeared when the tDCS treatment was terminated. Histological analysis at the end of the experiment revealed that the animals had moderately advanced PD, with 40-50% of dopamine neurons and fibers preserved on the injured side compared with those on the intact side. Although it remains a challenge to elucidate the neural mechanisms of the transient improvement in motor function induced by tDCS, the results of this study provide evidence that tDCS of the M1 produces positive behavioral outcomes in PD animals and provides the basis for further clinical research examining the application of tDCS in patients with PD.

Deep-Learning-Based Analysis Reveals a Social Behavior Deficit in Mice Exposed Prenatally to Nicotine

Cigarette smoking during pregnancy is known to be associated with the incidence of attention-deficit/hyperactive disorder (ADHD). Recent developments in deep learning algorithms enable us to assess the behavioral phenotypes of animal models without cognitive bias during manual analysis. In this study, we established prenatal nicotine exposure (PNE) mice and evaluated their behavioral phenotypes using DeepLabCut and SimBA. We optimized the training parameters of DeepLabCut for pose estimation and succeeded in labeling a single-mouse or two-mouse model with high fidelity during free-moving behavior. We applied the trained network to analyze the behavior of the mice and found that PNE mice exhibited impulsivity and a lessened working memory, which are characteristics of ADHD. PNE mice also showed elevated anxiety and deficits in social interaction, reminiscent of autism spectrum disorder (ASD). We further examined PNE mice by evaluating adult neurogenesis in the hippocampus, which is a pathological hallmark of ASD, and demonstrated that newborn neurons were decreased, specifically in the ventral part of the hippocampus, which is reported to be related to emotional and social behaviors. These results support the hypothesis that PNE is a risk factor for comorbidity with ADHD and ASD in mice.

Effects of ketamine on rat social behavior as analyzed by DeepLabCut and SimBA deep learning algorithms

Traditional methods of rat social behavior assessment are extremely time-consuming and susceptible to the subjective biases. In contrast, novel digital techniques allow for rapid and objective measurements. This study sought to assess the feasibility of implementing a digital workflow to compare the effects of (R,S)-ketamine and a veterinary ketamine preparation Vetoquinol (both at 20 mg/kg) on the social behaviors of rat pairs. Historical and novel videos were used to train the DeepLabCut neural network. The numerical data generated by DeepLabCut from 14 video samples, representing various body parts in time and space were subjected to the Simple Behavioral Analysis (SimBA) toolkit, to build classifiers for 12 distinct social and non-social behaviors. To validate the workflow, previously annotated by the trained observer historical videos were analyzed with SimBA classifiers, and regression analysis of the total time of social interactions yielded R 2 = 0.75, slope 1.04; p < 0.001 (N = 101). Remarkable similarities between human and computer annotations allowed for using the digital workflow to analyze 24 novel videos of rats treated with vehicle and ketamine preparations. Digital workflow revealed similarities in the reduction of social behavior by both compounds, and no substantial differences between them. However, the digital workflow also demonstrated ketamine-induced increases in self-grooming, increased transitions from social contacts to self-grooming, and no effects on adjacent lying time. This study confirms and extends the utility of deep learning in analyzing rat social behavior and highlights its efficiency and objectivity. It provides a faster and objective alternative to human workflow.

Automated non-contact measurement of the spine curvature at the sagittal plane using a deep neural network

BACKGROUND

Non-radiographical techniques have been suggested to measure the spine curvature at the sagittal plane. However, a neural network has not been used to measure the curvature.

METHODS

A single video camera captured images of a standing posture at the sagittal plane from twenty healthy males. Six marker positions along the spine's contour in each image were identified for measuring inclination, thoracic kyphosis, and lumbar lordosis angles. We estimated three inflection points around the neck, hip, and between the neck and hip, followed by identifying two adjacent marker positions per inflection point to compute its tangent. The angular deviation of each tangent line from the horizontal was computed to measure inclination angles. Thoracic kyphosis and lumbar lordosis angles were computed by the angular difference between the two adjacent tangents. A deep neural network was trained with 500,000 iterations using the labeled images from 18 participants (388 and 44 images for training and test set) and then evaluated using the unseen images (2 participants, 48 images; evaluation set).

FINDINGS

The mean total training and test errors were <2 pixels (∼ 0.6 cm). The total error in the evaluation set was qualitatively comparable (∼ 3 pixels = ∼ 0.9 cm), suggesting the model performance was maintained in the unseen data. The angle values between labeled and network-predicted marker positions were similar in the evaluation set.

INTERPRETATION

The network training with a relatively small number of images was successful based on the small error values observed in the evaluation set. The model may be an affordable, automated, and non-contact measurement tool for the human spine curvature.

Automated measurement of distance-walked as a "sixth vital sign" for detecting infusion reactions during preclinical testing

Infusion reactions are a major risk for advanced therapeutics (e.g., engineered proteins nanoparticles, etc.), which can trigger the complement cascade, anaphylaxis, and other life-threatening immune responses. However, during the early phases of development, it is uncommon to assess for infusion reactions, given the labor involved in measuring multiple physiological parameters in rodents. Therefore, we sought to develop an automated quantification of rodent locomotion to serve as a sensitive screening tool for infusion reactions, with minimal added labor-time for each experiment. Here we present the detailed methods for building a motion tracking cage for mice, requiring ∼$100 of materials, ∼2 h to build and set up completely, and employing freely available software (DeepLabCut). The distance-walked after injection was first shown to have the predicted effects for stimulants (caffeine), sedatives (ketamine), and toxins (lipopolysaccharide). Additionally, the distance-walked more sensitively detected the effects of these compounds than did pulse oximetry-based measurements of the classical vital signs of heart rate, respiratory rate, and blood oxygen saturation. Finally, we examined a nanomedicine formulation that has been in preclinical development, liposomes targeted to the cell adhesion molecule ICAM. While this formulation has been studied across dozens of publications, it has not previously been noted to produce an infusion reaction. However, the automated motion tracking cage showed that ICAM-liposomes markedly reduce the distance-walked, which we confirmed by measuring the other vital signs. Importantly, the motion tracking cage added < 5 min of labor time per 5-mouse condition, while pulse oximetry with a neck cuff (by far the most stable oximetry signal in mice) required ∼ 100 min of labor time. Thus, automated measurement of distance-walked can indeed serve as a "sixth vital sign" for detecting infusion reactions during preclinical testing. Additionally, the device to measure distance-walked is easy and cheap to build and requires negligible labor time for each experiment, enabling distance-walked to be recorded in nearly every infusion experiment.

A Deep-Learning Driven Investigation of the Circuit Basis for Reflexive Hypersensitivity to Thermal Pain

Objectively measuring animal behavior is vital to understanding the neural circuits underlying pain. Recent progress in machine vision has presented unprecedented scope in behavioral analysis. Here, we apply DeepLabCut (DLC) to dissect mouse behavior on the thermal-plate test - a commonly used paradigm to ascertain supraspinal contributions to noxious thermal sensation and pain hypersensitivity. We determine the signature characteristics of the pattern of mouse movement and posture in 3D in response to a range of temperatures from innocuous to noxious on the thermal-plate test. Next, we test how acute chemical and chronic inflammatory injuries sensitize mouse behaviors. Repeated exposure to noxious temperatures on the thermal plate can induce learning. In this study, we design a novel assay and formulate an analytical pipeline to facilitate the dissection of plasticity mechanisms in pain circuits in the brain. Last, we record and test how activating Tacr1 expressing PBN neurons (PBNTacr1) - a population responsive to sustained noxious stimuli- affects mouse behavior on the thermal plate test. Taken together, we demonstrate that by tracking a single body part of a mouse, we can reveal the behavioral signatures of mice exposed to noxious surface temperatures, report the alterations of the same when injured, and determine if a molecularly and anatomically defined pain-responsive circuit plays a role in the reflexive hypersensitivity to thermal pain.

Using DeepLabCut for markerless cardiac physiology and toxicity estimation in water fleas (Daphnia magna)

Daphnia magna is one species of water flea that has been used for a long time for ecotoxicity studies. In addition, Daphnia has a myogenic heart that is very useful for cardiotoxicity studies. Previous attempts to calculate the cardiac parameter endpoints in Daphnia suffer from the drawback of tedious operation and high variation due to manual counting errors. Even the previous method that utilized deep learning to help the process suffer from either overestimation of parameters or the need for specialized equipment to perform the analysis. In this study, we utilized DeepLabCut software previously used for animal pose tracking and demonstrated that ResNet_152 was the best fit for training the network. The trained network also showed comparable results with ImageJ and Kymograph, which was mostly done manually. In addition to that, several macro scripts in either Excel or Python format were developed to help summarize the data for faster analysis. The trained network was then challenged to analyze the potential cardiotoxicity of imidacloprid and pendimethalin in D. magna, and it showed that both pesticides cause alteration in their cardiac performance. Overall, this method provides a simple and automatic method to analyze the cardiac performance of Daphnia by utilizing DeepLabCut. The method proposed in this paper can contribute greatly to scientists conducting fast and accurate cardiotoxicity measurements when using Daphnia as a model.

Automated Segmentation of the Mouse Body Language to Study Stimulus-Evoked Emotional Behaviors

Understanding the neural basis of emotions is a critical step to uncover the biological substrates of neuropsychiatric disorders. To study this aspect in freely behaving mice, neuroscientists have relied on the observation of ethologically relevant bodily cues to infer the affective content of the subject, both in neutral conditions or in response to a stimulus. The best example of that is the widespread assessment of freezing in experiments testing both conditioned and unconditioned fear responses. While robust and powerful, these approaches come at a cost: they are usually confined within selected time windows, accounting for only a limited portion of the complexity of emotional fluctuation. Moreover, they often rely on visual inspection and subjective judgment, resulting in inconsistency across experiments and questionable result interpretations. To overcome these limitations, novel tools are arising, fostering a new avenue in the study of the mouse naturalistic behavior. In this work we developed a computational tool [stimulus-evoked behavioral tracking in 3D for rodents (SEB3R)] to automate and standardize an ethologically driven observation of freely moving mice. Using a combination of machine learning-based behavioral tracking and unsupervised cluster analysis, we identified statistically meaningful postures that could be used for empirical inference on a subsecond scale. We validated the efficacy of this tool in a stimulus-driven test, the whisker nuisance (WN) task, where mice are challenged with a prolonged and invasive whisker stimulation, showing that identified postures can be reliably used as a proxy for stimulus-driven fearful and explorative behaviors.

Long and short whiskers differently guide snout/pellet interaction in rat oral grasping

We studied the role of rat whisker/snout tactile sense during oral grasping, comparing control data with those obtained, respectively, 1-3 and 5-7 days after bilateral long or short whisker trimming and 3-5 and 8-10 days after bilateral infraorbital nerve (ION) severing. Two behavioural phases were identified: whisker-snout contact by nose-N or lip-L and snout-tongue contact. The second phase involved either: snout passing over stationary pellet (Still pellet); pellet rolling as the snout passed over it (Rolling pellet); pellet being pushed forward by the snout (Pushed pellet); or pellet being hit and pushed away (Hit/Lost pellet). In controls, success was 100%, with N-contact prevailing over L-contact in the first phase and Still pellet in the second. In long whisker-trimmed versus controls, success was still 100%, but L-contact increased in frequency, Pushed pellet prevailed and the second phase duration increased. In short whisker-trimmed versus controls, success remained 100%, with increased L-contact frequency; the first phase duration did not change, but the second phase increased since in pushed trials, the pellet rolled around the snout. In ION-severed versus controls, both phases changed drastically: L-contact frequency increased, Pushed pellet prevailed and contact was persistently maintained; Hit/Lost pellet emerged, Still and Rolling pellets disappeared and the oral-grasping sequence was not triggered. These results suggest that long and short whiskers, respectively, optimize the first and second phases of snout-pellet interaction and that whisker/snout sense is necessary to trigger oral grasping. Kinematic trajectory analysis supports the conclusion that movement from whisker to snout contact is an orientation response.

Automated pose estimation reveals walking characteristics associated with lameness in broilers

Walking ability of broilers can be improved by selective breeding, but large-scale phenotypic records are required. Currently, gait of individual broilers is scored by trained experts, however, precision phenotyping tools could offer a more objective and high-throughput alternative. We studied whether specific walking characteristics determined through pose estimation are linked to gait in broilers. We filmed male broilers from behind, walking through a 3 m × 0.4 m (length × width) corridor one by one, at 3 time points during their lifetime (at 14, 21, and 33 d of age). We used a deep learning model, developed in DeepLabCut, to detect and track 8 keypoints (head, neck, left and right knees, hocks, and feet) of broilers in the recorded videos. Using the keypoints of the legs, 6 pose features were quantified during the double support phase of walking, and 1 pose feature was quantified during steps, at maximum leg lift. Gait was scored on a scale from 0 to 5 by 4 experts, using the videos recorded on d 33, and the broilers were further classified as having either good gait (mean gait score ≤2) or suboptimal gait (mean gait score >2). The relationship of pose features on d 33 with gait was analyzed using the data of 84 broilers (good gait: 57.1%, suboptimal gait: 42.9%). Birds with suboptimal gait had sharper hock joint lateral angles and lower hock-feet distance ratios during double support on d 33, on average. During steps, relative step height was lower in birds with suboptimal gait. Step height and hock-feet distance ratio showed the largest mean deviations in broilers with suboptimal gait compared to those with good gait. We demonstrate that pose estimation can be used to assess walking characteristics during a large part of the productive life of broilers, and to phenotype and monitor broiler gait. These insights can be used to understand differences in the walking patterns of lame broilers, and to build more sophisticated gait prediction models.

Hypocretin/orexin neurons encode social discrimination and exhibit a sex-dependent necessity for social interaction

The hypothalamus plays a crucial role in the modulation of social behavior by encoding internal states. The hypothalamic hypocretin/orexin neurons, initially identified as regulators of sleep and appetite, are important for emotional and motivated behaviors. However, their role in social behavior remains unclear. Using fiber photometry and behavioral analysis, we show here that hypocretin neurons differentially encode social discrimination based on the nature of social encounters. The optogenetic inhibition of hypocretin neuron activity or blocking of hcrt-1 receptors reduces the amount of time mice are engaged in social interaction in males but not in females. Reduced hcrt-1 receptor signaling during social interaction is associated with altered activity in the insular cortex and ventral tegmental area in males. Our data implicating hypocretin neurons as sexually dimorphic regulators within social networks have significant implications for the treatment of neuropsychiatric diseases with social dysfunction, particularly considering varying prevalence among sexes.

A deep learning analysis of Drosophila body kinematics during magnetically tethered flight

Flying Drosophila rely on their vision to detect visual objects and adjust their flight course. Despite their robust fixation on a dark, vertical bar, our understanding of the underlying visuomotor neural circuits remains limited, in part due to difficulties in analyzing detailed body kinematics in a sensitive behavioral assay. In this study, we observed the body kinematics of flying Drosophila using a magnetically tethered flight assay, in which flies are free to rotate around their yaw axis, enabling naturalistic visual and proprioceptive feedback. Additionally, we used deep learning-based video analyses to characterize the kinematics of multiple body parts in flying animals. By applying this pipeline of behavioral experiments and analyses, we characterized the detailed body kinematics during rapid flight turns (or saccades) in two different visual conditions: spontaneous flight saccades under static screen and bar-fixating saccades while tracking a rotating bar. We found that both types of saccades involved movements of multiple body parts and that the overall dynamics were comparable. Our study highlights the importance of sensitive behavioral assays and analysis tools for characterizing complex visual behaviors.

Machine-learning-based video analysis of grasping behavior during recovery from cervical spinal cord injury

Comprehensive characterizations of hand grasping behaviors after cervical spinal cord injuries are fundamental for developing rehabilitation strategies to promote recovery in spinal-cord-injured primates. We used the machine-learning-based video analysis software, DeepLabCut, to sensitively quantify kinematic aspects of grasping behavioral deficits in squirrel monkeys with C5-level spinal cord injuries. Three squirrel monkeys were trained to grasp sugar pellets from wells of varying depths before and after a left unilateral lesion of the cervical dorsal column. Using DeepLabCut, we identified post-lesion deficits in kinematic grasping behavior that included changes in digit orientation, increased variance in vertical and horizontal digit movement, and longer time to complete the task. While video-based analyses of grasping behavior demonstrated deficits in fine-scale digit function that persisted through at least 14 weeks post-injury, traditional end-point behavioral analyses showed a recovery of global hand function as evidenced by recovery of the proportion of successful retrievals by approximately 14 weeks post-injury. The combination of traditional end-point and video-based kinematic analyses provides a more comprehensive characterization of grasping behavior and highlights that global grasping performance may recover despite persistent fine-scale kinematic deficits in digit function. Machine-learning-based video analysis of kinematic digit function, in conjunction with traditional end-point behavioral analyses of grasping behavior, provide sensitive and specific indices for monitoring recovery of fine-grained hand sensorimotor behavior after spinal cord injury that can aid future studies that seek to develop targeted therapeutic interventions for improving behavioral outcomes.

Video-based marker-free tracking and multi-scale analysis of mouse locomotor activity and behavioral aspects in an open field arena: A perspective approach to the quantification of complex gait disturbances associated with Alzheimer's disease

Introduction

Complex gait disturbances represent one of the prominent manifestations of various neurophysiological conditions, including widespread neurodegenerative disorders such as Alzheimer's and Parkinson's diseases. Therefore, instrumental measurement techniques and automatic computerized analysis appears essential for the differential diagnostics, as well as for the assessment of treatment effectiveness from experimental animal models to clinical settings.

Methods

Here we present a marker-free instrumental approach to the analysis of gait disturbances in animal models. Our approach is based on the analysis of video recordings obtained with a camera placed underneath an open field arena with transparent floor using the DeeperCut algorithm capable of online tracking of individual animal body parts, such as the snout, the paws and the tail. The extracted trajectories of animal body parts are next analyzed using an original computerized methodology that relies upon a generalized scalable model based on fractional Brownian motion with parameters identified by detrended partial cross-correlation analysis.

Results

We have shown that in a mouse model representative movement patterns are characterized by two asymptotic regimes characterized by integrated 1/f noise at small scales and nearly random displacements at large scales separated by a single crossover. More detailed analysis of gait disturbances revealed that the detrended cross-correlations between the movements of the snout, paws and tail relative to the animal body midpoint exhibit statistically significant discrepancies in the Alzheimer's disease mouse model compared to the control group at scales around the location of the crossover.

Discussion

We expect that the proposed approach, due to its universality, robustness and clear physical interpretation, is a promising direction for the design of applied analysis tools for the diagnostics of various gait disturbances and behavioral aspects in animal models. We further believe that the suggested mathematical models could be relevant as a complementary tool in clinical diagnostics of various neurophysiological conditions associated with movement disorders.

The accuracy of markerless motion capture combined with computer vision techniques for measuring running kinematics

BACKGROUND

Markerless motion capture based on low-cost 2-D video analysis in combination with computer vision techniques has the potential to provide accurate analysis of running technique in both a research and clinical setting. However, the accuracy of markerless motion capture for assessing running kinematics compared to a gold-standard approach remains largely unexplored.

OBJECTIVE

Here, we investigate the accuracy of custom-trained (DeepLabCut) and existing (OpenPose) computer vision techniques for assessing sagittal-plane hip, knee, and ankle running kinematics at speeds of 2.78 and 3.33 m s^-1 as compared to gold-standard marker-based motion capture.

METHODS

Differences between the markerless and marker-based approaches were assessed using statistical parameter mapping and expressed as root mean squared errors (RMSEs).

RESULTS

After temporal alignment and offset removal, both DeepLabCut and OpenPose showed no significant differences with the marker-based approach at 2.78 m s^-1 , but some significant differences remained at 3.33 m s^-1 . At 2.78 m s^-1 , RMSEs were 5.07, 7.91, and 5.60, and 5.92, 7.81, and 5.66 degrees for the hip, knee, and ankle for DeepLabCut and OpenPose, respectively. At 3.33 m s^-1 , RMSEs were 7.40, 10.9, 8.01, and 4.95, 7.45, and 5.76 for the hip, knee, and ankle for DeepLabCut and OpenPose, respectively.

CONCLUSION

The differences between OpenPose and the marker-based method were in line with or smaller than reported between other kinematic analysis methods and marker-based methods, while these differences were larger for DeepLabCut. Since the accuracy differed between individuals, OpenPose may be most useful to facilitate large-scale in-field data collection and investigation of group effects rather than individual-level analyses.

Machine learning based assessment of small-bodied fish tracking to evaluate spoiler baffle fish passage design

Fish passage research is important to mitigate the adverse effects of fragmented river habitats caused by waterway structures. The scale at which this research is undertaken varies from small-scale laboratory prototype studies to in-situ observations at various fish passage structures and bottlenecks. Using DeepLabCut, we introduce and evaluate a machine learning based workflow to track small-bodied fish in order to facilitate improved fish passage management. We specifically studied the behaviour and kinematics of Galaxias maculatus, a widespread diadromous Southern Hemisphere fish species. Upstream fish passage was studied in the presence of three different patches of spoiler baffles at an average water velocity of 0.4 m/s. In semi-supervised mode, the fish locations were extracted, and fish behaviour, such as swimming pathways and resting locations, was analysed based on extracted positions and recorded kinematic parameters. Individual fish behaviour and kinematic parameters were then used to assess the suitability of the three different spoiler baffle designs for enhancing fish passage. Using this technique, we were able to demonstrate where different spoiler baffle configurations resulted in significant differences in fish passage success and behaviour. For example, medium-spaced smaller baffles provided more accessible and uniform resting locations, which were required for efficient upstream passage. Results are discussed in relation to fish passage management at small instream structures.

Leptin deficiency-caused behavioral change - A comparative analysis using EthoVision and DeepLabCut

Introduction

Obese rodents e.g., the leptin-deficient (ob/ob) mouse exhibit remarkable behavioral changes and are therefore ideal models for evaluating mental disorders resulting from obesity. In doing so, female as well as male ob/ob mice at 8, 24, and 40 weeks of age underwent two common behavioral tests, namely the Open Field test and Elevated Plus Maze, to investigate behavioral alteration in a sex- and age dependent manner. The accuracy of these tests is often dependent on the observer that can subjectively influence the data.

Methods

To avoid this bias, mice were tracked with a video system. Video files were further analyzed by the compared use of two software, namely EthoVision (EV) and DeepLabCut (DLC). In DLC a Deep Learning application forms the basis for using artificial intelligence in behavioral research in the future, also with regard to the reduction of animal numbers.

Results

After no sex and partly also no age-related differences were found, comparison revealed that both software lead to almost identical results and are therefore similar in their basic outcomes, especially in the determination of velocity and total distance movement. Moreover, we observed additional benefits of DLC compared to EV as it enabled the interpretation of more complex behavior, such as rearing and leaning, in an automated manner.

Discussion

Based on the comparable results from both software, our study can serve as a starting point for investigating behavioral alterations in preclinical studies of obesity by using DLC to optimize and probably to predict behavioral observations in the future.

Deep learning-based behavioral profiling of rodent stroke recovery

Background

Stroke research heavily relies on rodent behavior when assessing underlying disease mechanisms and treatment efficacy. Although functional motor recovery is considered the primary targeted outcome, tests in rodents are still poorly reproducible and often unsuitable for unraveling the complex behavior after injury.

Results

Here, we provide a comprehensive 3D gait analysis of mice after focal cerebral ischemia based on the new deep learning-based software (DeepLabCut, DLC) that only requires basic behavioral equipment. We demonstrate a high precision 3D tracking of 10 body parts (including all relevant joints and reference landmarks) in several mouse strains. Building on this rigor motion tracking, a comprehensive post-analysis (with >100 parameters) unveils biologically relevant differences in locomotor profiles after a stroke over a time course of 3 weeks. We further refine the widely used ladder rung test using deep learning and compare its performance to human annotators. The generated DLC-assisted tests were then benchmarked to five widely used conventional behavioral set-ups (neurological scoring, rotarod, ladder rung walk, cylinder test, and single-pellet grasping) regarding sensitivity, accuracy, time use, and costs.

Conclusions

We conclude that deep learning-based motion tracking with comprehensive post-analysis provides accurate and sensitive data to describe the complex recovery of rodents following a stroke. The experimental set-up and analysis can also benefit a range of other neurological injuries that affect locomotion.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01434-9.

DeepLabCut increases markerless tracking efficiency in X-Ray video analysis of rodent locomotion

Despite the prevalence of rat models to study human disease and injury, existing methods for quantifying behavior through skeletal movements are problematic due to skin movement inaccuracies associated with optical video analysis; or, require invasive implanted markers or time consuming manual rotoscoping for x-ray video approaches. We examined the use of a machine learning tool, DeepLabCut, to perform automated, markerless tracking in bi-planar x-ray videos of locomoting rats. Models were trained on 590 pairs of video frames to identify 19 unique skeletal landmarks of the pelvic limb. Accuracy, precision, and time savings were assessed. Machine-identified landmarks deviated from manually labeled counterparts by 2.4±0.2 mm (n=1,710 landmarks). DeepLabCut decreased analysis time by over three orders of magnitude (1,627x) compared to manual labeling. Distribution of these models may enable the processing of a large volume of accurate x-ray kinematics locomotion data in a fraction of the time without requiring surgically implanted markers.

Markerless tracking enables distinction between strategic compensation and functional recovery after spinal cord injury

Injuries to the cervical spinal cord represent around 60% of all spinal cord injuries (SCIs). A major priority for patients with cervical SCIs is the recovery of any hand or arm function. The similarities between human and rodent "reach-to-eat movements" indicate that analyzing mouse forelimb reaching behavior may be a method of identifying clinically relevant treatments for people with cervical SCIs. One popular behavioral measure of forelimb functional recovery comprises the Single Pellet Retrieval Task (SPRT). The most common outcome measure for this task, however (percentage of pellets successfully retrieved), cannot readily distinguish between recovery of pre-injury motor patterns and strategic compensation. Our objective was to establish outcome measures for the SPRT that are readily adopted by different investigators and capable of measuring recovery of limb function after SCI. We used a simple semi-automated approach to high-speed tracking of mouse forepaw movements during pellet retrieval. DeepLabCut™, a machine learning based computer vision software package, was used to track individual features of the mouse forepaw, allowing a more detailed assessment of reaching behavior after SCI. Interestingly, kinematic analysis of movements pre- and post-injury illuminated persistent deficits in specific features of the reaching motor patterns, namely pronation and paw trajectory, that were poorly correlated with recovery of the ability to successfully retrieve pellets. Thus, we have developed an inexpensive method for detailed analysis of mouse reach-to-eat behavior following SCI. Further, our results suggest that binary success/fail outcome measures primarily assess an animal's ability to compensate rather than a restoration of normal function in the injured pathways and networks.

Supervised machine learning aided behavior classification in pigeons

Manual behavioral observations have been applied in both environment and laboratory experiments in order to analyze and quantify animal movement and behavior. Although these observations contributed tremendously to ecological and neuroscientific disciplines, there have been challenges and disadvantages following in their footsteps. They are not only time-consuming, labor-intensive, and error-prone but they can also be subjective, which induces further difficulties in reproducing the results. Therefore, there is an ongoing endeavor towards automated behavioral analysis, which has also paved the way for open-source software approaches. Even though these approaches theoretically can be applied to different animal groups, the current applications are mostly focused on mammals, especially rodents. However, extending those applications to other vertebrates, such as birds, is advisable not only for extending species-specific knowledge but also for contributing to the larger evolutionary picture and the role of behavior within. Here we present an open-source software package as a possible initiation of bird behavior classification. It can analyze pose-estimation data generated by established deep-learning-based pose-estimation tools such as DeepLabCut for building supervised machine learning predictive classifiers for pigeon behaviors, which can be broadened to support other bird species as well. We show that by training different machine learning and deep learning architectures using multivariate time series data as input, an F1 score of 0.874 can be achieved for a set of seven distinct behaviors. In addition, an algorithm for further tuning the bias of the predictions towards either precision or recall is introduced, which allows tailoring the classifier to specific needs.

Validating markerless pose estimation with 3D X-ray radiography

To reveal the neurophysiological underpinnings of natural movement, neural recordings must be paired with accurate tracking of limbs and postures. Here, we evaluated the accuracy of DeepLabCut (DLC), a deep learning markerless motion capture approach, by comparing it with a 3D X-ray video radiography system that tracks markers placed under the skin (XROMM). We recorded behavioral data simultaneously with XROMM and RGB video as marmosets foraged and reconstructed 3D kinematics in a common coordinate system. We used the toolkit Anipose to filter and triangulate DLC trajectories of 11 markers on the forelimb and torso and found a low median error (0.228 cm) between the two modalities corresponding to 2.0% of the range of motion. For studies allowing this relatively small error, DLC and similar markerless pose estimation tools enable the study of increasingly naturalistic behaviors in many fields including non-human primate motor control.

Applications and limitations of current markerless motion capture methods for clinical gait biomechanics

BACKGROUND

Markerless motion capture has the potential to perform movement analysis with reduced data collection and processing time compared to marker-based methods. This technology is now starting to be applied for clinical and rehabilitation applications and therefore it is crucial that users of these systems understand both their potential and limitations. This literature review aims to provide a comprehensive overview of the current state of markerless motion capture for both single camera and multi-camera systems. Additionally, this review explores how practical applications of markerless technology are being used in clinical and rehabilitation settings, and examines the future challenges and directions markerless research must explore to facilitate full integration of this technology within clinical biomechanics.

METHODOLOGY

A scoping review is needed to examine this emerging broad body of literature and determine where gaps in knowledge exist, this is key to developing motion capture methods that are cost effective and practically relevant to clinicians, coaches and researchers around the world. Literature searches were performed to examine studies that report accuracy of markerless motion capture methods, explore current practical applications of markerless motion capture methods in clinical biomechanics and identify gaps in our knowledge that are relevant to future developments in this area.

RESULTS

Markerless methods increase motion capture data versatility, enabling datasets to be re-analyzed using updated pose estimation algorithms and may even provide clinicians with the capability to collect data while patients are wearing normal clothing. While markerless temporospatial measures generally appear to be equivalent to marker-based motion capture, joint center locations and joint angles are not yet sufficiently accurate for clinical applications. Pose estimation algorithms are approaching similar error rates of marker-based motion capture, however, without comparison to a gold standard, such as bi-planar videoradiography, the true accuracy of markerless systems remains unknown.

CONCLUSIONS

Current open-source pose estimation algorithms were never designed for biomechanical applications, therefore, datasets on which they have been trained are inconsistently and inaccurately labelled. Improvements to labelling of open-source training data, as well as assessment of markerless accuracy against gold standard methods will be vital next steps in the development of this technology.

A New Look at Infant Problem-Solving: Using DeepLabCut to Investigate Exploratory Problem-Solving Approaches

When confronted with novel problems, problem-solvers must decide whether to copy a modeled solution or to explore their own unique solutions. While past work has established that infants can learn to solve problems both through their own exploration and through imitation, little work has explored the factors that influence which of these approaches infants select to solve a given problem. Moreover, past work has treated imitation and exploration as qualitatively distinct, although these two possibilities may exist along a continuum. Here, we apply a program novel to developmental psychology (DeepLabCut) to archival data (Lucca et al., 2020) to investigate the influence of the effort and success of an adult's modeled solution, and infants' firsthand experience with failure, on infants' imitative versus exploratory problem-solving approaches. Our results reveal that tendencies toward exploration are relatively immune to the information from the adult model, but that exploration generally increased in response to firsthand experience with failure. In addition, we found that increases in maximum force and decreases in trying time were associated with greater exploration, and that exploration subsequently predicted problem-solving success on a new iteration of the task. Thus, our results demonstrate that infants increase exploration in response to failure and that exploration may operate in a larger motivational framework with force, trying time, and expectations of task success.

Visual Field Analysis: A reliable method to score left and right eye use using automated tracking

Brain and behavioural asymmetries have been documented in various taxa. Many of these asymmetries involve preferential left and right eye use. However, measuring eye use through manual frame-by-frame analyses from video recordings is laborious and may lead to biases. Recent progress in technology has allowed the development of accurate tracking techniques for measuring animal behaviour. Amongst these techniques, DeepLabCut, a Python-based tracking toolbox using transfer learning with deep neural networks, offers the possibility to track different body parts with unprecedented accuracy. Exploiting the potentialities of DeepLabCut, we developed Visual Field Analysis, an additional open-source application for extracting eye use data. To our knowledge, this is the first application that can automatically quantify left–right preferences in eye use. Here we test the performance of our application in measuring preferential eye use in young domestic chicks. The comparison with manual scoring methods revealed a near perfect correlation in the measures of eye use obtained by Visual Field Analysis. With our application, eye use can be analysed reliably, objectively and at a fine scale in different experimental paradigms.

Moving outside the lab: Markerless motion capture accurately quantifies sagittal plane kinematics during the vertical jump

Markerless motion capture using deep learning approaches have potential to revolutionize the field of biomechanics by allowing researchers to collect data outside of the laboratory environment, yet there remain questions regarding the accuracy and ease of use of these approaches. The purpose of this study was to apply a markerless motion capture approach to extract lower limb angles in the sagittal plane during the vertical jump and to evaluate agreement between the custom trained model and gold standard motion capture. We performed this study using a large open source data set (N = 84) that included synchronized commercial video and gold standard motion capture. We split these data into a training set for model development (n = 69) and test set to evaluate capture performance relative to gold standard motion capture using coefficient of multiple correlations (CMC) (n = 15). We found very strong agreement between the custom trained markerless approach and marker-based motion capture within the test set across the entire movement (CMC > 0.991, RMSE < 3.22°), with at least strong CMC values across all trials for the hip (0.853 ± 0.23), knee (0.963 ± 0.471), and ankle (0.970 ± 0.055). The strong agreement between markerless and marker-based motion capture provides evidence that markerless motion capture is a viable tool to extend data collection to outside of the laboratory. As biomechanical research struggles with representative sampling practices, markerless motion capture has potential to transform biomechanical research away from traditional laboratory settings into venues convenient to populations that are under sampled without sacrificing measurement fidelity.

Maternal immune activation leads to atypical turning asymmetry and reduced DRD2 mRNA expression in a rat model of schizophrenia

Atypical asymmetries have been reported in individuals diagnosed with schizophrenia, linking higher symptom severity to weaker lateralization. Furthermore, both lateralization and schizophrenia are influenced by the dopaminergic system. However, whether a direct link between the etiology of schizophrenia and atypical asymmetries exists is yet to be investigated. In this study, we examined whether maternal immune activation (MIA), a developmental animal model for schizophrenia and known to alter the dopaminergic system, induces atypical lateralization in adolescent and adult offspring. As the dopaminergic system is a key player in both, we analyzed neuronal dopamine D2 receptor (DRD2) mRNA expression. MIA was induced by injecting pregnant rats with 10 mg/kg polyinosinic:polycytidylic (PolyI:C) at gestational day 15. Controls were injected with 0.9 % NaCl. Offspring were tested at adolescence or early adulthood for asymmetry of turning behavior in the open field test. The total number of left and right turns per animal was assessed using DeepLabCut. Strength and preferred side of asymmetry were analyzed by calculating lateralization quotients. Additionally, DRD2 mRNA expression in the prefrontal cortex of offspring at both ages was analyzed using real-time PCR. MIA was associated with a rightward turning behavior in adolescents. In adults, MIA was associated with an absence of turning bias, indicating reduced asymmetry after MIA. The analysis of DRD2 mRNA expression revealed significantly lower mRNA levels after MIA compared to controls in adolescent, but not adult animals. Our results reinforce the association between atypical asymmetries, reduced DRD2 mRNA expression, and schizophrenia. However, more preclinical research is needed.

A Deep Learning-Based Approach to Video-Based Eye Tracking for Human Psychophysics

Real-time gaze tracking provides crucial input to psychophysics studies and neuromarketing applications. Many of the modern eye-tracking solutions are expensive mainly due to the high-end processing hardware specialized for processing infrared-camera pictures. Here, we introduce a deep learning-based approach which uses the video frames of low-cost web cameras. Using DeepLabCut (DLC), an open-source toolbox for extracting points of interest from videos, we obtained facial landmarks critical to gaze location and estimated the point of gaze on a computer screen via a shallow neural network. Tested for three extreme poses, this architecture reached a median error of about one degree of visual angle. Our results contribute to the growing field of deep-learning approaches to eye-tracking, laying the foundation for further investigation by researchers in psychophysics or neuromarketing.

Mirror Self-Recognition in Pigeons: Beyond the Pass-or-Fail Criterion

Spontaneous mirror self-recognition is achieved by only a limited number of species, suggesting a sharp "cognitive Rubicon" that only few can pass. But is the demarcation line that sharp? In studies on monkeys, who do not recognize themselves in a mirror, animals can make a difference between their mirror image and an unknown conspecific. This evidence speaks for a gradualist view of mirror self-recognition. We hypothesize that such a gradual process possibly consists of at least two independent aptitudes, the ability to detect synchronicity between self- and foreign movement and the cognitive understanding that the mirror reflection is oneself. Pigeons are known to achieve the first but fail at the second aptitude. We therefore expected them to treat their mirror image differently from an unknown pigeon, without being able to understand that the mirror reflects their own image. We tested pigeons in a task where they either approached a mirror or a Plexiglas barrier to feed. Behind the Plexiglas an unknown pigeon walked at the same time toward the food bowl. Thus, we pitched a condition with a mirror-self and a foreign bird against each other, with both of them walking close toward the food bowl. By a detailed analysis of a whole suit of behavioral details, our results make it likely that the foreign pigeon was treated as a competitor while the mirror image caused hesitation as if being an uncanny conspecific. Our results are akin to those with monkeys and show that pigeons do not equal their mirror reflection with a conspecific, although being unable to recognize themselves in the mirror.

Real-time, low-latency closed-loop feedback using markerless posture tracking

The ability to control a behavioral task or stimulate neural activity based on animal behavior in real-time is an important tool for experimental neuroscientists. Ideally, such tools are noninvasive, low-latency, and provide interfaces to trigger external hardware based on posture. Recent advances in pose estimation with deep learning allows researchers to train deep neural networks to accurately quantify a wide variety of animal behaviors. Here, we provide a new DeepLabCut-Live! package that achieves low-latency real-time pose estimation (within 15 ms, >100 FPS), with an additional forward-prediction module that achieves zero-latency feedback, and a dynamic-cropping mode that allows for higher inference speeds. We also provide three options for using this tool with ease: (1) a stand-alone GUI (called DLC-Live! GUI), and integration into (2) Bonsai, and (3) AutoPilot. Lastly, we benchmarked performance on a wide range of systems so that experimentalists can easily decide what hardware is required for their needs.

Integrating XMALab and DeepLabCut for high-throughput XROMM

Marker tracking is a major bottleneck in studies involving X-ray reconstruction of moving morphology (XROMM). Here, we tested whether DeepLabCut, a new deep learning package built for markerless tracking, could be applied to videoradiographic data to improve data processing throughput. Our novel workflow integrates XMALab, the existing XROMM marker tracking software, and DeepLabCut while retaining each program's utility. XMALab is used for generating training datasets, error correction and 3D reconstruction, whereas the majority of marker tracking is transferred to DeepLabCut for automatic batch processing. In the two case studies that involved an in vivo behavior, our workflow achieved a 6 to 13-fold increase in data throughput. In the third case study, which involved an acyclic, post-mortem manipulation, DeepLabCut struggled to generalize to the range of novel poses and did not surpass the throughput of XMALab alone. Deployed in the proper context, this new workflow facilitates large scale XROMM studies that were previously precluded by software constraints.

Visual Attack on the Moving Prey by Cuttlefish

Visual attack for prey capture in cuttlefish involves three well characterized sequential stages: attention, positioning, and seizure. This visually guided behavior requires accurate sensorimotor integration of information on the target’s direction and tentacular strike control. While the behavior of cuttlefish visual attack on a stationary prey has been described qualitatively, the kinematics of visual attack on a moving target has not been analyzed quantitatively. A servomotor system controlling the movement of a shrimp prey and a high resolution imaging system recording the behavior of the cuttlefish predator, together with the newly developed DeepLabCut image processing system, were used to examine the tactics used by cuttlefish during a visual attack on moving prey. The results showed that cuttlefish visually tracked a moving prey target using mainly body movement, and that they maintained a similar speed to that of the moving prey right before making their tentacular strike. When cuttlefish shot out their tentacles for prey capture, they were able to either predict the target location based on the prey’s speed and compensate for the inherent sensorimotor delay or adjust the trajectory of their tentacular strike according to the prey’s direction of movement in order to account for any changes in prey position. These observations suggest that cuttlefish use the various visual tactics available to them flexibly in order to capture moving prey, and that they are able to extract direction and speed information from moving prey in order to allow an accurate visual attack.

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

Here, we describe a system capable of tracking specific mouse paw movements at high frame rates (70.17 Hz) with a high level of accuracy (mean = 0.95, SD < 0.01). Short-latency markerless tracking of specific body parts opens up the possibility of manipulating motor feedback. We present a software and hardware scheme built on DeepLabCut—a robust movement-tracking deep neural network framework—which enables real-time estimation of paw and digit movements of mice. Using this approach, we demonstrate movement-generated feedback by triggering a USB-GPIO (general-purpose input/output)-controlled LED when the movement of one paw, but not the other, selectively exceeds a preset threshold. The mean time delay between paw movement initiation and LED flash was 44.41 ms (SD = 36.39 ms), a latency sufficient for applying behaviorally triggered feedback. We adapt DeepLabCut for real-time tracking as an open-source package we term DeepCut2RealTime. The ability of the package to rapidly assess animal behavior was demonstrated by reinforcing specific movements within water-restricted, head-fixed mice. This system could inform future work on a behaviorally triggered “closed loop” brain–machine interface that could reinforce behaviors or deliver feedback to brain regions based on prespecified body movements.

Self-Generated Whisker Movements Drive State-Dependent Sensory Input to Developing Barrel Cortex

Cortical development is an activity-dependent process [1-3]. Regarding the role of activity in the developing somatosensory cortex, one persistent debate concerns the importance of sensory feedback from self-generated movements. Specifically, recent studies claim that cortical activity is generated intrinsically, independent of movement [3, 4]. However, other studies claim that behavioral state moderates the relationship between movement and cortical activity [5-7]. Thus, perhaps inattention to behavioral state leads to failures to detect movement-driven activity [8]. Here, we resolve this issue by associating local field activity (i.e., spindle bursts) and unit activity in the barrel cortex of 5-day-old rats with whisker movements during wake and myoclonic twitches of the whiskers during active (REM) sleep. Barrel activity increased significantly within 500 ms of whisker movements, especially after twitches. Also, higher-amplitude movements were more likely to trigger barrel activity; when we controlled for movement amplitude, barrel activity was again greater after a twitch than a wake movement. We then inverted the analysis to assess the likelihood that increases in barrel activity were preceded within 500 ms by whisker movements: at least 55% of barrel activity was attributable to sensory feedback from whisker movements. Finally, when periods with and without movement were compared, 70%-75% of barrel activity was movement related. These results confirm the importance of sensory feedback from movements in driving activity in sensorimotor cortex and underscore the necessity of monitoring sleep-wake states to ensure accurate assessments of the contributions of the sensory periphery to activity in developing somatosensory cortex.