The iFly tracking system for an automated locomotor and behavioural analysis of Drosophila melanogaster

Comparative analysis of tracking and behavioral patterns between wild-type and genetically modified fruit flies using computer vision and statistical methods

Collective animal behavior occurs in groups and swarms at almost every biological scale, from single-celled organisms to the largest animals on Earth. The intriguing mysteries behind these group behaviors have attracted many scholars, and while it is known that models can reproduce qualitative features of such complex behaviors, this requires data from real animals to demonstrate, and obtaining data on the exact features of these groups is tricky. In this paper, we propose the Hidden Markov Unscented Tracker (HMUT), which combines the state prediction capability of HMM and the high-precision nonlinear processing capability of UKF. This prediction-driven tracking mechanism enables HMUT to quickly adjust tracking strategies when facing sudden changes in target motion direction or rapid changes in speed, reducing the risk of tracking loss. Videos of fruit fly swarm movement in an enclosed environment are captured using stereo cameras. For the captured fruit fly images, the thresholded AKAZE algorithm is first used to detect the positions of individual fruit flies in the images, and the motion of the fruit flies is modeled using a multidimensional hidden Markov model (HMM). Tracking is then performed using the Unscented Kalman Filter algorithm to obtain the flight trajectories of the fruit flies in two camera views. Finally, 3D reconstruction of the trajectories in both views is achieved through polar coordinate constraints, resulting in 3D motion data of the fruit flies. Additionally, the efficiency and accuracy of the proposed algorithm are evaluated by simulating fruit fly swarm movement using the Boids algorithm. Finally, based on the tracked fruit fly flight data, behavioral characteristics of the fruit flies are analyzed from two perspectives. The first is a statistical analysis of the differences between the two behaviors. The second dimension involves clustering trajectory similarity using the DTW method based on fruit fly flight trajectories, further analyzing the similarity within clusters and differences between clusters.

High-resolution, high-throughput analysis of Drosophila geotactic behavior

Drosophila innate response to gravity, geotaxis, has been previously used to assess the impact of aging and disease on motor performance. Despite its rich history, fly geotaxis continues to be largely measured manually and assessed through simplistic metrics. The manual nature of this assay introduces substantial experimental variability while simplistic metrics provide limited analytic insights into the behavior. To address these shortcomings, we have constructed a fully automated, programable apparatus, and developed a multi-object tracking software capable of following sub-second movements of individual flies, thus allowing reproducible, detailed, and quantitative analysis of geotactic behavior. The apparatus triggers and monitors geotaxis of 10 fly cohorts simultaneously, with each cohort consisting of up to 7 flies. The tracking program isolates cohorts and records individual fly coordinate outputs allowing for simultaneous multi-group, multi-fly tracks per experiment, greatly improving throughput and resolution. The algorithm tracks individual flies during the entire run with ~97% accuracy, yielding detailed climbing curve, speed, and movement direction with 1/30 second resolution. Our tracking also allows the construction of multi-variable metrics and the detection of transitory movement phenotypes, such as slips and falls, which have thus far been neglected in geotaxis studies due to limited spatio-temporal resolution. Through a combination of automation and robust tracking, the platform is therefore poised to advance Drosophila geotaxis assay into a comprehensive assessment of locomotor behavior.

Tracking the flight and landing behaviour of western flower thrips in response to single and two-colour cues

Real-time 3D tracking and high-speed videography was used to examine the behaviour of a worldwide greenhouse pest, the western flower thrips (WFT), in response to different colours in the context of improving trap design. Measurements were taken of the number of landings on, and flight activity near, a lamp containing two LEDs of either the same colour or a combination of two colours presented side by side. Main findings show that landing patterns of WFT are different between colours, with landings on UV(+ red) as highly attractive stimulus being mostly distributed at the bottom half of the lamp, while for yellow also as very attractive and green as a 'neutral' stimulus, landings were clearly on the upper rim of the lamp. Additionally, a positive interaction with the UV-A(+ red) and yellow combination elicited the highest number of landings and flight time in front of the LED lamp. Conversely, a negative interaction was observed with decreased landings and flight time found for yellow when blue was present as the adjacent colour. Overall, differences between treatments were less obvious for flight times compared to number of landings, with tracking data suggesting that WFT might use different colours to orientate at different distances as they approach a visual stimulus.

Whole Animal Feeding FLat (WAFFL): a complete and comprehensive validation of a novel, high-throughput fly experimentation system

Non-mammalian model organisms have been essential for our understanding of the mechanisms that control development, disease, and physiology, but they are underutilized in pharmacological and toxicological phenotypic screening assays due to their low throughput in comparison with cell-based screens. To increase the utility of using Drosophila melanogaster in screening, we designed the Whole Animal Feeding FLat (WAFFL), a novel, flexible, and complete system for feeding, monitoring, and assaying flies in a high-throughput format. Our 3D printed system is compatible with inexpensive and readily available, commercial 96-well plate consumables and equipment. Experimenters can change the diet at will during the experiment and video record for behavior analysis, enabling precise dosing, measurement of feeding, and analysis of behavior in a 96-well plate format.

Artificial gravity partially protects space-induced neurological deficits in Drosophila melanogaster

Spaceflight poses risks to the central nervous system (CNS), and understanding neurological responses is important for future missions. We report CNS changes in Drosophila aboard the International Space Station in response to spaceflight microgravity (SFμg) and artificially simulated Earth gravity (SF1g) via inflight centrifugation as a countermeasure. While inflight behavioral analyses of SFμg exhibit increased activity, postflight analysis displays significant climbing defects, highlighting the sensitivity of behavior to altered gravity. Multi-omics analysis shows alterations in metabolic, oxidative stress and synaptic transmission pathways in both SFμg and SF1g; however, neurological changes immediately postflight, including neuronal loss, glial cell count alterations, oxidative damage, and apoptosis, are seen only in SFμg. Additionally, progressive neuronal loss and a glial phenotype in SF1g and SFμg brains, with pronounced phenotypes in SFμg, are seen upon acclimation to Earth conditions. Overall, our results indicate that artificial gravity partially protects the CNS from the adverse effects of spaceflight.

Roles of α-Synuclein and Disease-Associated Factors in Drosophila Models of Parkinson's Disease

α-Synuclein (αSyn) plays a major role in the pathogenesis of Parkinson’s disease (PD), which is the second most common neurodegenerative disease after Alzheimer’s disease. The accumulation of αSyn is a pathological hallmark of PD, and mutations in the SNCA gene encoding αSyn cause familial forms of PD. Moreover, the ectopic expression of αSyn has been demonstrated to mimic several key aspects of PD in experimental model systems. Among the various model systems, Drosophila melanogaster has several advantages for modeling human neurodegenerative diseases. Drosophila has a well-defined nervous system, and numerous tools have been established for its genetic analyses. The rapid generation cycle and short lifespan of Drosophila renders them suitable for high-throughput analyses. PD model flies expressing αSyn have contributed to our understanding of the roles of various disease-associated factors, including genetic and nongenetic factors, in the pathogenesis of PD. In this review, we summarize the molecular pathomechanisms revealed to date using αSyn-expressing Drosophila models of PD, and discuss the possibilities of using these models to demonstrate the biological significance of disease-associated factors.

Three-Dimensional Tracking of Multiple Small Insects by a Single Camera

Many systems to monitor insect behavior have been developed recently. Yet most of these can only detect two-dimensional behavior for convenient analysis and exclude other activities, such as jumping or flying. Therefore, the development of a three-dimensional (3D) monitoring system is necessary to investigate the 3D behavior of insects. In such a system, multiple-camera setups are often used to accomplish this purpose. Here, a system with a single camera for tracking small insects in a 3D space is proposed, eliminating the synchronization problems that typically occur when multiple cameras are instead used. With this setup, two other images are obtained via mirrors fixed at other viewing angles. Using the proposed algorithms, the tracking accuracy of five individual drain flies, Clogmia albipunctata (Williston) (Diptera: Psychodidae), flitting about in a spherical arena (78 mm in diameter) is as high as 98.7%, whereas the accuracy of 10 individuals is 96.3%. With this proposed method, the 3D trajectory monitoring experiments of insects can be performed more efficiently.

Exploring Aβ Proteotoxicity and Therapeutic Candidates Using Drosophila melanogaster

Alzheimer’s disease is a widespread and devastating neurological disorder associated with proteotoxic events caused by the misfolding and aggregation of the amyloid-β peptide. To find therapeutic strategies to combat this disease, Drosophila melanogaster has proved to be an excellent model organism that is able to uncover anti-proteotoxic candidates due to its outstanding genetic toolbox and resemblance to human disease genes. In this review, we highlight the use of Drosophila melanogaster to both study the proteotoxicity of the amyloid-β peptide and to screen for drug candidates. Expanding the knowledge of how the etiology of Alzheimer’s disease is related to proteotoxicity and how drugs can be used to block disease progression will hopefully shed further light on the field in the search for disease-modifying treatments.

α-Synuclein E46K Mutation and Involvement of Oxidative Stress in a Drosophila Model of Parkinson's Disease

Parkinson's disease (PD) is an age-associated neurodegenerative condition in which some genetic variants are known to increase disease susceptibility on interaction with environmental factors inducing oxidative stress. Different mutations in the SNCA gene are reported as the major genetic contributors to PD. E46K mutation pathogenicity has not been investigated as intensive as other SNCA gene mutations including A30P and A53T. In this study, based on the GAL4-UAS binary genetic tool, transgenic Drosophila melanogaster flies expressing wild-type and E46K-mutated copies of the human SNCA gene were constructed. Western blotting, immunohistochemical analysis, and light and confocal microscopy of flies' brains were undertaken along with the survival rate measurement, locomotor function assay, and ethanol and paraquat (PQ) tolerance to study α-synuclein neurotoxicity. Biochemical bioassays were carried out to investigate the activity of antioxidant enzymes and alterations in levels of oxidative markers following damages induced by human α-synuclein to the neurons of the transgenic flies. Overexpression of human α-synuclein in the central nervous system of these transgenic flies led to disorganized ommatidia structures and loss of dopaminergic neurons. E46K α-synuclein caused remarkable climbing defects, reduced survivorship, higher ethanol sensitivity, and increased PQ-mediated mortality. A noticeable decline in activity of catalase and superoxide dismutase enzymes besides considerable increase in the levels of lipid peroxidation and reactive oxygen species was observed in head capsule homogenates of α-synuclein-expressing flies, which indicates obvious involvement of oxidative stress as a causal factor in SNCA^E46K neurotoxicity. In all the investigations, E46K copy of the SNCA gene was found to impose more severe defects when compared to wild-type SNCA. It can be concluded that the constructed Drosophila models developed PD-like symptoms that facilitate comparative studies of molecular and cellular pathways implicated in the pathogenicity of different α-synuclein mutations.

In vivo Evaluation of a Newly Synthesized Acetylcholinesterase Inhibitor in a Transgenic Drosophila Model of Alzheimer's Disease

Alzheimer's disease is a neurodegenerative disease characterized by disrupted memory, learning functions, reduced life expectancy, and locomotor dysfunction, as a result of the accumulation and aggregation of amyloid peptides that cause neuronal damage in neuronal circuits. In the current study, we exploited a transgenic Drosophila melanogaster line, expressing amyloid-β peptides to investigate the efficacy of a newly synthesized acetylcholinesterase inhibitor, named XJP-1, as a potential AD therapy. Behavioral assays and confocal microscopy were used to characterize the drug effect on AD symptomatology and amyloid peptide deposition. The symptomatology induced in this particular transgenic model recapitulates the scenario observed in human AD patients, showing a shortened lifespan and reduced locomotor functions, along with a significant accumulation of amyloid plaques in the brain. XJP-1 treatment resulted in a significant improvement of AD symptoms and a reduction of amyloid plaques by diminishing the amyloid aggregation rate. In comparison with clinically effective AD drugs, our results demonstrated that XJP-1 has similar effects on AD symptomatology, but at 10 times lower drug concentration than donepezil. It also showed an earlier beneficial effect on the reduction of amyloid plaques at 10 days after drug treatment, as observed for donepezil at 20 days, while the other drugs tested have no such effect. As a novel and potent AChE inhibitor, our study demonstrates that inhibition of the enzyme AChE by XJP-1 treatment improves the amyloid-induced symptomatology in Drosophila, by reducing the number of amyloid plaques within the fruit fly CNS. Thus, compound XJP-1 has the therapeutic potential to be further investigated for the treatment of AD.

FreeClimber: automated quantification of climbing performance in Drosophila

Negative geotaxis (climbing) performance is a useful metric for quantifying Drosophila health. Manual methods to quantify climbing performance are tedious and often biased, while many available computational methods have challenging hardware or software requirements. We present an alternative: FreeClimber. This open source, Python-based platform subtracts a video's static background to improve detection for flies moving across heterogeneous backgrounds. FreeClimber calculates a cohort's velocity as the slope of the most linear portion of a mean vertical position versus time curve. It can run from a graphical user interface for optimization or a command line interface for high-throughput and automated batch processing, improving accessibility for users with different expertise. FreeClimber outputs calculated slopes, spot locations for follow-up analyses (e.g. tracking), and several visualizations and plots. We demonstrate FreeClimber's utility in a longitudinal study for endurance exercise performance in Drosophila mitonuclear genotypes using six distinct mitochondrial haplotypes paired with a common D. melanogaster nuclear background.

High-resolution behavioral mapping of electric fishes in Amazonian habitats

The study of animal behavior has been revolutionized by sophisticated methodologies that identify and track individuals in video recordings. Video recording of behavior, however, is challenging for many species and habitats including fishes that live in turbid water. Here we present a methodology for identifying and localizing weakly electric fishes on the centimeter scale with subsecond temporal resolution based solely on the electric signals generated by each individual. These signals are recorded with a grid of electrodes and analyzed using a two-part algorithm that identifies the signals from each individual fish and then estimates the position and orientation of each fish using Bayesian inference. Interestingly, because this system involves eavesdropping on electrocommunication signals, it permits monitoring of complex social and physical interactions in the wild. This approach has potential for large-scale non-invasive monitoring of aquatic habitats in the Amazon basin and other tropical freshwater systems.

Effects of small-molecule amyloid modulators on a Drosophila model of Parkinson's disease

Alpha-synuclein (aS) amyloid formation is involved in Parkinson's disease (PD); therefore, small molecules that target aS and affect its aggregation are of interest as future drug candidates. We recently reported modified ring-fused 2-pyridones that modulate aS amyloid formation in vitro. Here, we describe the effects of such molecules on behavioral parameters of a Drosophila model of PD (i.e., flies expressing human aS), using a new approach (implemented in a commercially available FlyTracker system) to quantify fly mobility. FlyTracker allows for automated analysis of walking and climbing locomotor behavior, as it collects large sequences of data over time in an unbiased manner. We found that the molecules per se have no toxic or kinetic effects on normal flies. Feeding aS-expressing flies with the amyloid-promoting molecule FN075, remarkably, resulted in increased fly mobility at early time points; however, this effect switched to reduced mobility at later time points, and flies had shorter life spans than controls. In contrast, an amyloid inhibitor increased both fly kinetics and life span. In agreement with increased aS amyloid formation, the FN075-fed flies had less soluble aS, and in vitro aS-FN075 interactions stimulated aS amyloid formation. In addition to a new quantitative approach to probe mobility (available in FlyTracker), our results imply that aS regulates brain activity such that initial removal (here, by FN075-triggered assembly of aS) allows for increased fly mobility.

The effects of ER morphology on synaptic structure and function in Drosophila melanogaster

Hereditary spastic paraplegia (HSP) is a set of genetic diseases caused by mutations in one of 72 genes that results in age-dependent corticospinal axon degeneration accompanied by spasticity and paralysis. Two genes implicated in HSPs encode proteins that regulate endoplasmic reticulum (ER) morphology. Atlastin 1 (ATL1, also known as SPG3A) encodes an ER membrane fusion GTPase and reticulon 2 (RTN2, also known as SPG12) helps shape ER tube formation. Here, we use a new fluorescent ER marker to show that the ER within wild-type Drosophila motor nerve terminals forms a network of tubules that is fragmented and made diffuse upon loss of the atlastin 1 ortholog atl. atl or Rtnl1 loss decreases evoked transmitter release and increases arborization. Similar to other HSP proteins, Atl inhibits bone morphogenetic protein (BMP) signaling, and loss of atl causes age-dependent locomotor deficits in adults. These results demonstrate a crucial role for ER in neuronal function, and identify mechanistic links between ER morphology, neuronal function, BMP signaling and adult behavior.

Analyzing animal behavior via classifying each video frame using convolutional neural networks

High-throughput analysis of animal behavior requires software to analyze videos. Such software analyzes each frame individually, detecting animals’ body parts. But the image analysis rarely attempts to recognize “behavioral states”—e.g., actions or facial expressions—directly from the image instead of using the detected body parts. Here, we show that convolutional neural networks (CNNs)—a machine learning approach that recently became the leading technique for object recognition, human pose estimation, and human action recognition—were able to recognize directly from images whether Drosophila were “on” (standing or walking) or “off” (not in physical contact with) egg-laying substrates for each frame of our videos. We used multiple nets and image transformations to optimize accuracy for our classification task, achieving a surprisingly low error rate of just 0.072%. Classifying one of our 8 h videos took less than 3 h using a fast GPU. The approach enabled uncovering a novel egg-laying-induced behavior modification in Drosophila. Furthermore, it should be readily applicable to other behavior analysis tasks.

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning

A lack of automated, quantitative, and accurate assessment of social behaviors in mammalian animal models has limited progress toward understanding mechanisms underlying social interactions and their disorders such as autism. Here we present a new integrated hardware and software system that combines video tracking, depth sensing, and machine learning for automatic detection and quantification of social behaviors involving close and dynamic interactions between two mice of different coat colors in their home cage. We designed a hardware setup that integrates traditional video cameras with a depth camera, developed computer vision tools to extract the body "pose" of individual animals in a social context, and used a supervised learning algorithm to classify several well-described social behaviors. We validated the robustness of the automated classifiers in various experimental settings and used them to examine how genetic background, such as that of Black and Tan Brachyury (BTBR) mice (a previously reported autism model), influences social behavior. Our integrated approach allows for rapid, automated measurement of social behaviors across diverse experimental designs and also affords the ability to develop new, objective behavioral metrics.

Estimating Orientation of Flying Fruit Flies

The recently growing interest in studying flight behaviours of fruit flies, Drosophila melanogaster, has highlighted the need for developing tools that acquire quantitative motion data. Despite recent advance of video tracking systems, acquiring a flying fly’s orientation remains a challenge for these tools. In this paper, we present a novel method for estimating individual flying fly’s orientation using image cues. Thanks to the line reconstruction algorithm in computer vision field, this work can thereby focus on the practical detail of implementation and evaluation of the orientation estimation algorithm. The orientation estimation algorithm can be incorporated into tracking algorithms. We rigorously evaluated the effectiveness and accuracy of the proposed algorithm by running experiments both on simulation data and on real-world data. This work complements methods for studying the fruit fly’s flight behaviours in a three-dimensional environment.

Long-duration animal tracking in difficult lighting conditions

High-throughput analysis of animal behavior requires software to analyze videos. Such software typically depends on the experiments' being performed in good lighting conditions, but this ideal is difficult or impossible to achieve for certain classes of experiments. Here, we describe techniques that allow long-duration positional tracking in difficult lighting conditions with strong shadows or recurring "on"/"off" changes in lighting. The latter condition will likely become increasingly common, e.g., for Drosophila due to the advent of red-shifted channel rhodopsins. The techniques enabled tracking with good accuracy in three types of experiments with difficult lighting conditions in our lab. Our technique handling shadows relies on single-animal tracking and on shadows' and flies' being accurately distinguishable by distance to the center of the arena (or a similar geometric rule); the other techniques should be broadly applicable. We implemented the techniques as extensions of the widely-used tracking software Ctrax; however, they are relatively simple, not specific to Drosophila, and could be added to other trackers as well.

Toward a science of computational ethology

The new field of “Computational Ethology” is made possible by advances in technology, mathematics, and engineering that allow scientists to automate the measurement and the analysis of animal behavior. We explore the opportunities and long-term directions of research in this area.

A low-cost method for analyzing seizure-like activity and movement in Drosophila

Video tracking systems have been used widely to analyze Drosophila melanogaster movement and detect various abnormalities in locomotive behavior. While these systems can provide a wealth of behavioral information, the cost and complexity of these systems can be prohibitive for many labs. We have developed a low-cost assay for measuring locomotive behavior and seizure movement in D. melanogaster. The system uses a web-cam to capture images that can be processed using a combination of inexpensive and free software to track the distance moved, the average velocity of movement and the duration of movement during a specified time-span. To demonstrate the utility of this system, we examined a group of D. melanogaster mutants, the Bang-sensitive (BS) paralytics, which are 3-10 times more susceptible to seizure-like activity (SLA) than wild type flies. Using this novel system, we were able to detect that the BS mutant bang senseless (bss) exhibits lower levels of exploratory locomotion in a novel environment than wild type flies. In addition, the system was used to identify that the drug metformin, which is commonly used to treat type II diabetes, reduces the intensity of SLA in the BS mutants.

Automated analysis of courtship suppression learning and memory in Drosophila melanogaster

Study of the fruit fly, Drosophila melanogaster, has yielded important insights into the underlying molecular mechanisms of learning and memory. Courtship conditioning is a well-established behavioral assay used to study Drosophila learning and memory. Here, we describe the development of software to analyze courtship suppression assay data that correctly identifies normal or abnormal learning and memory traits of individual flies. Development of this automated analysis software will significantly enhance our ability to use this assay in large-scale genetic screens and disease modeling. The software increases the consistency, objectivity, and types of data generated.

Three-dimensional tracking and behaviour monitoring of multiple fruit flies

The increasing interest in the investigation of social behaviours of a group of animals has heightened the need for developing tools that provide robust quantitative data. Drosophila melanogaster has emerged as an attractive model for behavioural analysis; however, there are still limited ways to monitor fly behaviour in a quantitative manner. To study social behaviour of a group of flies, acquiring the position of each individual over time is crucial. There are several studies that have tried to solve this problem and make this data acquisition automated. However, none of these studies has addressed the problem of keeping track of flies for a long period of time in three-dimensional space. Recently, we have developed an approach that enables us to detect and keep track of multiple flies in a three-dimensional arena for a long period of time, using multiple synchronized and calibrated cameras. After detecting flies in each view, correspondence between views is established using a novel approach we call the 'sequential Hungarian algorithm'. Subsequently, the three-dimensional positions of flies in space are reconstructed. We use the Hungarian algorithm and Kalman filter together for data association and tracking. We evaluated rigorously the system's performance for tracking and behaviour detection in multiple experiments, using from one to seven flies. Overall, this system presents a powerful new method for studying complex social interactions in a three-dimensional environment.

Automated tracking of animal posture and movement during exploration and sensory orientation behaviors

BACKGROUND

The nervous functions of an organism are primarily reflected in the behavior it is capable of. Measuring behavior quantitatively, at high-resolution and in an automated fashion provides valuable information about the underlying neural circuit computation. Accordingly, computer-vision applications for animal tracking are becoming a key complementary toolkit to genetic, molecular and electrophysiological characterization in systems neuroscience.

METHODOLOGY/PRINCIPAL FINDINGS

We present Sensory Orientation Software (SOS) to measure behavior and infer sensory experience correlates. SOS is a simple and versatile system to track body posture and motion of single animals in two-dimensional environments. In the presence of a sensory landscape, tracking the trajectory of the animal's sensors and its postural evolution provides a quantitative framework to study sensorimotor integration. To illustrate the utility of SOS, we examine the orientation behavior of fruit fly larvae in response to odor, temperature and light gradients. We show that SOS is suitable to carry out high-resolution behavioral tracking for a wide range of organisms including flatworms, fishes and mice.

CONCLUSIONS/SIGNIFICANCE

Our work contributes to the growing repertoire of behavioral analysis tools for collecting rich and fine-grained data to draw and test hypothesis about the functioning of the nervous system. By providing open-access to our code and documenting the software design, we aim to encourage the adaptation of SOS by a wide community of non-specialists to their particular model organism and questions of interest.