Automatic multiple zebrafish tracking based on improved HOG features

Anxiety in aquatics: Leveraging machine learning models to predict adult zebrafish behavior

Accurate analysis of anxiety behaviors in animal models is pivotal for advancing neuroscience research and drug discovery. This study compares the potential of DeepLabCut, ZebraLab, and machine learning models to analyze anxiety-related behaviors in adult zebrafish. Using a dataset comprising video recordings of unstressed and pre-stressed zebrafish, we extracted features such as total inactivity duration/immobility, time spent at the bottom, time spent at the top and turn angles (large and small). We observed that the data obtained using DeepLabCut and ZebraLab were highly correlated. Using this data, we annotated behaviors as anxious and not anxious and trained several machine learning models, including Logistic Regression, Decision Tree, K-Nearest Neighbours (KNN), Random Forests, Naive Bayes Classifiers, and Support Vector Machines (SVMs). The effectiveness of these machine learning models was validated and tested on independent datasets. We found that some machine learning models, such as Decision Tree and Random Forests, performed excellently to differentiate between anxious and non-anxious behavior, even in the control group, where the differences between subjects were more subtle. Our findings show that upcoming technologies, such as machine learning models, are able to effectively and accurately analyze anxiety behaviors in zebrafish and provide a cost-effective method to analyze animal behavior.

Zebrafishtracker3D: A 3D skeleton tracking algorithm for multiple zebrafish based on particle matching

Zebrafish are considered as model organisms in biological and medical research because of their high degree of homology with human genes. Automatic behavioral analysis of multiple zebrafish based on visual tracking is expected to improve research efficiency. However, vision-based multi-object tracking algorithms often suffer from data loss owing to mutual occlusion. In addition, simply tracking zebrafish as points is not sufficient-more detailed information, which is required for research on zebrafish behavior. In this paper, we propose Zebrafishtracker3D, which utilizes a skeleton stability strategy to reduce detection error caused by frequent overlapping of multiple zebrafish effectively and estimates zebrafish skeletons using head coordinates in the top view. Further, we transform the front- and top-view matching task into an optimization problem and propose a particle-matching method to perform 3D tracking. The robustness of the algorithm with respect to occlusion is estimated on the dataset comprising two and three zebrafish. Experimental results demonstrate that the proposed algorithm exhibits a multiple object tracking accuracy (MOTA) exceeding 90% in the top view and a 3D tracking matching accuracy exceeding 90% in the complex videos with frequent overlapping. It is noteworthy that each instance in the trace saves its skeleton. In addition, Zebrafishtracker3D is applied in the zebrafish courtship experiment, establishes the stability of the method in applications of life science, and proves that the data can be used for behavioral analysis. Zebrafishtracker3D is the first algorithm that realizes 3D skeleton tracking of multiple zebrafish simultaneously.

Bringing Artificial Intelligence (AI) into Environmental Toxicology Studies: A Perspective of AI-Enabled Zebrafish High-Throughput Screening

The booming development of artificial intelligence (AI) has brought excitement to many research fields that could benefit from its big data analysis capability for causative relationship establishment and knowledge generation. In toxicology studies using zebrafish, the microscopic images and videos that illustrate the developmental stages, phenotypic morphologies, and animal behaviors possess great potential to facilitate rapid hazard assessment and dissection of the toxicity mechanism of environmental pollutants. However, the traditional manual observation approach is both labor-intensive and time-consuming. In this Perspective, we aim to summarize the current AI-enabled image and video analysis tools to realize the full potential of AI. For image analysis, AI-based tools allow fast and objective determination of morphological features and extraction of quantitative information from images of various sorts. The advantages of providing accurate and reproducible results while avoiding human intervention play a critical role in speeding up the screening process. For video analysis, AI-based tools enable the tracking of dynamic changes in both microscopic cellular events and macroscopic animal behaviors. The subtle changes revealed by video analysis could serve as sensitive indicators of adverse outcomes. With AI-based toxicity analysis in its infancy, exciting developments and applications are expected to appear in the years to come.

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions. Traditional tracking methods (e.g., marking each animal with dye or surgically implanting microchips) can be invasive and may have an impact on the social behavior being measured. To overcome these shortcomings, video-based methods for tracking unmarked animals, such as fruit flies and zebrafish, have been developed. However, tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns. In this study, we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network (R-CNN) deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology. The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior. The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.

Enhancing Detection Quality Rate with a Combined HOG and CNN for Real-Time Multiple Object Tracking across Non-Overlapping Multiple Cameras

Multi-object tracking in video surveillance is subjected to illumination variation, blurring, motion, and similarity variations during the identification process in real-world practice. The previously proposed applications have difficulties in learning the appearances and differentiating the objects from sundry detections. They mostly rely heavily on local features and tend to lose vital global structured features such as contour features. This contributes to their inability to accurately detect, classify or distinguish the fooling images. In this paper, we propose a paradigm aimed at eliminating these tracking difficulties by enhancing the detection quality rate through the combination of a convolutional neural network (CNN) and a histogram of oriented gradient (HOG) descriptor. We trained the algorithm with an input of 120 × 32 images size and cleaned and converted them into binary for reducing the numbers of false positives. In testing, we eliminated the background on frames size and applied morphological operations and Laplacian of Gaussian model (LOG) mixture after blobs. The images further underwent feature extraction and computation with the HOG descriptor to simplify the structural information of the objects in the captured video images. We stored the appearance features in an array and passed them into the network (CNN) for further processing. We have applied and evaluated our algorithm for real-time multiple object tracking on various city streets using EPFL multi-camera pedestrian datasets. The experimental results illustrate that our proposed technique improves the detection rate and data associations. Our algorithm outperformed the online state-of-the-art approach by recording the highest in precisions and specificity rates.

Zebrafish automatic monitoring system for conditioning and behavioral analysis

Studies using zebrafish (Danio rerio) in neuro-behavioural research are growing. Measuring fish behavior by computational methods is one of the most efficient ways to avoid human bias in experimental analyses, extending them to various approaches. Sometimes, thorough analyses are difficult to do, as fish can behave unpredictably during an experimental strategy. However, the analyses can be implemented in an automated way, using an online strategy and video processing for a complete assessment of the zebrafish behavior, based on the detection and tracking of fish during an activity. Here, a fully automatic conditioning and detailed analysis of zebrafish behavior is presented. Microcontrolled components were used to control the delivery of visual and sound stimuli, in addition to the concise amounts of food after conditioned stimuli for adult zebrafish groups in a conventional tank. The images were captured and processed for automatic detection of the fish, and the training of the fish was done in two evaluation strategies: simple and complex. In simple conditioning, the zebrafish showed significant responses from the second attempt, learning that the conditioned stimulus was a predictor of food presentation in a specific space of the tank, where the food was dumped. When the fish were subjected to two stimuli for decision-making in the food reward, the zebrafish obtained better responses to red light stimuli in relation to vibration. The behavior change was clear in stimulated fish in relation to the control group, thus, the distances traveled and the speed were greater, while the polarization was lower in stimulated fish. This automated system allows for the conditioning and assessment of zebrafish behavior online, with greater stability in experiments, and in the analysis of the behavior of individual fish or fish schools, including learning and memory studies.

Current Methods to Investigate Nociception and Pain in Zebrafish

Pain is an unpleasant, negative emotion and its debilitating effects are complex to manage. Mammalian models have long dominated research on nociception and pain, but there is increasing evidence for comparable processes in fish. The need to improve existing pain models for drug research and the obligation for 3R refinement of fish procedures facilitated the development of numerous new assays of nociception and pain in fish. The zebrafish is already a well-established animal model in many other research areas like toxicity testing, as model for diseases or regeneration and has great potential in pain research, too. Methods of electrophysiology, molecular biology, analysis of reflexive or non-reflexive behavior and fluorescent imaging are routinely applied but it is the combination of these tools what makes the zebrafish model so powerful. Simultaneously, observing complex behavior in free-swimming larvae, as well as their neuronal activity at the cellular level, opens new avenues for pain research. This review aims to supply a toolbox for researchers by summarizing current methods to study nociception and pain in zebrafish. We identify treatments with the best algogenic potential, be it chemical, thermal or electric stimuli and discuss options of analgesia to counter effects of nociception and pain by opioids, non-steroidal anti-inflammatory drugs (NSAIDs) or local anesthetics. In addition, we critically evaluate these practices, identify gaps of knowledge and outline potential future developments.

Zebrafish tracking using YOLOv2 and Kalman filter

Fish show rapid movements in various behavioral activities or associated with the presence of food. However, in periods of rapid movement, the rate at which occlusion occurs among the fish is quite high, causing inconsistency in the detection and tracking of fish, hindering the fish's identity and behavioral trajectory over a long period of time. Although some algorithms have been proposed to solve these problems, most of their applications were made in groups of fish that swim in shallow water and calm behavior, with few sudden movements. To solve these problems, a convolutional network of object recognition, YOLOv2, was used to delimit the region of the fish heads to optimize individual fish detection. In the tracking phase, the Kalman filter was used to estimate the best state of the fish's head position in each frame and, subsequently, the trajectories of each fish were connected among the frames. The results of the algorithm show adequate performances in the trajectories of groups of zebrafish that exhibited rapid movements.

An assistive computer vision tool to automatically detect changes in fish behavior in response to ambient odor

The analysis of fish behavior in response to odor stimulation is a crucial component of the general study of cross-modal sensory integration in vertebrates. In zebrafish, the centrifugal pathway runs between the olfactory bulb and the neural retina, originating at the terminalis neuron in the olfactory bulb. Any changes in the ambient odor of a fish's environment warrant a change in visual sensitivity and can trigger mating-like behavior in males due to increased GnRH signaling in the terminalis neuron. Behavioral experiments to study this phenomenon are commonly conducted in a controlled environment where a video of the fish is recorded over time before and after the application of chemicals to the water. Given the subtleties of behavioral change, trained biologists are currently required to annotate such videos as part of a study. This process of manually analyzing the videos is time-consuming, requires multiple experts to avoid human error/bias and cannot be easily crowdsourced on the Internet. Machine learning algorithms from computer vision, on the other hand, have proven to be effective for video annotation tasks because they are fast, accurate, and, if designed properly, can be less biased than humans. In this work, we propose to automate the entire process of analyzing videos of behavior changes in zebrafish by using tools from computer vision, relying on minimal expert supervision. The overall objective of this work is to create a generalized tool to predict animal behaviors from videos using state-of-the-art deep learning models, with the dual goal of advancing understanding in biology and engineering a more robust and powerful artificial information processing system for biologists.

The zebrafish subcortical social brain as a model for studying social behavior disorders

Social behaviors are essential for the survival and reproduction of social species. Many, if not most, neuropsychiatric disorders in humans are either associated with underlying social deficits or are accompanied by social dysfunctions. Traditionally, rodent models have been used to model these behavioral impairments. However, rodent assays are often difficult to scale up and adapt to high-throughput formats, which severely limits their use for systems-level science. In recent years, an increasing number of studies have used zebrafish (Danio rerio) as a model system to study social behavior. These studies have demonstrated clear potential in overcoming some of the limitations of rodent models. In this Review, we explore the evolutionary conservation of a subcortical social brain between teleosts and mammals as the biological basis for using zebrafish to model human social behavior disorders, while summarizing relevant experimental tools and assays. We then discuss the recent advances gleaned from zebrafish social behavior assays, the applications of these assays to studying related disorders, and the opportunities and challenges that lie ahead.