Automatic Classification of Cichlid Behaviors Using 3D Convolutional Residual Networks

SMART-BARN: Scalable multimodal arena for real-time tracking behavior of animals in large numbers

The SMART-BARN (scalable multimodal arena for real-time tracking behavior of animals in large numbers) achieves fast, robust acquisition of movement, behavior, communication, and interactions of animals in groups, within a large (14.7 meters by 6.6 meters by 3.8 meters), three-dimensional environment using multiple information channels. Behavior is measured from a wide range of taxa (insects, birds, mammals, etc.) and body size (from moths to humans) simultaneously. This system integrates multiple, concurrent measurement techniques including submillimeter precision and high-speed (300 hertz) motion capture, acoustic recording and localization, automated behavioral recognition (computer vision), and remote computer-controlled interactive units (e.g., automated feeders and animal-borne devices). The data streams are available in real time allowing highly controlled and behavior-dependent closed-loop experiments, while producing comprehensive datasets for offline analysis. The diverse capabilities of SMART-BARN are demonstrated through three challenging avian case studies, while highlighting its broad applicability to the fine-scale analysis of collective animal behavior across species.

Cellular profiling of a recently-evolved social behavior in cichlid fishes

Social behaviors are diverse in nature, but it is unclear how conserved genes, brain regions, and cell populations generate this diversity. Here we investigate bower-building, a recently-evolved social behavior in cichlid fishes. We use single nucleus RNA-sequencing in 38 individuals to show signatures of recent behavior in specific neuronal populations, and building-associated rebalancing of neuronal proportions in the putative homolog of the hippocampal formation. Using comparative genomics across 27 species, we trace bower-associated genome evolution to a subpopulation of glia lining the dorsal telencephalon. We show evidence that building-associated neural activity and a departure from quiescence in this glial subpopulation together regulate hippocampal-like neuronal rebalancing. Our work links behavior-associated genomic variation to specific brain cell types and their functions, and suggests a social behavior has evolved through changes in glia.

Indirect Genetic Effects: A Cross-disciplinary Perspective on Empirical Studies

Indirect genetic effects (IGE) occur when an individual's phenotype is influenced by genetic variation in conspecifics. Opportunities for IGE are ubiquitous, and, when present, IGE have profound implications for behavioral, evolutionary, agricultural, and biomedical genetics. Despite their importance, the empirical study of IGE lags behind the development of theory. In large part, this lag can be attributed to the fact that measuring IGE, and deconvoluting them from the direct genetic effects of an individual's own genotype, is subject to many potential pitfalls. In this Perspective, we describe current challenges that empiricists across all disciplines will encounter in measuring and understanding IGE. Using ideas and examples spanning evolutionary, agricultural, and biomedical genetics, we also describe potential solutions to these challenges, focusing on opportunities provided by recent advances in genomic, monitoring, and phenotyping technologies. We hope that this cross-disciplinary assessment will advance the goal of understanding the pervasive effects of conspecific interactions in biology.

Automatic Recognition of Fish Behavior with a Fusion of RGB and Optical Flow Data Based on Deep Learning

Simple Summary

Animal behaviors are critical for survival, which is expressed over a long period of time. The emergence of computer vision and deep learning technologies creates new possibilities for understanding the biological basis of these behaviors and accurately quantifying behaviors, which contributes to attaining high production efficiency and precise management in precision farming. Here, we demonstrate that a dual-stream 3D convolutional neural network with RGB and optical flow video clips as input can be used to classify behavior states of fish schools. The FlowNet2 based on deep learning, combined with a 3D convolutional neural network, was first applied to identify fish behavior. Additionally, the results indicate that the proposed non-invasive recognition method can quickly, accurately, and automatically identify fish behaviors across hundreds of hours of video.

Abstract

The rapid and precise recognition of fish behavior is critical in perceiving health and welfare by allowing farmers to make informed management decisions on recirculating aquaculture systems while reducing labor. The conventional recognition methods are to obtain movement information by implanting sensors on the skin or in the body of the fish, which can affect the normal behavior and welfare of the fish. We present a novel nondestructive method with spatiotemporal and motion information based on deep learning for real-time recognition of fish schools’ behavior. In this work, a dual-stream 3D convolutional neural network (DSC3D) was proposed for the recognition of five behavior states of fish schools, including feeding, hypoxia, hypothermia, frightening and normal behavior. This DSC3D combines spatiotemporal features and motion features by using FlowNet2 and 3D convolutional neural networks and shows significant results suitable for industrial applications in automatic monitoring of fish behavior, with an average accuracy rate of 95.79%. The model evaluation results on the test dataset further demonstrated that our proposed method could be used as an effective tool for the intelligent perception of fish health status.

Automated measurement of long-term bower behaviors in Lake Malawi cichlids using depth sensing and action recognition

In the wild, behaviors are often expressed over long time periods in complex and dynamic environments, and many behaviors include direct interaction with the environment itself. However, measuring behavior in naturalistic settings is difficult, and this has limited progress in understanding the mechanisms underlying many naturally evolved behaviors that are critical for survival and reproduction. Here we describe an automated system for measuring long-term bower construction behaviors in Lake Malawi cichlid fishes, in which males use their mouths to sculpt sand into large species-specific structures for courtship and mating. We integrate two orthogonal methods, depth sensing and action recognition, to simultaneously track the developing bower structure and the thousands of individual sand manipulation behaviors performed throughout construction. By registering these two data streams, we show that behaviors can be topographically mapped onto a dynamic 3D sand surface through time. The system runs reliably in multiple species, across many aquariums simultaneously, and for up to weeks at a time. Using this system, we show strong differences in construction behavior and bower form that reflect species differences in nature, and we gain new insights into spatial, temporal, social dimensions of bower construction, feeding, and quivering behaviors. Taken together, our work highlights how low-cost tools can automatically quantify behavior in naturalistic and social environments over long timescales in the lab.