Dynamic time warping and sparse representation classification for birdsong phrase classification using limited training data

A Novel Bird Sound Recognition Method Based on Multifeature Fusion and a Transformer Encoder

Birds play a vital role in the study of ecosystems and biodiversity. Accurate bird identification helps monitor biodiversity, understand the functions of ecosystems, and develop effective conservation strategies. However, previous bird sound recognition methods often relied on single features and overlooked the spatial information associated with these features, leading to low accuracy. Recognizing this gap, the present study proposed a bird sound recognition method that employs multiple convolutional neural-based networks and a transformer encoder to provide a reliable solution for identifying and classifying birds based on their unique sounds. We manually extracted various acoustic features as model inputs, and feature fusion was applied to obtain the final set of feature vectors. Feature fusion combines the deep features extracted by various networks, resulting in a more comprehensive feature set, thereby improving recognition accuracy. The multiple integrated acoustic features, such as mel frequency cepstral coefficients (MFCC), chroma features (Chroma) and Tonnetz features, were encoded by a transformer encoder. The transformer encoder effectively extracted the positional relationships between bird sound features, resulting in enhanced recognition accuracy. The experimental results demonstrated the exceptional performance of our method with an accuracy of 97.99%, a recall of 96.14%, an F1 score of 96.88% and a precision of 97.97% on the Birdsdata dataset. Furthermore, our method achieved an accuracy of 93.18%, a recall of 92.43%, an F1 score of 93.14% and a precision of 93.25% on the Cornell Bird Challenge 2020 (CBC) dataset.

Birdsong classification based on ensemble multi-scale convolutional neural network

With the intensification of ecosystem damage, birds have become the symbolic species of the ecosystem. Ornithology with interdisciplinary technical research plays a great significance for protecting birds and evaluating ecosystem quality. Deep learning shows great progress for birdsongs recognition. However, as the number of network layers increases in traditional CNN, semantic information gradually becomes richer and detailed information disappears. Secondly, the global information carried by the entire input may be lost in convolution, pooling, or other operations, and these problems will weaken the performance of classification. In order to solve such problems, based on the feature spectrogram from the wavelet transform for the birdsongs, this paper explored the multi-scale convolution neural network (MSCNN) and proposed an ensemble multi-scale convolution neural network (EMSCNN) classification framework. The experiments compared the MSCNN and EMSCNN models with other CNN models including LeNet, VGG16, ResNet101, MobileNetV2, EfficientNetB7, Darknet53 and SPP-net. The results showed that the MSCNN model achieved an accuracy of 89.61%, and EMSCNN achieved an accuracy of 91.49%. In the experiments on the recognition of 30 species of birds, our models effectively improved the classification effect with high stability and efficiency, indicating that the models have better generalization ability and are suitable for birdsongs species recognition. It provides methodological and technical scheme reference for bird classification research.

Computational geometric tools for quantitative comparison of locomotory behavior

A fundamental challenge for behavioral neuroscientists is to accurately quantify (dis)similarities in animal behavior without excluding inherent variability present between individuals. We explored two new applications of curve and shape alignment techniques to address this issue. As a proof-of-concept we applied these methods to compare normal or alarmed behavior in pairs of medaka (Oryzias latipes). The curve alignment method we call Behavioral Distortion Distance (BDD) revealed that alarmed fish display less predictable swimming over time, even if individuals incorporate the same action patterns like immobility, sudden changes in swimming trajectory, or changing their position in the water column. The Conformal Spatiotemporal Distance (CSD) technique on the other hand revealed that, in spite of the unpredictability, alarmed individuals exhibit lower variability in overall swim patterns, possibly accounting for the widely held notion of "stereotypy" in alarm responses. More generally, we propose that these new applications of established computational geometric techniques are useful in combination to represent, compare, and quantify complex behaviors consisting of common action patterns that differ in duration, sequence, or frequency.

Episodic Ultradian Events-Ultradian Rhythms

In the fast lane of chronobiology, ultradian events are short-term rhythms that have been observed since the beginning of modern biology and were quantified about a century ago. They are ubiquitous in all biological systems and found in all organisms, from unicellular organisms to mammals, and from single cells to complex biological functions in multicellular animals. Since these events are aperiodic and last for a few minutes to a few hours, they are better classified as episodic ultradian events (EUEs). Their origin is unclear. However, they could have a molecular basis and could be controlled by hormonal inputs-in vertebrates, they originate from the activity of the central nervous system. EUEs are receiving increasing attention but their aperiodic nature requires specific sampling and analytic tools. While longer scale rhythms are adaptations to predictable changes in the environment, in theory, EUEs could contribute to adaptation by preparing organisms and biological functions for unpredictability.

Evidence for a Coupled Oscillator Model of Endocrine Ultradian Rhythms

Whereas long-period temporal structures in endocrine dynamics have been well studied, endocrine rhythms on the scale of hours are relatively unexplored. The study of these ultradian rhythms (URs) has remained nascent, in part, because a theoretical framework unifying ultradian patterns across systems has not been established. The present overview proposes a conceptual coupled oscillator network model of URs in which oscillating hormonal outputs, or nodes, are connected by edges representing the strength of node-node coupling. We propose that variable-strength coupling exists both within and across classic hormonal axes. Because coupled oscillators synchronize, such a model implies that changes across hormonal systems could be inferred by surveying accessible nodes in the network. This implication would at once simplify the study of URs and open new avenues of exploration into conditions affecting coupling. In support of this proposed framework, we review mammalian evidence for (1) URs of the gut-brain axis and the hypothalamo-pituitary-thyroid, -adrenal, and -gonadal axes, (2) UR coupling within and across these axes; and (3) the relation of these URs to body temperature. URs across these systems exhibit behavior broadly consistent with a coupled oscillator network, maintaining both consistent URs and coupling within and across axes. This model may aid the exploration of mammalian physiology at high temporal resolution and improve the understanding of endocrine system dynamics within individuals.

Tensorial dynamic time warping with articulation index representation for efficient audio-template learning

Audio classification techniques often depend on the availability of a large labeled training dataset for successful performance. However, in many application domains of audio classification (e.g., wildlife monitoring), obtaining labeled data is still a costly and laborious process. Motivated by this observation, a technique is proposed to efficiently learn a clean template from a few labeled, but likely corrupted (by noise and interferences), data samples. This learning can be done efficiently via tensorial dynamic time warping on the articulation index-based time-frequency representations of audio data. The learned template can then be used in audio classification following the standard template-based approach. Experimental results show that the proposed approach outperforms both (1) the recurrent neural network approach and (2) the state-of-the-art in the template-based approach on a wildlife detection application with few training samples.

Active learning for bird sound classification via a kernel-based extreme learning machine

In recent years, research fields, including ecology, bioacoustics, signal processing, and machine learning, have made bird sound recognition a part of their focus. This has led to significant advancements within the field of ornithology, such as improved understanding of evolution, local biodiversity, mating rituals, and even the implications and realities associated to climate change. The volume of unlabeled bird sound data is now overwhelming, and comparatively little exploration is being made into methods for how best to handle them. In this study, two active learning (AL) methods are proposed, sparse-instance-based active learning (SI-AL), and least-confidence-score-based active learning (LCS-AL), both effectively reducing the need for expert human annotation. To both of these AL paradigms, a kernel-based extreme learning machine (KELM) is then integrated, and a comparison is made to the conventional support vector machine (SVM). Experimental results demonstrate that, when the classifier capacity is improved from an unweighted average recall of 60%-80%, KELM can outperform SVM even when a limited proportion of human annotations are used from the pool of data in both cases of SI-AL (minimum 34.5% vs minimum 59.0%) and LCS-AL (minimum 17.3% vs minimum 28.4%).

A robust automatic birdsong phrase classification: A template-based approach

Automatic phrase detection systems of bird sounds are useful in several applications as they reduce the need for manual annotations. However, birdphrase detection is challenging due to limited training data and background noise. Limited data occur because of limited recordings or the existence of rare phrases. Background noise interference occurs because of the intrinsic nature of the recording environment such as wind or other animals. This paper presents a different approach to birdsong phrase classification using template-based techniques suitable even for limited training data and noisy environments. The algorithm utilizes dynamic time-warping (DTW) and prominent (high-energy) time-frequency regions of training spectrograms to derive templates. The performance of the proposed algorithm is compared with the traditional DTW and hidden Markov models (HMMs) methods under several training and test conditions. DTW works well when the data are limited, while HMMs do better when more data are available, yet they both suffer when the background noise is severe. The proposed algorithm outperforms DTW and HMMs in most training and testing conditions, usually with a high margin when the background noise level is high. The innovation of this work is that the proposed algorithm is robust to both limited training data and background noise.

Complexity, Predictability and Time Homogeneity of Syntax in the Songs of Cassin's Vireo (Vireo cassinii)

Many species of animals deliver vocalizations in sequences presumed to be governed by internal rules, though the nature and complexity of these syntactical rules have been investigated in relatively few species. Here I present an investigation into the song syntax of fourteen male Cassin's Vireos (Vireo cassinii), a species whose song sequences are highly temporally structured. I compare their song sequences to three candidate models of varying levels of complexity-zero-order, first-order and second-order Markov models-and employ novel methods to interpolate between these three models. A variety of analyses, including sequence simulations, Fisher's exact tests, and model likelihood analyses, showed that the songs of this species are too complex to be described by a zero-order or first-order Markov model. The model that best fit the data was intermediate in complexity between a first- and second-order model, though I also present evidence that some transition probabilities are conditioned on up to three preceding phrases. In addition, sequences were shown to be predictable with more than 54% accuracy overall, and predictability was positively correlated with the rate of song delivery. An assessment of the time homogeneity of syntax showed that transition probabilities between phrase types are largely stable over time, but that there was some evidence for modest changes in syntax within and between breeding seasons, a finding that I interpret to represent changes in breeding stage and social context rather than irreversible, secular shifts in syntax over time. These findings constitute a valuable addition to our understanding of bird song syntax in free-living birds, and will contribute to future attempts to understand the evolutionary importance of bird song syntax in avian communication.